linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nazar Mokrynskyi <nazar@mokrynskyi.com>
To: linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: Another btrfs corruption (unable to find ref byte, kernel 5.0-rc4)
Date: Thu, 7 Feb 2019 06:07:52 +0200	[thread overview]
Message-ID: <5b0d2e94-6e4e-aecd-3eda-459c4a96bb13@mokrynskyi.com> (raw)

NOTE: I don't need asistance with data recovery, everything was restored shortly from fully automated backups, I only hope this information is useful for developers in some way.

So my primary BTRFS filesystem corrupted itself again.

Software:
Kernel 5.0-rc4
btrfs-progs v4.20.1
Ubuntu 19.04 (development branch)

It is running on NVMe SSD on top of full-disk LUKS with BFQ scheduler.

Mounting options (multiple subvolumes like this):
compress=lzo,noatime,ssd,subvol=/root

Also I'm using automated snapshots a lot and there are a few directories with CoW disabled if that matters.

Filesystem was created with Ubuntu 18.10 Live USB (kernel 4.18.0-10-generic, btrfs-progs v4.16.1).

It corrupted itself and remounted in read-only state, so I had to hard reset it and from Ubuntu 18.10 Live USB run scrub:

> [   49.674792] Btrfs loaded, crc32c=crc32c-intel
> [   49.679962] BTRFS: device fsid 5170aca4-061a-4c6c-ab00-bd7fc8ae6030 devid 1 transid 178701 /dev/dm-0
> [   52.199834] BTRFS info (device dm-0): disk space caching is enabled
> [   52.199839] BTRFS info (device dm-0): has skinny extents
> [   52.239346] BTRFS info (device dm-0): enabling ssd optimizations
> [   82.909833] WARNING: CPU: 14 PID: 6082 at fs/btrfs/extent-tree.c:6944 __btrfs_free_extent.isra.72+0x751/0xac0 [btrfs]
> [   82.909834] Modules linked in: btrfs zstd_compress libcrc32c xor raid6_pq dm_crypt algif_skcipher af_alg intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_codec_hdmi snd_hda_intel kvm snd_usb_audio snd_hda_codec snd_usbmidi_lib snd_hda_core snd_hwdep snd_pcm irqbypass crct10dif_pclmul snd_seq_midi snd_seq_midi_event crc32_pclmul snd_rawmidi ghash_clmulni_intel uvcvideo pcbc snd_seq videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common snd_seq_device snd_timer aesni_intel videodev snd aes_x86_64 crypto_simd cdc_acm soundcore media input_leds joydev cryptd mei_me glue_helper intel_wmi_thunderbolt intel_cstate mei intel_rapl_perf acpi_pad mac_hid sch_fq_codel parport_pc ppdev lp parport ip_tables x_tables autofs4 overlay nls_iso8859_1 dm_mirror dm_region_hash dm_log
> [   82.909854]  hid_generic usbhid hid uas usb_storage amdkfd amd_iommu_v2 amdgpu nouveau chash gpu_sched ttm mxm_wmi drm_kms_helper syscopyarea sysfillrect sysimgblt nvme fb_sys_fops igb e1000e atlantic drm ahci dca i2c_algo_bit nvme_core libahci wmi video
> [   82.909864] CPU: 14 PID: 6082 Comm: btrfs-cleaner Not tainted 4.18.0-10-generic #11-Ubuntu
> [   82.909864] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z370 Professional Gaming i7, BIOS P3.40 11/08/2018
> [   82.909871] RIP: 0010:__btrfs_free_extent.isra.72+0x751/0xac0 [btrfs]
> [   82.909871] Code: ff 75 18 ff 75 10 e8 7e a6 ff ff c6 85 6c ff ff ff 00 41 89 c5 58 5a 45 85 ed 0f 84 b9 f9 ff ff 41 83 fd fe 0f 85 6e fc ff ff <0f> 0b 49 8b 3c 24 e8 44 c8 00 00 ff 75 18 4c 8b 4d 10 4d 89 f8 48
> [   82.909885] RSP: 0018:ffffa71bc6833bf8 EFLAGS: 00010246
> [   82.909885] RAX: 00000000000c81bc RBX: 0000000f3e55e000 RCX: 0000000000000000
> [   82.909886] RDX: 000000000017b000 RSI: 0000000000000000 RDI: ffff89cfd82d4f50
> [   82.909886] RBP: ffffa71bc6833ca0 R08: ffffa71bc6833b14 R09: 0000000000000000
> [   82.909887] R10: 0000000000000102 R11: 0000000000000000 R12: ffff89cfd8338af0
> [   82.909887] R13: 00000000fffffffe R14: 0000000e8e9f8000 R15: 000000000000010b
> [   82.909888] FS:  0000000000000000(0000) GS:ffff89d09e580000(0000) knlGS:0000000000000000
> [   82.909888] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   82.909889] CR2: 0000564398e70760 CR3: 000000076ce0a002 CR4: 00000000003606e0
> [   82.909889] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   82.909890] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   82.909890] Call Trace:
> [   82.909898]  __btrfs_run_delayed_refs+0x20e/0x1010 [btrfs]
> [   82.909904]  ? add_pinned_bytes+0x67/0x70 [btrfs]
> [   82.909909]  ? btrfs_free_tree_block+0x167/0x2d0 [btrfs]
> [   82.909916]  btrfs_run_delayed_refs+0x80/0x190 [btrfs]
> [   82.909923]  btrfs_should_end_transaction+0x47/0x60 [btrfs]
> [   82.909928]  btrfs_drop_snapshot+0x3d1/0x800 [btrfs]
> [   82.909936]  btrfs_clean_one_deleted_snapshot+0xbb/0xf0 [btrfs]
> [   82.909942]  cleaner_kthread+0x136/0x160 [btrfs]
> [   82.909944]  kthread+0x120/0x140
> [   82.909950]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [   82.909951]  ? kthread_bind+0x40/0x40
> [   82.909953]  ret_from_fork+0x35/0x40
> [   82.909953] ---[ end trace 32964933c87d1d27 ]---
> [   82.909955] BTRFS info (device dm-0): leaf 656310272 gen 178703 total ptrs 117 free space 4483 owner 2
> [   82.909956]     item 0 key (65470308352 168 4096) itemoff 16233 itemsize 50
> [   82.909956]         extent refs 2 gen 170784 flags 1
> [   82.909957]         ref#0: shared data backref parent 62803951616 count 1
> [   82.909957]         ref#1: shared data backref parent 62522392576 count 1
> [   82.909958]     item 1 key (65470312448 168 4096) itemoff 16014 itemsize 219
> [   82.909959]         extent refs 15 gen 170784 flags 1
> [   82.909959]         ref#0: shared data backref parent 63455215616 count 1
> [   82.909960]         ref#1: shared data backref parent 63314100224 count 1
> [   82.909960]         ref#2: shared data backref parent 63224987648 count 1
> [   82.909961]         ref#3: shared data backref parent 63096176640 count 1
> [   82.909961]         ref#4: shared data backref parent 63008653312 count 1
> [   82.909962]         ref#5: shared data backref parent 62889902080 count 1
> [   82.909962]         ref#6: shared data backref parent 62803951616 count 1
> [   82.909963]         ref#7: shared data backref parent 62672060416 count 1
> [   82.909963]         ref#8: shared data backref parent 62522392576 count 1
> [   82.909964]         ref#9: shared data backref parent 40772386816 count 1
> [   82.909964]         ref#10: shared data backref parent 40439906304 count 1
> [   82.909965]         ref#11: shared data backref parent 40155185152 count 1
> [   82.909965]         ref#12: shared data backref parent 28943499264 count 1
> [   82.909966]         ref#13: shared data backref parent 28263088128 count 1
> [   82.909966]         ref#14: shared data backref parent 242253824 count 1
> [   82.909967]     item 2 key (65470320640 168 4096) itemoff 15886 itemsize 128
> [   82.909968]         extent refs 8 gen 170784 flags 1
> [   82.909968]         ref#0: shared data backref parent 63455248384 count 1
> [   82.909968]         ref#1: shared data backref parent 63206113280 count 1
> [   82.909969]         ref#2: shared data backref parent 63008669696 count 1
> [   82.909969]         ref#3: shared data backref parent 62761549824 count 1
> [   82.909970]         ref#4: shared data backref parent 62522408960 count 1
> [   82.909970]         ref#5: shared data backref parent 28943548416 count 1
> [   82.909971]         ref#6: shared data backref parent 28263038976 count 1
> [   82.909971]         ref#7: shared data backref parent 241270784 count 1
> [   82.909972]     item 3 key (65470324736 168 35184372092928) itemoff 15719 itemsize 167
> [   82.909973]         extent refs 11 gen 170787 flags 1
> [   82.909973]         ref#0: shared data backref parent 63455199232 count 1
> [   82.909973]         ref#1: shared data backref parent 63225004032 count 1
> [   82.909974]         ref#2: shared data backref parent 63008636928 count 1
> [   82.909974]         ref#3: shared data backref parent 62803918848 count 1
> [   82.909975]         ref#4: shared data backref parent 62522359808 count 1
> [   82.909975]         ref#5: shared data backref parent 40772304896 count 1
> [   82.909976]         ref#6: shared data backref parent 40440938496 count 1
> [   82.909976]         ref#7: shared data backref parent 40156839936 count 1
> [   82.909977]         ref#8: shared data backref parent 28943482880 count 1
> [   82.909977]         ref#9: shared data backref parent 28263006208 count 1
> [   82.909978]         ref#10: shared data backref parent 242237440 count 1
> [   82.909979]     item 4 key (65470328832 168 4096) itemoff 15656 itemsize 63
> [   82.909979]         extent refs 3 gen 170787 flags 1
> [   82.909979]         ref#0: shared data backref parent 63008636928 count 1
> [   82.909980]         ref#1: shared data backref parent 62803918848 count 1
> [   82.909980]         ref#2: shared data backref parent 62522359808 count 1
> [   82.909981]     item 5 key (65470332928 168 4096) itemoff 15567 itemsize 89
> [   82.909982]         extent refs 5 gen 170787 flags 1
> [   82.909982]         ref#0: shared data backref parent 63455199232 count 1
> [   82.909982]         ref#1: shared data backref parent 63225004032 count 1
> [   82.909983]         ref#2: shared data backref parent 63008636928 count 1
> [   82.909983]         ref#3: shared data backref parent 62803918848 count 1
> [   82.909984]         ref#4: shared data backref parent 62522359808 count 1
> [   82.909985]     item 6 key (65470337024 168 4096) itemoff 15283 itemsize 284
> [   82.909985]         extent refs 20 gen 170787 flags 1
> [   82.909985]         ref#0: shared data backref parent 63517229056 count 1
> [   82.909986]         ref#1: shared data backref parent 63455215616 count 1
> [   82.909986]         ref#2: shared data backref parent 63314100224 count 1
> [   82.909987]         ref#3: shared data backref parent 63224987648 count 1
> [   82.909987]         ref#4: shared data backref parent 63096176640 count 1
> [   82.909988]         ref#5: shared data backref parent 63008653312 count 1
> [   82.909988]         ref#6: shared data backref parent 62889902080 count 1
> [   82.909989]         ref#7: shared data backref parent 62803951616 count 1
> [   82.909989]         ref#8: shared data backref parent 62672060416 count 1
> [   82.909990]         ref#9: shared data backref parent 62522392576 count 1
> [   82.909990]         ref#10: shared data backref parent 40772386816 count 1
> [   82.909991]         ref#11: shared data backref parent 40439906304 count 1
> [   82.909991]         ref#12: shared data backref parent 40432418816 count 1
> [   82.909992]         ref#13: shared data backref parent 40155185152 count 1
> [   82.909992]         ref#14: shared data backref parent 40109506560 count 1
> [   82.909993]         ref#15: shared data backref parent 28943499264 count 1
> [   82.909993]         ref#16: shared data backref parent 28794585088 count 1
> [   82.909994]         ref#17: shared data backref parent 28263088128 count 1
> [   82.909994]         ref#18: shared data backref parent 387235840 count 1
> [   82.909995]         ref#19: shared data backref parent 242253824 count 1
> [   82.909996]     item 7 key (65470341120 168 4096) itemoff 15207 itemsize 76
> [   82.909996]         extent refs 4 gen 170787 flags 1
> [   82.909996]         ref#0: shared data backref parent 63224987648 count 1
> [   82.909997]         ref#1: shared data backref parent 63008653312 count 1
> [   82.909997]         ref#2: shared data backref parent 62803951616 count 1
> [   82.909998]         ref#3: shared data backref parent 62522392576 count 1
> [   82.909999]     item 8 key (65470345216 168 4096) itemoff 15144 itemsize 63
> [   82.909999]         extent refs 3 gen 170787 flags 1
> [   82.909999]         ref#0: shared data backref parent 63008653312 count 1
> [   82.910000]         ref#1: shared data backref parent 62803951616 count 1
> [   82.910000]         ref#2: shared data backref parent 62522392576 count 1
> [   82.910001]     item 9 key (65470349312 168 4096) itemoff 15055 itemsize 89
> [   82.910001]         extent refs 5 gen 170787 flags 1
> [   82.910002]         ref#0: shared data backref parent 63455215616 count 1
> [   82.910002]         ref#1: shared data backref parent 63224987648 count 1
> [   82.910003]         ref#2: shared data backref parent 63008653312 count 1
> [   82.910003]         ref#3: shared data backref parent 62803951616 count 1
> [   82.910004]         ref#4: shared data backref parent 62522392576 count 1
> [   82.910005]     item 10 key (65470353408 168 4096) itemoff 14771 itemsize 284
> [   82.910005]         extent refs 20 gen 170787 flags 1
> [   82.910005]         ref#0: shared data backref parent 63517229056 count 1
> [   82.910006]         ref#1: shared data backref parent 63455215616 count 1
> [   82.910006]         ref#2: shared data backref parent 63314100224 count 1
> [   82.910007]         ref#3: shared data backref parent 63224987648 count 1
> [   82.910007]         ref#4: shared data backref parent 63096176640 count 1
> [   82.910008]         ref#5: shared data backref parent 63008653312 count 1
> [   82.910008]         ref#6: shared data backref parent 62889902080 count 1
> [   82.910009]         ref#7: shared data backref parent 62803951616 count 1
> [   82.910009]         ref#8: shared data backref parent 62672060416 count 1
> [   82.910010]         ref#9: shared data backref parent 62522392576 count 1
> [   82.910010]         ref#10: shared data backref parent 40772386816 count 1
> [   82.910011]         ref#11: shared data backref parent 40439906304 count 1
> [   82.910011]         ref#12: shared data backref parent 40432418816 count 1
> [   82.910012]         ref#13: shared data backref parent 40155185152 count 1
> [   82.910012]         ref#14: shared data backref parent 40109506560 count 1
> [   82.910013]         ref#15: shared data backref parent 28943499264 count 1
> [   82.910013]         ref#16: shared data backref parent 28794585088 count 1
> [   82.910014]         ref#17: shared data backref parent 28263088128 count 1
> [   82.910014]         ref#18: shared data backref parent 387235840 count 1
> [   82.910015]         ref#19: shared data backref parent 242253824 count 1
> [   82.910016]     item 11 key (65470357504 168 4096) itemoff 14708 itemsize 63
> [   82.910016]         extent refs 3 gen 170788 flags 1
> [   82.910016]         ref#0: shared data backref parent 63224954880 count 1
> [   82.910017]         ref#1: shared data backref parent 63008587776 count 1
> [   82.910017]         ref#2: shared data backref parent 62803886080 count 1
> [   82.910018]     item 12 key (65470365696 168 4096) itemoff 14096 itemsize 612
> [   82.910018]         extent refs 44 gen 156088 flags 1
> [   82.910019]         ref#0: extent data backref root 267 objectid 819644 offset 4124672 count 1
> [   82.910019]         ref#1: shared data backref parent 63556435968 count 1
> [   82.910020]         ref#2: shared data backref parent 63553519616 count 1
> [   82.910020]         ref#3: shared data backref parent 63504007168 count 1
> [   82.910021]         ref#4: shared data backref parent 63490015232 count 1
> [   82.910021]         ref#5: shared data backref parent 63457132544 count 1
> [   82.910022]         ref#6: shared data backref parent 63289114624 count 1
> [   82.910022]         ref#7: shared data backref parent 63258591232 count 1
> [   82.910023]         ref#8: shared data backref parent 63182405632 count 1
> [   82.910023]         ref#9: shared data backref parent 63006113792 count 1
> [   82.910024]         ref#10: shared data backref parent 62973706240 count 1
> [   82.910024]         ref#11: shared data backref parent 62931189760 count 1
> [   82.910025]         ref#12: shared data backref parent 62925520896 count 1
> [   82.910025]         ref#13: shared data backref parent 62889951232 count 1
> [   82.910026]         ref#14: shared data backref parent 62828412928 count 1
> [   82.910027]         ref#15: shared data backref parent 62796120064 count 1
> [   82.910027]         ref#16: shared data backref parent 62791139328 count 1
> [   82.910028]         ref#17: shared data backref parent 62740938752 count 1
> [   82.910028]         ref#18: shared data backref parent 62698946560 count 1
> [   82.910029]         ref#19: shared data backref parent 62627315712 count 1
> [   82.910029]         ref#20: shared data backref parent 62579228672 count 1
> [   82.910030]         ref#21: shared data backref parent 62503010304 count 1
> [   82.910030]         ref#22: shared data backref parent 40803909632 count 1
> [   82.910031]         ref#23: shared data backref parent 40800124928 count 1
> [   82.910031]         ref#24: shared data backref parent 40772042752 count 1
> [   82.910032]         ref#25: shared data backref parent 40713715712 count 1
> [   82.910032]         ref#26: shared data backref parent 40697331712 count 1
> [   82.910033]         ref#27: shared data backref parent 40690958336 count 1
> [   82.910033]         ref#28: shared data backref parent 40615264256 count 1
> [   82.910034]         ref#29: shared data backref parent 40514273280 count 1
> [   82.910034]         ref#30: shared data backref parent 40416378880 count 1
> [   82.910035]         ref#31: shared data backref parent 40315781120 count 1
> [   82.910035]         ref#32: shared data backref parent 40260599808 count 1
> [   82.910036]         ref#33: shared data backref parent 40102936576 count 1
> [   82.910036]         ref#34: shared data backref parent 39983742976 count 1
> [   82.910037]         ref#35: shared data backref parent 39929085952 count 1
> [   82.910037]         ref#36: shared data backref parent 39853129728 count 1
> [   82.910038]         ref#37: shared data backref parent 28927901696 count 1
> [   82.910038]         ref#38: shared data backref parent 28090925056 count 1
> [   82.910039]         ref#39: shared data backref parent 389513216 count 1
> [   82.910039]         ref#40: shared data backref parent 387301376 count 1
> [   82.910040]         ref#41: shared data backref parent 321716224 count 1
> [   82.910040]         ref#42: shared data backref parent 215941120 count 1
> [   82.910041]         ref#43: shared data backref parent 41533440 count 1
> [   82.910042]     item 13 key (65470369792 168 4096) itemoff 13916 itemsize 180
> [   82.910042]         extent refs 12 gen 170792 flags 1
> [   82.910042]         ref#0: shared data backref parent 63455215616 count 1
> [   82.910043]         ref#1: shared data backref parent 63224987648 count 1
> [   82.910043]         ref#2: shared data backref parent 63008653312 count 1
> [   82.910044]         ref#3: shared data backref parent 62803951616 count 1
> [   82.910044]         ref#4: shared data backref parent 62672060416 count 1
> [   82.910045]         ref#5: shared data backref parent 62522392576 count 1
> [   82.910045]         ref#6: shared data backref parent 40772386816 count 1
> [   82.910046]         ref#7: shared data backref parent 40439906304 count 1
> [   82.910046]         ref#8: shared data backref parent 40155185152 count 1
> [   82.910047]         ref#9: shared data backref parent 28943499264 count 1
> [   82.910047]         ref#10: shared data backref parent 28263088128 count 1
> [   82.910048]         ref#11: shared data backref parent 242253824 count 1
> [   82.910049]     item 14 key (65470373888 168 4096) itemoff 13502 itemsize 414
> [   82.910049]         extent refs 30 gen 170793 flags 1
> [   82.910049]         ref#0: shared data backref parent 63540953088 count 1
> [   82.910050]         ref#1: shared data backref parent 63517212672 count 1
> [   82.910050]         ref#2: shared data backref parent 63459491840 count 1
> [   82.910051]         ref#3: shared data backref parent 63315623936 count 1
> [   82.910051]         ref#4: shared data backref parent 63314083840 count 1
> [   82.910052]         ref#5: shared data backref parent 63224954880 count 1
> [   82.910052]         ref#6: shared data backref parent 63095816192 count 1
> [   82.910053]         ref#7: shared data backref parent 63069995008 count 1
> [   82.910053]         ref#8: shared data backref parent 63008587776 count 1
> [   82.910054]         ref#9: shared data backref parent 62889852928 count 1
> [   82.910054]         ref#10: shared data backref parent 62876680192 count 1
> [   82.910055]         ref#11: shared data backref parent 62803886080 count 1
> [   82.910055]         ref#12: shared data backref parent 62673485824 count 1
> [   82.910056]         ref#13: shared data backref parent 62670618624 count 1
> [   82.910056]         ref#14: shared data backref parent 62534598656 count 1
> [   82.910057]         ref#15: shared data backref parent 40772288512 count 1
> [   82.910058]         ref#16: shared data backref parent 40729018368 count 1
> [   82.910058]         ref#17: shared data backref parent 40499953664 count 1
> [   82.910059]         ref#18: shared data backref parent 40440922112 count 1
> [   82.910059]         ref#19: shared data backref parent 40432353280 count 1
> [   82.910060]         ref#20: shared data backref parent 40177680384 count 1
> [   82.910060]         ref#21: shared data backref parent 40155938816 count 1
> [   82.910061]         ref#22: shared data backref parent 40109391872 count 1
> [   82.910061]         ref#23: shared data backref parent 28942811136 count 1
> [   82.910062]         ref#24: shared data backref parent 28806053888 count 1
> [   82.910062]         ref#25: shared data backref parent 28794404864 count 1
> [   82.910063]         ref#26: shared data backref parent 28240003072 count 1
> [   82.910063]         ref#27: shared data backref parent 750813184 count 1
> [   82.910064]         ref#28: shared data backref parent 386875392 count 1
> [   82.910064]         ref#29: shared data backref parent 239239168 count 1
> [   82.910065]     item 15 key (65470377984 168 4096) itemoff 13465 itemsize 37
> [   82.910065]         extent refs 1 gen 162535 flags 1
> [   82.910066]         ref#0: shared data backref parent 62630952960 count 1
> [   82.910066]     item 16 key (65470382080 168 4096) itemoff 13402 itemsize 63
> [   82.910067]         extent refs 3 gen 170793 flags 1
> [   82.910067]         ref#0: shared data backref parent 63224971264 count 1
> [   82.910067]         ref#1: shared data backref parent 63008604160 count 1
> [   82.910068]         ref#2: shared data backref parent 62803902464 count 1
> [   82.910069]     item 17 key (65470386176 168 4096) itemoff 13365 itemsize 37
> [   82.910069]         extent refs 1 gen 170806 flags 1
> [   82.910069]         ref#0: shared data backref parent 62522458112 count 1
> [   82.910070]     item 18 key (65470390272 168 4096) itemoff 13328 itemsize 37
> [   82.910071]         extent refs 1 gen 170806 flags 1
> [   82.910071]         ref#0: shared data backref parent 62522507264 count 1
> [   82.910072]     item 19 key (65470394368 168 4096) itemoff 13226 itemsize 102
> [   82.910072]         extent refs 6 gen 170809 flags 1
> [   82.910072]         ref#0: shared data backref parent 63455215616 count 1
> [   82.910073]         ref#1: shared data backref parent 63224987648 count 1
> [   82.910073]         ref#2: shared data backref parent 63008653312 count 1
> [   82.910074]         ref#3: shared data backref parent 62803951616 count 1
> [   82.910074]         ref#4: shared data backref parent 28263088128 count 1
> [   82.910075]         ref#5: shared data backref parent 242253824 count 1
> [   82.910076]     item 20 key (65470402560 168 4096) itemoff 13189 itemsize 37
> [   82.910076]         extent refs 1 gen 170809 flags 1
> [   82.910076]         ref#0: shared data backref parent 62803984384 count 1
> [   82.910077]     item 21 key (65470443520 168 8192) itemoff 13152 itemsize 37
> [   82.910077]         extent refs 1 gen 170809 flags 1
> [   82.910077]         ref#0: shared data backref parent 62803984384 count 1
> [   82.910078]     item 22 key (65470451712 168 4096) itemoff 13115 itemsize 37
> [   82.910079]         extent refs 1 gen 170809 flags 1
> [   82.910079]         ref#0: shared data backref parent 62803984384 count 1
> [   82.910080]     item 23 key (65470455808 168 4096) itemoff 13078 itemsize 37
> [   82.910080]         extent refs 1 gen 170809 flags 1
> [   82.910080]         ref#0: shared data backref parent 62555881472 count 1
> [   82.910081]     item 24 key (65470459904 168 4096) itemoff 13041 itemsize 37
> [   82.910081]         extent refs 1 gen 170809 flags 1
> [   82.910082]         ref#0: shared data backref parent 62761713664 count 1
> [   82.910082]     item 25 key (65470464000 168 4096) itemoff 13004 itemsize 37
> [   82.910083]         extent refs 1 gen 170809 flags 1
> [   82.910083]         ref#0: shared data backref parent 62761713664 count 1
> [   82.910084]     item 26 key (65470468096 168 4096) itemoff 12967 itemsize 37
> [   82.910084]         extent refs 1 gen 170809 flags 1
> [   82.910084]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910085]     item 27 key (65470472192 168 4096) itemoff 12930 itemsize 37
> [   82.910085]         extent refs 1 gen 170809 flags 1
> [   82.910086]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910086]     item 28 key (65470476288 168 4096) itemoff 12893 itemsize 37
> [   82.910087]         extent refs 1 gen 170809 flags 1
> [   82.910087]         ref#0: shared data backref parent 62804787200 count 1
> [   82.910088]     item 29 key (65470480384 168 4096) itemoff 12856 itemsize 37
> [   82.910088]         extent refs 1 gen 170809 flags 1
> [   82.910088]         ref#0: shared data backref parent 62804787200 count 1
> [   82.910089]     item 30 key (65470484480 168 4096) itemoff 12819 itemsize 37
> [   82.910090]         extent refs 1 gen 170809 flags 1
> [   82.910090]         ref#0: shared data backref parent 62804787200 count 1
> [   82.910091]     item 31 key (65470488576 168 4096) itemoff 12782 itemsize 37
> [   82.910091]         extent refs 1 gen 170809 flags 1
> [   82.910091]         ref#0: shared data backref parent 62804787200 count 1
> [   82.910092]     item 32 key (65470492672 168 4096) itemoff 11793 itemsize 989
> [   82.910092]         extent refs 88 gen 170809 flags 1
> [   82.910093]         ref#0: extent data backref root 267 objectid 819644 offset 13860864 count 1
> [   82.910093]         ref#1: shared data backref parent 63557009408 count 1
> [   82.910094]         ref#2: shared data backref parent 63541084160 count 1
> [   82.910094]         ref#3: shared data backref parent 63494209536 count 1
> [   82.910095]         ref#4: shared data backref parent 63491358720 count 1
> [   82.910095]         ref#5: shared data backref parent 63457280000 count 1
> [   82.910096]         ref#6: shared data backref parent 63455821824 count 1
> [   82.910096]         ref#7: shared data backref parent 63419351040 count 1
> [   82.910097]         ref#8: shared data backref parent 63349981184 count 1
> [   82.910097]         ref#9: shared data backref parent 63336382464 count 1
> [   82.910098]         ref#10: shared data backref parent 63315853312 count 1
> [   82.910098]         ref#11: shared data backref parent 63314214912 count 1
> [   82.910099]         ref#12: shared data backref parent 63256330240 count 1
> [   82.910099]         ref#13: shared data backref parent 63225085952 count 1
> [   82.910100]         ref#14: shared data backref parent 63182602240 count 1
> [   82.910100]         ref#15: shared data backref parent 63154913280 count 1
> [   82.910101]         ref#16: shared data backref parent 63117574144 count 1
> [   82.910101]         ref#17: shared data backref parent 63106957312 count 1
> [   82.910102]         ref#18: shared data backref parent 63096274944 count 1
> [   82.910102]         ref#19: shared data backref parent 63083757568 count 1
> [   82.910103]         ref#20: shared data backref parent 63073304576 count 1
> [   82.910103]         ref#21: shared data backref parent 63070158848 count 1
> [   82.910104]         ref#22: shared data backref parent 63008735232 count 1
> [   82.910104]         ref#23: shared data backref parent 63006670848 count 1
> [   82.910105]         ref#24: shared data backref parent 62973509632 count 1
> [   82.910105]         ref#25: shared data backref parent 62957273088 count 1
> [   82.910106]         ref#26: shared data backref parent 62926422016 count 1
> [   82.910107]         ref#27: shared data backref parent 62890016768 count 1
> [   82.910107]         ref#28: shared data backref parent 62810505216 count 1
> [   82.910108]         ref#29: shared data backref parent 62804787200 count 1
> [   82.910108]         ref#30: shared data backref parent 62771904512 count 1
> [   82.910109]         ref#31: shared data backref parent 62745755648 count 1
> [   82.910109]         ref#32: shared data backref parent 62742396928 count 1
> [   82.910110]         ref#33: shared data backref parent 62732877824 count 1
> [   82.910110]         ref#34: shared data backref parent 62713298944 count 1
> [   82.910111]         ref#35: shared data backref parent 62707154944 count 1
> [   82.910111]         ref#36: shared data backref parent 62689509376 count 1
> [   82.910112]         ref#37: shared data backref parent 62670782464 count 1
> [   82.910112]         ref#38: shared data backref parent 62664982528 count 1
> [   82.910113]         ref#39: shared data backref parent 62662213632 count 1
> [   82.910113]         ref#40: shared data backref parent 62601854976 count 1
> [   82.910114]         ref#41: shared data backref parent 62580080640 count 1
> [   82.910114]         ref#42: shared data backref parent 62534778880 count 1
> [   82.910115]         ref#43: shared data backref parent 40804941824 count 1
> [   82.910115]         ref#44: shared data backref parent 40801501184 count 1
> [   82.910116]         ref#45: shared data backref parent 40772599808 count 1
> [   82.910116]         ref#46: shared data backref parent 40732753920 count 1
> [   82.910117]         ref#47: shared data backref parent 40693153792 count 1
> [   82.910117]         ref#48: shared data backref parent 40677425152 count 1
> [   82.910118]         ref#49: shared data backref parent 40583200768 count 1
> [   82.910118]         ref#50: shared data backref parent 40500379648 count 1
> [   82.910119]         ref#51: shared data backref parent 40432517120 count 1
> [   82.910119]         ref#52: shared data backref parent 40431140864 count 1
> [   82.910120]         ref#53: shared data backref parent 40383086592 count 1
> [   82.910120]         ref#54: shared data backref parent 40239628288 count 1
> [   82.910121]         ref#55: shared data backref parent 40239415296 count 1
> [   82.910121]         ref#56: shared data backref parent 40156889088 count 1
> [   82.910122]         ref#57: shared data backref parent 40156299264 count 1
> [   82.910122]         ref#58: shared data backref parent 40107229184 count 1
> [   82.910123]         ref#59: shared data backref parent 40103608320 count 1
> [   82.910123]         ref#60: shared data backref parent 39940440064 count 1
> [   82.910124]         ref#61: shared data backref parent 39930118144 count 1
> [   82.910124]         ref#62: shared data backref parent 28975251456 count 1
> [   82.910125]         ref#63: shared data backref parent 28943613952 count 1
> [   82.910125]         ref#64: shared data backref parent 28806512640 count 1
> [   82.910126]         ref#65: shared data backref parent 28799287296 count 1
> [   82.910126]         ref#66: shared data backref parent 28451323904 count 1
> [   82.910127]         ref#67: shared data backref parent 28263153664 count 1
> [   82.910127]         ref#68: shared data backref parent 28232269824 count 1
> [   82.910128]         ref#69: shared data backref parent 656375808 count 1
> [   82.910128]         ref#70: shared data backref parent 539115520 count 1
> [   82.910129]         ref#71: shared data backref parent 387366912 count 1
> [   82.910130]         ref#72: shared data backref parent 243924992 count 1
> [   82.910130]     item 33 key (65470492672 184 28162473984) itemoff 11789 itemsize 4
> [   82.910131]         shared data backref count 1
> [   82.910131]     item 34 key (65470492672 184 28806840320) itemoff 11785 itemsize 4
> [   82.910131]         shared data backref count 1
> [   82.910132]     item 35 key (65470492672 184 28927229952) itemoff 11781 itemsize 4
> [   82.910132]         shared data backref count 1
> [   82.910133]     item 36 key (65470492672 184 28945874944) itemoff 11777 itemsize 4
> [   82.910133]         shared data backref count 1
> [   82.910134]     item 37 key (65470492672 184 40604188672) itemoff 11773 itemsize 4
> [   82.910134]         shared data backref count 1
> [   82.910134]     item 38 key (65470492672 184 40681668608) itemoff 11769 itemsize 4
> [   82.910135]         shared data backref count 1
> [   82.910135]     item 39 key (65470492672 184 62554701824) itemoff 11765 itemsize 4
> [   82.910135]         shared data backref count 1
> [   82.910136]     item 40 key (65470492672 184 62581096448) itemoff 11761 itemsize 4
> [   82.910136]         shared data backref count 1
> [   82.910137]     item 41 key (65470492672 184 62722949120) itemoff 11757 itemsize 4
> [   82.910137]         shared data backref count 1
> [   82.910137]     item 42 key (65470492672 184 62952390656) itemoff 11753 itemsize 4
> [   82.910138]         shared data backref count 1
> [   82.910138]     item 43 key (65470492672 184 63040733184) itemoff 11749 itemsize 4
> [   82.910138]         shared data backref count 1
> [   82.910139]     item 44 key (65470492672 184 63099731968) itemoff 11745 itemsize 4
> [   82.910139]         shared data backref count 1
> [   82.910140]     item 45 key (65470492672 184 63268077568) itemoff 11741 itemsize 4
> [   82.910140]         shared data backref count 1
> [   82.910140]     item 46 key (65470492672 184 63371657216) itemoff 11737 itemsize 4
> [   82.910141]         shared data backref count 1
> [   82.910141]     item 47 key (65470492672 184 63378145280) itemoff 11733 itemsize 4
> [   82.910141]         shared data backref count 1
> [   82.910142]     item 48 key (65470496768 168 4096) itemoff 11696 itemsize 37
> [   82.910142]         extent refs 1 gen 170809 flags 1
> [   82.910142]         ref#0: shared data backref parent 62761467904 count 1
> [   82.910143]     item 49 key (65470500864 168 4096) itemoff 11659 itemsize 37
> [   82.910144]         extent refs 1 gen 170809 flags 1
> [   82.910144]         ref#0: shared data backref parent 62761598976 count 1
> [   82.910145]     item 50 key (65470504960 168 4096) itemoff 11076 itemsize 583
> [   82.910145]         extent refs 43 gen 170809 flags 1
> [   82.910145]         ref#0: shared data backref parent 63550799872 count 1
> [   82.910146]         ref#1: shared data backref parent 63508496384 count 1
> [   82.910146]         ref#2: shared data backref parent 63480430592 count 1
> [   82.910147]         ref#3: shared data backref parent 63387418624 count 1
> [   82.910147]         ref#4: shared data backref parent 63370756096 count 1
> [   82.910148]         ref#5: shared data backref parent 63355355136 count 1
> [   82.910148]         ref#6: shared data backref parent 63261196288 count 1
> [   82.910149]         ref#7: shared data backref parent 63208521728 count 1
> [   82.910149]         ref#8: shared data backref parent 63184355328 count 1
> [   82.910150]         ref#9: shared data backref parent 63128027136 count 1
> [   82.910150]         ref#10: shared data backref parent 63066669056 count 1
> [   82.910151]         ref#11: shared data backref parent 63043780608 count 1
> [   82.910151]         ref#12: shared data backref parent 63015501824 count 1
> [   82.910152]         ref#13: shared data backref parent 62946148352 count 1
> [   82.910152]         ref#14: shared data backref parent 62925127680 count 1
> [   82.910153]         ref#15: shared data backref parent 62880858112 count 1
> [   82.910153]         ref#16: shared data backref parent 62834311168 count 1
> [   82.910154]         ref#17: shared data backref parent 62723932160 count 1
> [   82.910155]         ref#18: shared data backref parent 62687199232 count 1
> [   82.910155]         ref#19: shared data backref parent 62640291840 count 1
> [   82.910156]         ref#20: shared data backref parent 62608162816 count 1
> [   82.910156]         ref#21: shared data backref parent 62534615040 count 1
> [   82.910157]         ref#22: shared data backref parent 62518984704 count 1
> [   82.910157]         ref#23: shared data backref parent 62509776896 count 1
> [   82.910158]         ref#24: shared data backref parent 40632041472 count 1
> [   82.910158]         ref#25: shared data backref parent 40583282688 count 1
> [   82.910159]         ref#26: shared data backref parent 40528920576 count 1
> [   82.910159]         ref#27: shared data backref parent 40512946176 count 1
> [   82.910160]         ref#28: shared data backref parent 40490319872 count 1
> [   82.910160]         ref#29: shared data backref parent 40327495680 count 1
> [   82.910161]         ref#30: shared data backref parent 40237416448 count 1
> [   82.910161]         ref#31: shared data backref parent 40212545536 count 1
> [   82.910162]         ref#32: shared data backref parent 39985201152 count 1
> [   82.910162]         ref#33: shared data backref parent 39809941504 count 1
> [   82.910163]         ref#34: shared data backref parent 28943400960 count 1
> [   82.910163]         ref#35: shared data backref parent 28792651776 count 1
> [   82.910164]         ref#36: shared data backref parent 28437135360 count 1
> [   82.910164]         ref#37: shared data backref parent 28382773248 count 1
> [   82.910165]         ref#38: shared data backref parent 28173221888 count 1
> [   82.910165]         ref#39: shared data backref parent 374931456 count 1
> [   82.910166]         ref#40: shared data backref parent 244285440 count 1
> [   82.910166]         ref#41: shared data backref parent 201129984 count 1
> [   82.910167]         ref#42: shared data backref parent 10780672 count 1
> [   82.910168]     item 51 key (65470509056 168 4096) itemoff 10792 itemsize 284
> [   82.910168]         extent refs 20 gen 170812 flags 1
> [   82.910168]         ref#0: shared data backref parent 63517196288 count 1
> [   82.910169]         ref#1: shared data backref parent 63455199232 count 1
> [   82.910169]         ref#2: shared data backref parent 63314067456 count 1
> [   82.910170]         ref#3: shared data backref parent 63225004032 count 1
> [   82.910170]         ref#4: shared data backref parent 63096160256 count 1
> [   82.910171]         ref#5: shared data backref parent 63008636928 count 1
> [   82.910171]         ref#6: shared data backref parent 62889885696 count 1
> [   82.910172]         ref#7: shared data backref parent 62803918848 count 1
> [   82.910172]         ref#8: shared data backref parent 62672044032 count 1
> [   82.910173]         ref#9: shared data backref parent 40772304896 count 1
> [   82.910173]         ref#10: shared data backref parent 40732213248 count 1
> [   82.910174]         ref#11: shared data backref parent 40440938496 count 1
> [   82.910174]         ref#12: shared data backref parent 40432386048 count 1
> [   82.910175]         ref#13: shared data backref parent 40156839936 count 1
> [   82.910175]         ref#14: shared data backref parent 40109408256 count 1
> [   82.910176]         ref#15: shared data backref parent 28943482880 count 1
> [   82.910176]         ref#16: shared data backref parent 28794568704 count 1
> [   82.910177]         ref#17: shared data backref parent 28263006208 count 1
> [   82.910177]         ref#18: shared data backref parent 387252224 count 1
> [   82.910178]         ref#19: shared data backref parent 242237440 count 1
> [   82.910179]     item 52 key (65470513152 168 4096) itemoff 10495 itemsize 297
> [   82.910179]         extent refs 21 gen 170812 flags 1
> [   82.910179]         ref#0: shared data backref parent 63533907968 count 1
> [   82.910180]         ref#1: shared data backref parent 63377145856 count 1
> [   82.910180]         ref#2: shared data backref parent 63304040448 count 1
> [   82.910181]         ref#3: shared data backref parent 63184338944 count 1
> [   82.910181]         ref#4: shared data backref parent 63067865088 count 1
> [   82.910182]         ref#5: shared data backref parent 62972690432 count 1
> [   82.910182]         ref#6: shared data backref parent 62880841728 count 1
> [   82.910183]         ref#7: shared data backref parent 62722883584 count 1
> [   82.910183]         ref#8: shared data backref parent 62568628224 count 1
> [   82.910184]         ref#9: shared data backref parent 62534615040 count 1
> [   82.910185]         ref#10: shared data backref parent 62529159168 count 1
> [   82.910185]         ref#11: shared data backref parent 40533098496 count 1
> [   82.910186]         ref#12: shared data backref parent 40528920576 count 1
> [   82.910186]         ref#13: shared data backref parent 40298315776 count 1
> [   82.910187]         ref#14: shared data backref parent 40212119552 count 1
> [   82.910187]         ref#15: shared data backref parent 39948861440 count 1
> [   82.910188]         ref#16: shared data backref parent 28943794176 count 1
> [   82.910188]         ref#17: shared data backref parent 28651995136 count 1
> [   82.910189]         ref#18: shared data backref parent 28173205504 count 1
> [   82.910189]         ref#19: shared data backref parent 491618304 count 1
> [   82.910190]         ref#20: shared data backref parent 218791936 count 1
> [   82.910190]     item 53 key (65470517248 168 8192) itemoff 10458 itemsize 37
> [   82.910191]         extent refs 1 gen 170822 flags 1
> [   82.910191]         ref#0: shared data backref parent 62678024192 count 1
> [   82.910192]     item 54 key (65470525440 168 8192) itemoff 10421 itemsize 37
> [   82.910192]         extent refs 1 gen 170822 flags 1
> [   82.910192]         ref#0: shared data backref parent 62678024192 count 1
> [   82.910193]     item 55 key (65470533632 168 8192) itemoff 10384 itemsize 37
> [   82.910194]         extent refs 1 gen 170824 flags 1
> [   82.910194]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910195]     item 56 key (65470541824 168 8192) itemoff 10347 itemsize 37
> [   82.910195]         extent refs 1 gen 170824 flags 1
> [   82.910195]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910196]     item 57 key (65470554112 168 4096) itemoff 10310 itemsize 37
> [   82.910196]         extent refs 1 gen 170817 flags 1
> [   82.910197]         ref#0: shared data backref parent 62803984384 count 1
> [   82.910197]     item 58 key (65470558208 168 4096) itemoff 10273 itemsize 37
> [   82.910198]         extent refs 1 gen 170817 flags 1
> [   82.910198]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910199]     item 59 key (65470562304 168 4096) itemoff 10236 itemsize 37
> [   82.910199]         extent refs 1 gen 170817 flags 1
> [   82.910199]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910200]     item 60 key (65470566400 168 4096) itemoff 10199 itemsize 37
> [   82.910201]         extent refs 1 gen 170817 flags 1
> [   82.910201]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910202]     item 61 key (65470570496 168 4096) itemoff 10162 itemsize 37
> [   82.910202]         extent refs 1 gen 170817 flags 1
> [   82.910202]         ref#0: shared data backref parent 62804000768 count 1
> [   82.910203]     item 62 key (65470574592 168 4096) itemoff 10125 itemsize 37
> [   82.910203]         extent refs 1 gen 170817 flags 1
> [   82.910204]         ref#0: shared data backref parent 62760517632 count 1
> [   82.910204]     item 63 key (65470578688 168 4096) itemoff 10088 itemsize 37
> [   82.910205]         extent refs 1 gen 170817 flags 1
> [   82.910205]         ref#0: shared data backref parent 62760517632 count 1
> [   82.910206]     item 64 key (65470582784 168 8192) itemoff 10051 itemsize 37
> [   82.910206]         extent refs 1 gen 170824 flags 1
> [   82.910206]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910207]     item 65 key (65470590976 168 4096) itemoff 9845 itemsize 206
> [   82.910207]         extent refs 14 gen 170827 flags 1
> [   82.910208]         ref#0: shared data backref parent 63459491840 count 1
> [   82.910208]         ref#1: shared data backref parent 63314083840 count 1
> [   82.910209]         ref#2: shared data backref parent 63224954880 count 1
> [   82.910209]         ref#3: shared data backref parent 63095816192 count 1
> [   82.910210]         ref#4: shared data backref parent 63008587776 count 1
> [   82.910210]         ref#5: shared data backref parent 62889852928 count 1
> [   82.910211]         ref#6: shared data backref parent 62803886080 count 1
> [   82.910211]         ref#7: shared data backref parent 62673485824 count 1
> [   82.910212]         ref#8: shared data backref parent 40772288512 count 1
> [   82.910212]         ref#9: shared data backref parent 40440922112 count 1
> [   82.910213]         ref#10: shared data backref parent 40177680384 count 1
> [   82.910213]         ref#11: shared data backref parent 28942811136 count 1
> [   82.910214]         ref#12: shared data backref parent 28240003072 count 1
> [   82.910214]         ref#13: shared data backref parent 239239168 count 1
> [   82.910215]     item 66 key (65470599168 168 4096) itemoff 9782 itemsize 63
> [   82.910216]         extent refs 3 gen 170831 flags 1
> [   82.910216]         ref#0: shared data backref parent 63224971264 count 1
> [   82.910216]         ref#1: shared data backref parent 63008604160 count 1
> [   82.910217]         ref#2: shared data backref parent 62803902464 count 1
> [   82.910218]     item 67 key (65470603264 168 4096) itemoff 9563 itemsize 219
> [   82.910218]         extent refs 15 gen 170831 flags 1
> [   82.910218]         ref#0: shared data backref parent 63517196288 count 1
> [   82.910219]         ref#1: shared data backref parent 63455199232 count 1
> [   82.910219]         ref#2: shared data backref parent 63314067456 count 1
> [   82.910220]         ref#3: shared data backref parent 63225004032 count 1
> [   82.910220]         ref#4: shared data backref parent 63096160256 count 1
> [   82.910221]         ref#5: shared data backref parent 63008636928 count 1
> [   82.910221]         ref#6: shared data backref parent 62889885696 count 1
> [   82.910222]         ref#7: shared data backref parent 62803918848 count 1
> [   82.910222]         ref#8: shared data backref parent 62672044032 count 1
> [   82.910223]         ref#9: shared data backref parent 40772304896 count 1
> [   82.910223]         ref#10: shared data backref parent 40440938496 count 1
> [   82.910224]         ref#11: shared data backref parent 40156839936 count 1
> [   82.910224]         ref#12: shared data backref parent 28943482880 count 1
> [   82.910225]         ref#13: shared data backref parent 28263006208 count 1
> [   82.910225]         ref#14: shared data backref parent 242237440 count 1
> [   82.910226]     item 68 key (65470607360 168 4096) itemoff 9409 itemsize 154
> [   82.910226]         extent refs 10 gen 170841 flags 1
> [   82.910227]         ref#0: shared data backref parent 63459491840 count 1
> [   82.910227]         ref#1: shared data backref parent 63224954880 count 1
> [   82.910228]         ref#2: shared data backref parent 63008587776 count 1
> [   82.910228]         ref#3: shared data backref parent 62803886080 count 1
> [   82.910229]         ref#4: shared data backref parent 40772288512 count 1
> [   82.910229]         ref#5: shared data backref parent 40440922112 count 1
> [   82.910230]         ref#6: shared data backref parent 40177680384 count 1
> [   82.910230]         ref#7: shared data backref parent 28942811136 count 1
> [   82.910231]         ref#8: shared data backref parent 28240003072 count 1
> [   82.910231]         ref#9: shared data backref parent 239239168 count 1
> [   82.910232]     item 69 key (65470611456 168 4096) itemoff 9060 itemsize 349
> [   82.910232]         extent refs 25 gen 170841 flags 1
> [   82.910233]         ref#0: shared data backref parent 63540953088 count 1
> [   82.910233]         ref#1: shared data backref parent 63517212672 count 1
> [   82.910234]         ref#2: shared data backref parent 63459491840 count 1
> [   82.910234]         ref#3: shared data backref parent 63315623936 count 1
> [   82.910235]         ref#4: shared data backref parent 63314083840 count 1
> [   82.910235]         ref#5: shared data backref parent 63224954880 count 1
> [   82.910236]         ref#6: shared data backref parent 63095816192 count 1
> [   82.910236]         ref#7: shared data backref parent 63069995008 count 1
> [   82.910237]         ref#8: shared data backref parent 63008587776 count 1
> [   82.910237]         ref#9: shared data backref parent 62889852928 count 1
> [   82.910238]         ref#10: shared data backref parent 62876680192 count 1
> [   82.910238]         ref#11: shared data backref parent 62803886080 count 1
> [   82.910239]         ref#12: shared data backref parent 62673485824 count 1
> [   82.910239]         ref#13: shared data backref parent 62670618624 count 1
> [   82.910240]         ref#14: shared data backref parent 40772288512 count 1
> [   82.910240]         ref#15: shared data backref parent 40729018368 count 1
> [   82.910241]         ref#16: shared data backref parent 40440922112 count 1
> [   82.910241]         ref#17: shared data backref parent 40432353280 count 1
> [   82.910242]         ref#18: shared data backref parent 40177680384 count 1
> [   82.910242]         ref#19: shared data backref parent 40109391872 count 1
> [   82.910243]         ref#20: shared data backref parent 28942811136 count 1
> [   82.910243]         ref#21: shared data backref parent 28794404864 count 1
> [   82.910244]         ref#22: shared data backref parent 28240003072 count 1
> [   82.910244]         ref#23: shared data backref parent 386875392 count 1
> [   82.910245]         ref#24: shared data backref parent 239239168 count 1
> [   82.910246]     item 70 key (65470615552 168 4096) itemoff 8932 itemsize 128
> [   82.910246]         extent refs 8 gen 170841 flags 1
> [   82.910246]         ref#0: shared data backref parent 63455199232 count 1
> [   82.910247]         ref#1: shared data backref parent 63225004032 count 1
> [   82.910247]         ref#2: shared data backref parent 63008636928 count 1
> [   82.910248]         ref#3: shared data backref parent 62803918848 count 1
> [   82.910248]         ref#4: shared data backref parent 40156839936 count 1
> [   82.910249]         ref#5: shared data backref parent 28943482880 count 1
> [   82.910249]         ref#6: shared data backref parent 28263006208 count 1
> [   82.910250]         ref#7: shared data backref parent 242237440 count 1
> [   82.910251]     item 71 key (65470619648 168 4096) itemoff 8895 itemsize 37
> [   82.910251]         extent refs 1 gen 170841 flags 1
> [   82.910251]         ref#0: shared data backref parent 62803951616 count 1
> [   82.910252]     item 72 key (65470623744 168 4096) itemoff 8728 itemsize 167
> [   82.910252]         extent refs 11 gen 170841 flags 1
> [   82.910253]         ref#0: shared data backref parent 63455215616 count 1
> [   82.910253]         ref#1: shared data backref parent 63224987648 count 1
> [   82.910254]         ref#2: shared data backref parent 63008653312 count 1
> [   82.910254]         ref#3: shared data backref parent 62803951616 count 1
> [   82.910255]         ref#4: shared data backref parent 62672060416 count 1
> [   82.910255]         ref#5: shared data backref parent 40772386816 count 1
> [   82.910256]         ref#6: shared data backref parent 40439906304 count 1
> [   82.910256]         ref#7: shared data backref parent 40155185152 count 1
> [   82.910257]         ref#8: shared data backref parent 28943499264 count 1
> [   82.910257]         ref#9: shared data backref parent 28263088128 count 1
> [   82.910258]         ref#10: shared data backref parent 242253824 count 1
> [   82.910259]     item 73 key (65470627840 168 4096) itemoff 8691 itemsize 37
> [   82.910259]         extent refs 1 gen 170841 flags 1
> [   82.910259]         ref#0: shared data backref parent 62761549824 count 1
> [   82.910260]     item 74 key (65470631936 168 16384) itemoff 8654 itemsize 37
> [   82.910260]         extent refs 1 gen 170824 flags 1
> [   82.910261]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910261]     item 75 key (65470648320 168 8192) itemoff 8617 itemsize 37
> [   82.910262]         extent refs 1 gen 170824 flags 1
> [   82.910262]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910263]     item 76 key (65470656512 168 4096) itemoff 8005 itemsize 612
> [   82.910263]         extent refs 44 gen 156236 flags 1
> [   82.910263]         ref#0: extent data backref root 267 objectid 819644 offset 4132864 count 1
> [   82.910264]         ref#1: shared data backref parent 63556435968 count 1
> [   82.910264]         ref#2: shared data backref parent 63553519616 count 1
> [   82.910265]         ref#3: shared data backref parent 63504007168 count 1
> [   82.910266]         ref#4: shared data backref parent 63490015232 count 1
> [   82.910266]         ref#5: shared data backref parent 63457132544 count 1
> [   82.910267]         ref#6: shared data backref parent 63289114624 count 1
> [   82.910267]         ref#7: shared data backref parent 63258591232 count 1
> [   82.910268]         ref#8: shared data backref parent 63182405632 count 1
> [   82.910268]         ref#9: shared data backref parent 63006113792 count 1
> [   82.910269]         ref#10: shared data backref parent 62973706240 count 1
> [   82.910269]         ref#11: shared data backref parent 62931189760 count 1
> [   82.910270]         ref#12: shared data backref parent 62925520896 count 1
> [   82.910270]         ref#13: shared data backref parent 62889951232 count 1
> [   82.910271]         ref#14: shared data backref parent 62828412928 count 1
> [   82.910271]         ref#15: shared data backref parent 62796120064 count 1
> [   82.910272]         ref#16: shared data backref parent 62791139328 count 1
> [   82.910272]         ref#17: shared data backref parent 62740938752 count 1
> [   82.910273]         ref#18: shared data backref parent 62698946560 count 1
> [   82.910273]         ref#19: shared data backref parent 62627315712 count 1
> [   82.910274]         ref#20: shared data backref parent 62579228672 count 1
> [   82.910274]         ref#21: shared data backref parent 62503010304 count 1
> [   82.910275]         ref#22: shared data backref parent 40803909632 count 1
> [   82.910275]         ref#23: shared data backref parent 40800124928 count 1
> [   82.910276]         ref#24: shared data backref parent 40772042752 count 1
> [   82.910276]         ref#25: shared data backref parent 40713715712 count 1
> [   82.910277]         ref#26: shared data backref parent 40697331712 count 1
> [   82.910277]         ref#27: shared data backref parent 40690958336 count 1
> [   82.910278]         ref#28: shared data backref parent 40615264256 count 1
> [   82.910278]         ref#29: shared data backref parent 40514273280 count 1
> [   82.910279]         ref#30: shared data backref parent 40416378880 count 1
> [   82.910279]         ref#31: shared data backref parent 40315781120 count 1
> [   82.910280]         ref#32: shared data backref parent 40260599808 count 1
> [   82.910280]         ref#33: shared data backref parent 40102936576 count 1
> [   82.910281]         ref#34: shared data backref parent 39983742976 count 1
> [   82.910281]         ref#35: shared data backref parent 39929085952 count 1
> [   82.910282]         ref#36: shared data backref parent 39853129728 count 1
> [   82.910282]         ref#37: shared data backref parent 28927901696 count 1
> [   82.910283]         ref#38: shared data backref parent 28090925056 count 1
> [   82.910283]         ref#39: shared data backref parent 389513216 count 1
> [   82.910284]         ref#40: shared data backref parent 387301376 count 1
> [   82.910284]         ref#41: shared data backref parent 321716224 count 1
> [   82.910285]         ref#42: shared data backref parent 215941120 count 1
> [   82.910285]         ref#43: shared data backref parent 41533440 count 1
> [   82.910286]     item 77 key (65470660608 168 172032) itemoff 7968 itemsize 37
> [   82.910287]         extent refs 2 gen 170729 flags 1
> [   82.910287]         ref#0: shared data backref parent 40358248448 count 2
> [   82.910288]     item 78 key (65470832640 168 65536) itemoff 7918 itemsize 50
> [   82.910288]         extent refs 2 gen 170817 flags 1
> [   82.910288]         ref#0: shared data backref parent 63469371392 count 1
> [   82.910289]         ref#1: shared data backref parent 62612914176 count 1
> [   82.910290]     item 79 key (65470898176 168 24576) itemoff 7881 itemsize 37
> [   82.910290]         extent refs 1 gen 170824 flags 1
> [   82.910290]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910291]     item 80 key (65470922752 168 8192) itemoff 7844 itemsize 37
> [   82.910291]         extent refs 1 gen 170824 flags 1
> [   82.910292]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910292]     item 81 key (65470930944 168 8192) itemoff 7807 itemsize 37
> [   82.910293]         extent refs 1 gen 170824 flags 1
> [   82.910293]         ref#0: shared data backref parent 62678024192 count 1
> [   82.910294]     item 82 key (65470939136 168 4096) itemoff 7744 itemsize 63
> [   82.910294]         extent refs 3 gen 170827 flags 1
> [   82.910294]         ref#0: shared data backref parent 63224954880 count 1
> [   82.910295]         ref#1: shared data backref parent 63008587776 count 1
> [   82.910295]         ref#2: shared data backref parent 62803886080 count 1
> [   82.910296]     item 83 key (65470955520 168 49152) itemoff 7707 itemsize 37
> [   82.910297]         extent refs 1 gen 170824 flags 1
> [   82.910297]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910298]     item 84 key (65471004672 168 8192) itemoff 7670 itemsize 37
> [   82.910298]         extent refs 1 gen 170824 flags 1
> [   82.910298]         ref#0: shared data backref parent 62677909504 count 1
> [   82.910299]     item 85 key (65471012864 168 4096) itemoff 7594 itemsize 76
> [   82.910299]         extent refs 4 gen 170827 flags 1
> [   82.910300]         ref#0: shared data backref parent 63454756864 count 1
> [   82.910300]         ref#1: shared data backref parent 63224971264 count 1
> [   82.910301]         ref#2: shared data backref parent 63008604160 count 1
> [   82.910301]         ref#3: shared data backref parent 62803902464 count 1
> [   82.910302]     item 86 key (65471016960 168 4096) itemoff 7528 itemsize 66
> [   82.910302]         extent refs 91 gen 162505 flags 1
> [   82.910303]         ref#0: extent data backref root 267 objectid 819644 offset 10670080 count 1
> [   82.910303]         ref#1: shared data backref parent 62631034880 count 1
> [   82.910304]     item 87 key (65471016960 184 217448448) itemoff 7524 itemsize 4
> [   82.910304]         shared data backref count 1
> [   82.910305]     item 88 key (65471016960 184 242335744) itemoff 7520 itemsize 4
> [   82.910305]         shared data backref count 1
> [   82.910306]     item 89 key (65471016960 184 243793920) itemoff 7516 itemsize 4
> [   82.910306]         shared data backref count 1
> [   82.910306]     item 90 key (65471016960 184 321814528) itemoff 7512 itemsize 4
> [   82.910307]         shared data backref count 1
> [   82.910307]     item 91 key (65471016960 184 387334144) itemoff 7508 itemsize 4
> [   82.910307]         shared data backref count 1
> [   82.910308]     item 92 key (65471016960 184 726777856) itemoff 7504 itemsize 4
> [   82.910308]         shared data backref count 1
> [   82.910309]     item 93 key (65471016960 184 750911488) itemoff 7500 itemsize 4
> [   82.910309]         shared data backref count 1
> [   82.910309]     item 94 key (65471016960 184 28167471104) itemoff 7496 itemsize 4
> [   82.910310]         shared data backref count 1
> [   82.910310]     item 95 key (65471016960 184 28232138752) itemoff 7492 itemsize 4
> [   82.910310]         shared data backref count 1
> [   82.910311]     item 96 key (65471016960 184 28263120896) itemoff 7488 itemsize 4
> [   82.910311]         shared data backref count 1
> [   82.910312]     item 97 key (65471016960 184 28451307520) itemoff 7484 itemsize 4
> [   82.910312]         shared data backref count 1
> [   82.910313]     item 98 key (65471016960 184 28602023936) itemoff 7480 itemsize 4
> [   82.910313]         shared data backref count 1
> [   82.910313]     item 99 key (65471016960 184 28794716160) itemoff 7476 itemsize 4
> [   82.910314]         shared data backref count 1
> [   82.910314]     item 100 key (65471016960 184 28801351680) itemoff 7472 itemsize 4
> [   82.910314]         shared data backref count 1
> [   82.910315]     item 101 key (65471016960 184 28806479872) itemoff 7468 itemsize 4
> [   82.910315]         shared data backref count 1
> [   82.910316]     item 102 key (65471016960 184 28943597568) itemoff 7464 itemsize 4
> [   82.910316]         shared data backref count 1
> [   82.910316]     item 103 key (65471016960 184 28975054848) itemoff 7460 itemsize 4
> [   82.910317]         shared data backref count 1
> [   82.910317]     item 104 key (65471016960 184 39972552704) itemoff 7456 itemsize 4
> [   82.910317]         shared data backref count 1
> [   82.910318]     item 105 key (65471016960 184 39972847616) itemoff 7452 itemsize 4
> [   82.910318]         shared data backref count 1
> [   82.910319]     item 106 key (65471016960 184 40109883392) itemoff 7448 itemsize 4
> [   82.910319]         shared data backref count 1
> [   82.910319]     item 107 key (65471016960 184 40122040320) itemoff 7444 itemsize 4
> [   82.910320]         shared data backref count 1
> [   82.910320]     item 108 key (65471016960 184 40156282880) itemoff 7440 itemsize 4
> [   82.910320]         shared data backref count 1
> [   82.910321]     item 109 key (65471016960 184 40156872704) itemoff 7436 itemsize 4
> [   82.910321]         shared data backref count 1
> [   82.910322]     item 110 key (65471016960 184 40210579456) itemoff 7432 itemsize 4
> [   82.910322]         shared data backref count 1
> [   82.910322]     item 111 key (65471016960 184 40251097088) itemoff 7428 itemsize 4
> [   82.910323]         shared data backref count 1
> [   82.910323]     item 112 key (65471016960 184 40298840064) itemoff 7424 itemsize 4
> [   82.910323]         shared data backref count 1
> [   82.910324]     item 113 key (65471016960 184 40367964160) itemoff 7420 itemsize 4
> [   82.910324]         shared data backref count 1
> [   82.910325]     item 114 key (65471016960 184 40382824448) itemoff 7416 itemsize 4
> [   82.910325]         shared data backref count 1
> [   82.910326]     item 115 key (65471016960 184 40432484352) itemoff 7412 itemsize 4
> [   82.910326]         shared data backref count 1
> [   82.910326]     item 116 key (65471016960 184 40441036800) itemoff 7408 itemsize 4
> [   82.910327]         shared data backref count 1
> [   82.910327] BTRFS error (device dm-0): unable to find ref byte nr 65470324736 parent 62522359808 root 267  owner 819644 offset 1552384
> [   82.910329] ------------[ cut here ]------------
> [   82.910329] BTRFS: Transaction aborted (error -2)
> [   82.910339] WARNING: CPU: 14 PID: 6082 at fs/btrfs/extent-tree.c:6950 __btrfs_free_extent.isra.72+0x7ad/0xac0 [btrfs]
> [   82.910339] Modules linked in: btrfs zstd_compress libcrc32c xor raid6_pq dm_crypt algif_skcipher af_alg intel_rapl x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_codec_hdmi snd_hda_intel kvm snd_usb_audio snd_hda_codec snd_usbmidi_lib snd_hda_core snd_hwdep snd_pcm irqbypass crct10dif_pclmul snd_seq_midi snd_seq_midi_event crc32_pclmul snd_rawmidi ghash_clmulni_intel uvcvideo pcbc snd_seq videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common snd_seq_device snd_timer aesni_intel videodev snd aes_x86_64 crypto_simd cdc_acm soundcore media input_leds joydev cryptd mei_me glue_helper intel_wmi_thunderbolt intel_cstate mei intel_rapl_perf acpi_pad mac_hid sch_fq_codel parport_pc ppdev lp parport ip_tables x_tables autofs4 overlay nls_iso8859_1 dm_mirror dm_region_hash dm_log
> [   82.910354]  hid_generic usbhid hid uas usb_storage amdkfd amd_iommu_v2 amdgpu nouveau chash gpu_sched ttm mxm_wmi drm_kms_helper syscopyarea sysfillrect sysimgblt nvme fb_sys_fops igb e1000e atlantic drm ahci dca i2c_algo_bit nvme_core libahci wmi video
> [   82.910361] CPU: 14 PID: 6082 Comm: btrfs-cleaner Tainted: G        W         4.18.0-10-generic #11-Ubuntu
> [   82.910361] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z370 Professional Gaming i7, BIOS P3.40 11/08/2018
> [   82.910367] RIP: 0010:__btrfs_free_extent.isra.72+0x7ad/0xac0 [btrfs]
> [   82.910367] Code: 48 8b 40 50 f0 48 0f ba a8 10 ce 00 00 02 0f 92 c0 41 59 84 c0 0f 85 1d c1 09 00 44 89 ee 48 c7 c7 70 e2 ee c0 e8 75 4a 45 f8 <0f> 0b e9 07 c1 09 00 4c 89 e7 e8 14 07 ff ff 48 8b 7d 88 4d 89 f9
> [   82.910380] RSP: 0018:ffffa71bc6833bf8 EFLAGS: 00010282
> [   82.910381] RAX: 0000000000000000 RBX: 0000000f3e55e000 RCX: 0000000000000006
> [   82.910381] RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff89d09e5964b0
> [   82.910382] RBP: ffffa71bc6833ca0 R08: 0000000000000001 R09: 0000000000000805
> [   82.910382] R10: 0000000000000004 R11: 0000000000000000 R12: ffff89cfd8338af0
> [   82.910382] R13: 00000000fffffffe R14: 0000000e8e9f8000 R15: 000000000000010b
> [   82.910383] FS:  0000000000000000(0000) GS:ffff89d09e580000(0000) knlGS:0000000000000000
> [   82.910383] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   82.910384] CR2: 0000564398e70760 CR3: 000000076ce0a002 CR4: 00000000003606e0
> [   82.910384] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   82.910385] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   82.910385] Call Trace:
> [   82.910391]  __btrfs_run_delayed_refs+0x20e/0x1010 [btrfs]
> [   82.910397]  ? add_pinned_bytes+0x67/0x70 [btrfs]
> [   82.910402]  ? btrfs_free_tree_block+0x167/0x2d0 [btrfs]
> [   82.910408]  btrfs_run_delayed_refs+0x80/0x190 [btrfs]
> [   82.910415]  btrfs_should_end_transaction+0x47/0x60 [btrfs]
> [   82.910421]  btrfs_drop_snapshot+0x3d1/0x800 [btrfs]
> [   82.910428]  btrfs_clean_one_deleted_snapshot+0xbb/0xf0 [btrfs]
> [   82.910434]  cleaner_kthread+0x136/0x160 [btrfs]
> [   82.910435]  kthread+0x120/0x140
> [   82.910442]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [   82.910443]  ? kthread_bind+0x40/0x40
> [   82.910443]  ret_from_fork+0x35/0x40
> [   82.910444] ---[ end trace 32964933c87d1d28 ]---
> [   82.910445] BTRFS: error (device dm-0) in __btrfs_free_extent:6950: errno=-2 No such entry
> [   82.910446] BTRFS info (device dm-0): forced readonly
> [   82.910447] BTRFS: error (device dm-0) in btrfs_run_delayed_refs:3057: errno=-2 No such entry
> [   82.910763] BTRFS error (device dm-0): pending csums is 221184
It became clear that filesystem is ready to be thown to /dev/null, so I've made an image for future investigation and recreated everything from ~10 minutes old backup.

If there is anything suspicios there and I can run useful tests on filesystem image - let me know, I'll spin up qcow2 CoW image and can do anything with it in VMs.

Unfortunately, it was just over a month since filesystem creation. Other BTRFS filesystem containing backups is perfectly fine (was created late April 2018), but it is only used for sending snapshots to it and not much other activity.

-- 
Sincerely, Nazar Mokrynskyi
github.com/nazar-pc


             reply	other threads:[~2019-02-07  4:08 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-07  4:07 Nazar Mokrynskyi [this message]
2019-02-07  6:26 ` Another btrfs corruption (unable to find ref byte, kernel 5.0-rc4) Nikolay Borisov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5b0d2e94-6e4e-aecd-3eda-459c4a96bb13@mokrynskyi.com \
    --to=nazar@mokrynskyi.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).