[COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.

linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
@ 2018-08-09 15:15 Giuseppe Della Bianca
  2018-08-09 18:48 ` Jeff Mahoney
  0 siblings, 1 reply; 20+ messages in thread
From: Giuseppe Della Bianca @ 2018-08-09 15:15 UTC (permalink / raw)
  To: linux-btrfs

Hi.

My system: 
- Fedora 28 x86_64
- kernel-4.17.7-200
- btrfs-progs-4.15.1-1

The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:

- Run command 'btrfs subvolume sync -s 2 xyz' .
- After some time the kernel reports an error on the filesystem.
  (error that existed before the command was launched.)
- The filesystem goes in read-only mode.
- The command hangs.


Best regards.

gdb




^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2018-08-09 15:15 [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs Giuseppe Della Bianca
@ 2018-08-09 18:48 ` Jeff Mahoney
  2018-08-10 16:57   ` Giuseppe Della Bianca
  2019-01-01 16:37   ` Giuseppe Della Bianca
  0 siblings, 2 replies; 20+ messages in thread
From: Jeff Mahoney @ 2018-08-09 18:48 UTC (permalink / raw)
  To: bepi, linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 644 bytes --]

On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote:
> Hi.
> 
> My system: 
> - Fedora 28 x86_64
> - kernel-4.17.7-200
> - btrfs-progs-4.15.1-1
> 
> The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:
> 
> - Run command 'btrfs subvolume sync -s 2 xyz' .
> - After some time the kernel reports an error on the filesystem.
>   (error that existed before the command was launched.)
> - The filesystem goes in read-only mode.
> - The command hangs.

Can you provide the output of 'dmesg' and the contents of
/proc/<pid>/stack where <pid> is the pid of the btrfs command process?

-Jeff
-- 
Jeff Mahoney
SUSE Labs



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2018-08-09 18:48 ` Jeff Mahoney
@ 2018-08-10 16:57   ` Giuseppe Della Bianca
  2019-01-01 16:37   ` Giuseppe Della Bianca
  1 sibling, 0 replies; 20+ messages in thread
From: Giuseppe Della Bianca @ 2018-08-10 16:57 UTC (permalink / raw)
  To: Jeff Mahoney; +Cc: linux-btrfs

In data giovedì 9 agosto 2018 20:48:03 CEST, Jeff Mahoney ha scritto:
> On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote:
> > Hi.
> > 
> > My system:
> > - Fedora 28 x86_64
> > - kernel-4.17.7-200
> > - btrfs-progs-4.15.1-1
> > 
> > The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:
> > 
> > - Run command 'btrfs subvolume sync -s 2 xyz' .
> > - After some time the kernel reports an error on the filesystem.
> > 
> >   (error that existed before the command was launched.)
> > 
> > - The filesystem goes in read-only mode.
> > - The command hangs.
> 
> Can you provide the output of 'dmesg' and the contents of
> /proc/<pid>/stack where <pid> is the pid of the btrfs command process?
> 
> -Jeff

For pid info we have to wait for the problem to reoccur.
(the filesystem has been restored.)

Kernel messages

Aug  7 15:45:33 exnet kernel: WARNING: CPU: 2 PID: 9700 at fs/btrfs/extent-
tree.c:7001 __btrfs_free_extent.isra.70+0x782/0xb10 [btrfs]
Aug  7 15:45:33 exnet kernel: Modules linked in: ppp_deflate bsd_comp 
ppp_async ppp_generic slhc cdc_ether usbnet option mii usb_wwan uas 
usb_storage fuse tun bridge devlink ebtable_filter ebtables bnx2fc cnic uio 
8021q fcoe libfcoe garp mrp stp llc libfc scsi_transport_fc nf_log_ipv4 
nf_log_common xt_LOG xt_limit xt_multiport xt_CHECKSUM iptable_mangle 
ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 
nf_defrag_ipv4 nf_nat_ipv4 nf_nat ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 
nf_defrag_ipv6 xt_conntrack nf_conntrack uinput ip6table_filter ip6_tables 
eeepc_wmi asus_wmi sparse_keymap iTCO_wdt iTCO_vendor_support rfkill ppdev 
snd_hda_codec_hdmi wmi_bmof mxm_wmi snd_usb_audio snd_hda_codec_realtek 
snd_usbmidi_lib snd_hda_codec_generic snd_rawmidi intel_rapl 
x86_pkg_temp_thermal intel_powerclamp coretemp
Aug  7 15:45:33 exnet kernel: snd_hda_intel kvm_intel kvm snd_hda_codec 
snd_hda_core snd_hwdep snd_seq snd_seq_device irqbypass crct10dif_pclmul 
crc32_pclmul snd_pcm ghash_clmulni_intel intel_cstate intel_uncore 
intel_rapl_perf snd_timer snd mei_me soundcore i2c_i801 mei shpchp parport_pc 
parport wmi acpi_pad vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) binfmt_misc 
vboxdrv(OE) btrfs libcrc32c xor zstd_decompress zstd_compress xxhash raid6_pq 
i915 i2c_algo_bit drm_kms_helper e1000e nvme drm crc32c_intel serio_raw 
nvme_core video analog gameport joydev i2c_dev
Aug  7 15:45:33 exnet kernel: CPU: 2 PID: 9700 Comm: btrfs-cleaner Tainted: G           
OE     4.17.7-200.fc28.x86_64 #1
Aug  7 15:45:33 exnet kernel: Hardware name: System manufacturer System 
Product Name/Z170M-PLUS, BIOS 0704 02/18/2016
Aug  7 15:45:33 exnet kernel: RIP: 0010:__btrfs_free_extent.isra.
70+0x782/0xb10 [btrfs]
Aug  7 15:45:33 exnet kernel: RSP: 0018:ffffaf1c4992bc30 EFLAGS: 00010246
Aug  7 15:45:33 exnet kernel: RAX: 00000000fffffffe RBX: ffff8fe421682230 RCX: 
0000000000000000
Aug  7 15:45:33 exnet kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
ffff8fe428f7b068
Aug  7 15:45:33 exnet kernel: RBP: 0000000e401ec000 R08: ffffaf1c4992bba4 R09: 
0000000000000046
Aug  7 15:45:33 exnet kernel: R10: 00000000000002ef R11: 0000000000000000 R12: 
ffff8fe43cd50000
Aug  7 15:45:33 exnet kernel: R13: 00000000fffffffe R14: 0000000000000000 R15: 
0000000000000578
Aug  7 15:45:33 exnet kernel: FS:  0000000000000000(0000) 
GS:ffff8fe5bbd00000(0000) knlGS:0000000000000000
Aug  7 15:45:33 exnet kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 
0000000080050033
Aug  7 15:45:33 exnet kernel: CR2: 0000564e5fb75650 CR3: 000000028f20a004 CR4: 
00000000003606e0
Aug  7 15:45:33 exnet kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
Aug  7 15:45:33 exnet kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
Aug  7 15:45:33 exnet kernel: Call Trace:
Aug  7 15:45:33 exnet kernel: __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
Aug  7 15:45:33 exnet kernel: ? btrfs_set_disk_extent_flags+0x72/0xb0 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_run_delayed_refs+0x78/0x180 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_should_end_transaction+0x3e/0x60 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
Aug  7 15:45:33 exnet kernel: ? btree_submit_bio_start+0x20/0x20 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_clean_one_deleted_snapshot+0xba/0xe0 
[btrfs]
Aug  7 15:45:33 exnet kernel: cleaner_kthread+0x129/0x160 [btrfs]
Aug  7 15:45:33 exnet kernel: kthread+0x112/0x130
Aug  7 15:45:33 exnet kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Aug  7 15:45:33 exnet kernel: ret_from_fork+0x35/0x40
Aug  7 15:45:33 exnet kernel: Code: b8 00 00 00 48 8b 7c 24 18 e8 bb b7 ff ff 
41 89 c5 58 5a c6 44 24 24 00 45 85 ed 0f 84 97 f9 ff ff 41 83 fd fe 0f 85 63 
fc ff ff <0f> 0b 48 8b 3b e8 94 ca 00 00 4d 89 f8 4c 89 f1 48 89 ea ff b4 
Aug  7 15:45:33 exnet kernel: ---[ end trace 961b1007d36aa769 ]---
Aug  7 15:45:33 exnet kernel: BTRFS info (device sda3): leaf 88886329344 gen 
1601 total ptrs 135 free space 5456 owner 2
Aug  7 15:45:33 exnet kernel: #011item 0 key (61204873216 169 0) itemoff 16142 
itemsize 141
Aug  7 15:45:33 exnet kernel: #011#011extent refs 13 gen 1476 flags 258
Aug  7 15:45:33 exnet kernel: #011#011ref#0: tree block backref root 1407
Aug  7 15:45:33 exnet kernel: #011#011ref#1: tree block backref root 1405
Aug  7 15:45:33 exnet kernel: #011#011ref#2: tree block backref root 1403
Aug  7 15:45:33 exnet kernel: #011#011ref#3: shared block backref parent 
88626577408
Aug  7 15:45:33 exnet kernel: #011#011ref#4: shared block backref parent 
88508121088
Aug  7 15:45:33 exnet kernel: #011#011ref#5: shared block backref parent 
88417042432
Aug  7 15:45:33 exnet kernel: #011#011ref#6: shared block backref parent 
88244256768
Aug  7 15:45:33 exnet kernel: #011#011ref#7: shared block backref parent 
88084234240
Aug  7 15:45:33 exnet kernel: #011#011ref#8: shared block backref parent 
80272637952

................................................................

................................................................

................................................................

Aug  7 15:45:33 exnet kernel: #011#011extent refs 1 gen 1476 flags 258
Aug  7 15:45:33 exnet kernel: #011#011ref#0: shared block backref parent 
61205299200
Aug  7 15:45:33 exnet kernel: #011item 134 key (61207068672 169 0) itemoff 
8831 itemsize 33
Aug  7 15:45:33 exnet kernel: #011#011extent refs 1 gen 1476 flags 258
Aug  7 15:45:33 exnet kernel: #011#011ref#0: shared block backref parent 
61205299200
Aug  7 15:45:33 exnet kernel: BTRFS error (device sda3): unable to find ref 
byte nr 61205299200 parent 0 root 1400  owner 1 offset 0
Aug  7 15:45:33 exnet kernel: ------------[ cut here ]------------
Aug  7 15:45:33 exnet kernel: WARNING: CPU: 2 PID: 9700 at fs/btrfs/extent-
tree.c:7007 __btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
Aug  7 15:45:33 exnet kernel: Modules linked in: ppp_deflate bsd_comp 
ppp_async ppp_generic slhc cdc_ether usbnet option mii usb_wwan uas 
usb_storage fuse tun bridge devlink ebtable_filter ebtables bnx2fc cnic uio 
8021q fcoe libfcoe garp mrp stp llc libfc scsi_transport_fc nf_log_ipv4 
nf_log_common xt_LOG xt_limit xt_multiport xt_CHECKSUM iptable_mangle 
ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 
nf_defrag_ipv4 nf_nat_ipv4 nf_nat ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 
nf_defrag_ipv6 xt_conntrack nf_conntrack uinput ip6table_filter ip6_tables 
eeepc_wmi asus_wmi sparse_keymap iTCO_wdt iTCO_vendor_support rfkill ppdev 
snd_hda_codec_hdmi wmi_bmof mxm_wmi snd_usb_audio snd_hda_codec_realtek 
snd_usbmidi_lib snd_hda_codec_generic snd_rawmidi intel_rapl 
x86_pkg_temp_thermal intel_powerclamp coretemp
Aug  7 15:45:33 exnet kernel: snd_hda_intel kvm_intel kvm snd_hda_codec 
snd_hda_core snd_hwdep snd_seq snd_seq_device irqbypass crct10dif_pclmul 
crc32_pclmul snd_pcm ghash_clmulni_intel intel_cstate intel_uncore 
intel_rapl_perf snd_timer snd mei_me soundcore i2c_i801 mei shpchp parport_pc 
parport wmi acpi_pad vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) binfmt_misc 
vboxdrv(OE) btrfs libcrc32c xor zstd_decompress zstd_compress xxhash raid6_pq 
i915 i2c_algo_bit drm_kms_helper e1000e nvme drm crc32c_intel serio_raw 
nvme_core video analog gameport joydev i2c_dev
Aug  7 15:45:33 exnet kernel: CPU: 2 PID: 9700 Comm: btrfs-cleaner Tainted: G        
W  OE     4.17.7-200.fc28.x86_64 #1
Aug  7 15:45:33 exnet kernel: Hardware name: System manufacturer System 
Product Name/Z170M-PLUS, BIOS 0704 02/18/2016
Aug  7 15:45:33 exnet kernel: RIP: 0010:__btrfs_free_extent.isra.
70+0x7e2/0xb10 [btrfs]
Aug  7 15:45:33 exnet kernel: RSP: 0018:ffffaf1c4992bc30 EFLAGS: 00010286
Aug  7 15:45:33 exnet kernel: RAX: 0000000000000000 RBX: ffff8fe421682230 RCX: 
0000000000000006
Aug  7 15:45:33 exnet kernel: RDX: 0000000000000007 RSI: 0000000000000086 RDI: 
ffff8fe5bbd16930
Aug  7 15:45:33 exnet kernel: RBP: 0000000e401ec000 R08: 0000000000000038 R09: 
00000000000006d1
Aug  7 15:45:33 exnet kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 
ffff8fe43cd50000
Aug  7 15:45:33 exnet kernel: R13: 00000000fffffffe R14: 0000000000000000 R15: 
0000000000000578
Aug  7 15:45:33 exnet kernel: FS:  0000000000000000(0000) 
GS:ffff8fe5bbd00000(0000) knlGS:0000000000000000
Aug  7 15:45:33 exnet kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 
0000000080050033
Aug  7 15:45:33 exnet kernel: CR2: 0000564e5fb75650 CR3: 000000028f20a004 CR4: 
00000000003606e0
Aug  7 15:45:33 exnet kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
Aug  7 15:45:33 exnet kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 
0000000000000400
Aug  7 15:45:33 exnet kernel: Call Trace:
Aug  7 15:45:33 exnet kernel: __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
Aug  7 15:45:33 exnet kernel: ? btrfs_set_disk_extent_flags+0x72/0xb0 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_run_delayed_refs+0x78/0x180 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_should_end_transaction+0x3e/0x60 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
Aug  7 15:45:33 exnet kernel: ? btree_submit_bio_start+0x20/0x20 [btrfs]
Aug  7 15:45:33 exnet kernel: btrfs_clean_one_deleted_snapshot+0xba/0xe0 
[btrfs]
Aug  7 15:45:33 exnet kernel: cleaner_kthread+0x129/0x160 [btrfs]
Aug  7 15:45:33 exnet kernel: kthread+0x112/0x130
Aug  7 15:45:33 exnet kernel: ? kthread_create_worker_on_cpu+0x70/0x70
Aug  7 15:45:33 exnet kernel: ret_from_fork+0x35/0x40
Aug  7 15:45:33 exnet kernel: Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 
02 0f 92 c0 41 59 84 c0 0f 85 6c a9 09 00 44 89 ee 48 c7 c7 e8 2e 7c c0 e8 68 
9a 99 ed <0f> 0b e9 56 a9 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d 
Aug  7 15:45:33 exnet kernel: ---[ end trace 961b1007d36aa76a ]---
Aug  7 15:45:33 exnet kernel: BTRFS: error (device sda3) in 
__btrfs_free_extent:7007: errno=-2 No such entry
Aug  7 15:45:33 exnet kernel: BTRFS info (device sda3): forced readonly
Aug  7 15:45:33 exnet kernel: BTRFS: error (device sda3) in 
btrfs_run_delayed_refs:3076: errno=-2 No such entry


gdb

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2018-08-09 18:48 ` Jeff Mahoney
  2018-08-10 16:57   ` Giuseppe Della Bianca
@ 2019-01-01 16:37   ` Giuseppe Della Bianca
  2019-01-04 20:34     ` Jeff Mahoney
  2019-01-07 23:11     ` Filipe Manana
  1 sibling, 2 replies; 20+ messages in thread
From: Giuseppe Della Bianca @ 2019-01-01 16:37 UTC (permalink / raw)
  To: Jeff Mahoney; +Cc: linux-btrfs

In data giovedì 9 agosto 2018 20:48:03 CEST, Jeff Mahoney ha scritto:
> On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote:
> > Hi.
> > 
> > My system:
> > - Fedora 28 x86_64
> > - kernel-4.17.7-200
> > - btrfs-progs-4.15.1-1
> > 
> > The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:
> > 
> > - Run command 'btrfs subvolume sync -s 2 xyz' .
> > - After some time the kernel reports an error on the filesystem.
> > 
> >   (error that existed before the command was launched.)
> > 
> > - The filesystem goes in read-only mode.
> > - The command hangs.
> 
> Can you provide the output of 'dmesg' and the contents of
> /proc/<pid>/stack where <pid> is the pid of the btrfs command process?
> 
> -Jeff

I had the problem again.
This is the information you asked for:

root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2 /
tmp/tmp.p9SiQ1GnpV

cat /proc/17133/stack 
[<0>] hrtimer_nanosleep+0xce/0x1e0
[<0>] __x64_sys_nanosleep+0x77/0xa0
[<0>] do_syscall_64+0x5b/0x160
[<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[<0>] 0xffffffffffffffff

...............
[ 1558.053812] CPU: 0 PID: 16871 Comm: btrfs-cleaner Not tainted 
4.17.19-200.fc28.x86_64 #1
[ 1558.053815] Hardware name: System manufacturer System Product Name/M2N, 
BIOS 0902    02/16/2009
[ 1558.053904] RIP: 0010:__btrfs_free_extent.isra.70+0x782/0xb10 [btrfs]
[ 1558.053907] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010246
[ 1558.053912] RAX: 00000000fffffffe RBX: ffff8b71313f8380 RCX: 
0000000000000000
[ 1558.053914] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
ffff8b711d8ef578
[ 1558.053916] RBP: 00000011cb804000 R08: ffffa63985f8fba4 R09: 
0000000000000018
[ 1558.053919] R10: 0000000000000835 R11: 0000000000000000 R12: 
ffff8b715f5d4000
[ 1558.053921] R13: 00000000fffffffe R14: 0000000000000000 R15: 
00000000000005aa
[ 1558.053925] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
0000000000000000
[ 1558.053928] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1558.053930] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4: 
00000000000006f0
[ 1558.053934] Call Trace:
[ 1558.054011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
[ 1558.054011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
[ 1558.054011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
[ 1558.054011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
[ 1558.054011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
[ 1558.054011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
[ 1558.054011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
[ 1558.054011]  cleaner_kthread+0x129/0x160 [btrfs]
[ 1558.054011]  kthread+0x112/0x130
[ 1558.054011]  ? kthread_create_worker_on_cpu+0x70/0x70
[ 1558.054011]  ret_from_fork+0x35/0x40
[ 1558.054011] Code: b8 00 00 00 48 8b 7c 24 18 e8 bb b7 ff ff 41 89 c5 58 5a 
c6 44 24 24 00 45 85 ed 0f 84 97 f9 ff ff 41 83 fd fe 0f 85 63 fc ff ff <0f> 
0b 48 8b 3b e8 84 ca 00 00 4d 89 f8 4c 89 f1 48 89 ea ff b4 
[ 1558.054011] ---[ end trace c4eefd1fdc016cea ]---
[ 1558.054832] BTRFS info (device sda3): leaf 62678286336 gen 6388 total ptrs 
148 free space 6202 owner 2
[ 1558.054837]  item 0 key (76427296768 169 0) itemoff 16250 itemsize 33
[ 1558.054841]          extent refs 1 gen 6256 flags 258
[ 1558.054844]          ref#0: shared block backref parent 76512641024
[ 1558.054850]  item 1 key (76427313152 169 0) itemoff 16217 itemsize 33
[ 1558.054853]          extent refs 1 gen 6256 flags 258
[ 1558.054854]          ref#0: shared block backref parent 76512641024

]zac[
...................................
]zac[

[ 1558.056504]  item 145 key (76429721600 169 1) itemoff 10004 itemsize 33
[ 1558.056506]          extent refs 1 gen 6303 flags 258
[ 1558.056508]          ref#0: tree block backref root 1434
[ 1558.056512]  item 146 key (76429737984 169 0) itemoff 9962 itemsize 42
[ 1558.056515]          extent refs 2 gen 6277 flags 258
[ 1558.056516]          ref#0: shared block backref parent 81665769472
[ 1558.056519]          ref#1: shared block backref parent 66383904768
[ 1558.056524]  item 147 key (76429754368 169 0) itemoff 9902 itemsize 60
[ 1558.056526]          extent refs 4 gen 6256 flags 258
[ 1558.056528]          ref#0: shared block backref parent 81731174400
[ 1558.056531]          ref#1: shared block backref parent 76512641024
[ 1558.056533]          ref#2: shared block backref parent 76450250752
[ 1558.056536]          ref#3: shared block backref parent 66397339648
[ 1558.056544] BTRFS error (device sda3): unable to find ref byte nr 
76428623872 parent 0 root 1450  owner 1 offset 0
[ 1558.056554] ------------[ cut here ]------------
[ 1558.056556] BTRFS: Transaction aborted (error -2)
[ 1558.056718] WARNING: CPU: 0 PID: 16871 at fs/btrfs/extent-tree.c:7007 
__btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
[ 1558.056720] Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE 
nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast 
xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack devlink ip_set 
nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat 
nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw 
ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 
nf_nat nf_conntrack iptable_mangle iptable_raw iptable_security ebtable_filter 
ebtables ip6table_filter ip6_tables it87 hwmon_vid sunrpc snd_hda_codec_analog 
snd_hda_codec_generic snd_hda_codec_hdmi btrfs powernow_k8 amd64_edac_mod 
edac_mce_amd snd_hda_intel snd_hda_codec kvm_amd xor ccp zstd_compress 
snd_hda_core snd_hwdep kvm snd_seq snd_seq_device ppdev snd_pcm raid6_pq 
irqbypass
[ 1558.056785]  libcrc32c k8temp snd_timer zstd_decompress xxhash i2c_nforce2 
snd asus_atk0110 soundcore shpchp parport_pc parport ata_generic pata_acpi 
nouveau video mxm_wmi wmi i2c_algo_bit drm_kms_helper ttm drm serio_raw 
sata_nv forcedeth
[ 1558.056819] CPU: 0 PID: 16871 Comm: btrfs-cleaner Tainted: G        W         
4.17.19-200.fc28.x86_64 #1
[ 1558.056821] Hardware name: System manufacturer System Product Name/M2N, 
BIOS 0902    02/16/2009
[ 1558.056905] RIP: 0010:__btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
[ 1558.056908] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010286
[ 1558.056912] RAX: 0000000000000000 RBX: ffff8b71313f8380 RCX: 
0000000000000006
[ 1558.056914] RDX: 0000000000000007 RSI: 0000000000000086 RDI: 
ffff8b717fa16930
[ 1558.056916] RBP: 00000011cb804000 R08: 0000000000000038 R09: 
000000000000059a
[ 1558.056918] R10: 0000000000000000 R11: 0000000000000001 R12: 
ffff8b715f5d4000
[ 1558.056920] R13: 00000000fffffffe R14: 0000000000000000 R15: 
00000000000005aa
[ 1558.056924] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
0000000000000000
[ 1558.056927] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1558.056929] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4: 
00000000000006f0
[ 1558.056931] Call Trace:
[ 1558.057011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
[ 1558.057011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
[ 1558.057011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
[ 1558.057011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
[ 1558.057011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
[ 1558.057011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
[ 1558.057011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
[ 1558.057011]  cleaner_kthread+0x129/0x160 [btrfs]
[ 1558.057011]  kthread+0x112/0x130
[ 1558.057011]  ? kthread_create_worker_on_cpu+0x70/0x70
[ 1558.057011]  ret_from_fork+0x35/0x40
[ 1558.057011] Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 02 0f 92 c0 41 
59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 c0 e8 58 33 a0 e7 <0f> 
0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d 
[ 1558.057011] ---[ end trace c4eefd1fdc016ceb ]---
[ 1558.057734] BTRFS: error (device sda3) in __btrfs_free_extent:7007: 
errno=-2 No such entry
[ 1558.057742] BTRFS info (device sda3): forced readonly
[ 1558.057757] BTRFS: error (device sda3) in btrfs_run_delayed_refs:3076: 
errno=-2 No such entry

gdb



^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-01 16:37   ` Giuseppe Della Bianca
@ 2019-01-04 20:34     ` Jeff Mahoney
  2019-01-05 12:30       ` Giuseppe Della Bianca
  2019-01-07 23:11     ` Filipe Manana
  1 sibling, 1 reply; 20+ messages in thread
From: Jeff Mahoney @ 2019-01-04 20:34 UTC (permalink / raw)
  To: Giuseppe Della Bianca; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 8981 bytes --]

On 1/1/19 11:37 AM, Giuseppe Della Bianca wrote:
> In data giovedì 9 agosto 2018 20:48:03 CEST, Jeff Mahoney ha scritto:
>> On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote:
>>> Hi.
>>>
>>> My system:
>>> - Fedora 28 x86_64
>>> - kernel-4.17.7-200
>>> - btrfs-progs-4.15.1-1
>>>
>>> The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:
>>>
>>> - Run command 'btrfs subvolume sync -s 2 xyz' .
>>> - After some time the kernel reports an error on the filesystem.
>>>
>>>   (error that existed before the command was launched.)
>>>
>>> - The filesystem goes in read-only mode.
>>> - The command hangs.
>>
>> Can you provide the output of 'dmesg' and the contents of
>> /proc/<pid>/stack where <pid> is the pid of the btrfs command process?
>>
>> -Jeff
> 
> I had the problem again.
> This is the information you asked for:
> 
> root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2 /
> tmp/tmp.p9SiQ1GnpV
> 
> cat /proc/17133/stack 
> [<0>] hrtimer_nanosleep+0xce/0x1e0
> [<0>] __x64_sys_nanosleep+0x77/0xa0
> [<0>] do_syscall_64+0x5b/0x160
> [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [<0>] 0xffffffffffffffff

Ok, so this is it just sleeping between tree searches.

> ...............
> [ 1558.053812] CPU: 0 PID: 16871 Comm: btrfs-cleaner Not tainted 
> 4.17.19-200.fc28.x86_64 #1
> [ 1558.053815] Hardware name: System manufacturer System Product Name/M2N, 
> BIOS 0902    02/16/2009
> [ 1558.053904] RIP: 0010:__btrfs_free_extent.isra.70+0x782/0xb10 [btrfs]
> [ 1558.053907] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010246
> [ 1558.053912] RAX: 00000000fffffffe RBX: ffff8b71313f8380 RCX: 
> 0000000000000000
> [ 1558.053914] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 
> ffff8b711d8ef578
> [ 1558.053916] RBP: 00000011cb804000 R08: ffffa63985f8fba4 R09: 
> 0000000000000018
> [ 1558.053919] R10: 0000000000000835 R11: 0000000000000000 R12: 
> ffff8b715f5d4000
> [ 1558.053921] R13: 00000000fffffffe R14: 0000000000000000 R15: 
> 00000000000005aa
> [ 1558.053925] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
> 0000000000000000
> [ 1558.053928] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1558.053930] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4: 
> 00000000000006f0
> [ 1558.053934] Call Trace:
> [ 1558.054011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> [ 1558.054011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> [ 1558.054011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> [ 1558.054011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> [ 1558.054011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> [ 1558.054011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [ 1558.054011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> [ 1558.054011]  cleaner_kthread+0x129/0x160 [btrfs]
> [ 1558.054011]  kthread+0x112/0x130
> [ 1558.054011]  ? kthread_create_worker_on_cpu+0x70/0x70
> [ 1558.054011]  ret_from_fork+0x35/0x40
> [ 1558.054011] Code: b8 00 00 00 48 8b 7c 24 18 e8 bb b7 ff ff 41 89 c5 58 5a 
> c6 44 24 24 00 45 85 ed 0f 84 97 f9 ff ff 41 83 fd fe 0f 85 63 fc ff ff <0f> 
> 0b 48 8b 3b e8 84 ca 00 00 4d 89 f8 4c 89 f1 48 89 ea ff b4 
> [ 1558.054011] ---[ end trace c4eefd1fdc016cea ]---
> [ 1558.054832] BTRFS info (device sda3): leaf 62678286336 gen 6388 total ptrs 
> 148 free space 6202 owner 2
> [ 1558.054837]  item 0 key (76427296768 169 0) itemoff 16250 itemsize 33
> [ 1558.054841]          extent refs 1 gen 6256 flags 258
> [ 1558.054844]          ref#0: shared block backref parent 76512641024
> [ 1558.054850]  item 1 key (76427313152 169 0) itemoff 16217 itemsize 33
> [ 1558.054853]          extent refs 1 gen 6256 flags 258
> [ 1558.054854]          ref#0: shared block backref parent 76512641024
> 
> ]zac[
> ...................................
> ]zac[

This part is actually important since we see below that we're searching
for bytenr 76428623872 which, if present, would be in the cut portion of
your log.

> [ 1558.056504]  item 145 key (76429721600 169 1) itemoff 10004 itemsize 33
> [ 1558.056506]          extent refs 1 gen 6303 flags 258
> [ 1558.056508]          ref#0: tree block backref root 1434
> [ 1558.056512]  item 146 key (76429737984 169 0) itemoff 9962 itemsize 42
> [ 1558.056515]          extent refs 2 gen 6277 flags 258
> [ 1558.056516]          ref#0: shared block backref parent 81665769472
> [ 1558.056519]          ref#1: shared block backref parent 66383904768
> [ 1558.056524]  item 147 key (76429754368 169 0) itemoff 9902 itemsize 60
> [ 1558.056526]          extent refs 4 gen 6256 flags 258
> [ 1558.056528]          ref#0: shared block backref parent 81731174400
> [ 1558.056531]          ref#1: shared block backref parent 76512641024
> [ 1558.056533]          ref#2: shared block backref parent 76450250752
> [ 1558.056536]          ref#3: shared block backref parent 66397339648
> [ 1558.056544] BTRFS error (device sda3): unable to find ref byte nr 
> 76428623872 parent 0 root 1450  owner 1 offset 0
> [ 1558.056554] ------------[ cut here ]------------
> [ 1558.056556] BTRFS: Transaction aborted (error -2)
> [ 1558.056718] WARNING: CPU: 0 PID: 16871 at fs/btrfs/extent-tree.c:7007 
> __btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
> [ 1558.056720] Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE 
> nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast 
> xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack devlink ip_set 
> nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat 
> nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw 
> ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 
> nf_nat nf_conntrack iptable_mangle iptable_raw iptable_security ebtable_filter 
> ebtables ip6table_filter ip6_tables it87 hwmon_vid sunrpc snd_hda_codec_analog 
> snd_hda_codec_generic snd_hda_codec_hdmi btrfs powernow_k8 amd64_edac_mod 
> edac_mce_amd snd_hda_intel snd_hda_codec kvm_amd xor ccp zstd_compress 
> snd_hda_core snd_hwdep kvm snd_seq snd_seq_device ppdev snd_pcm raid6_pq 
> irqbypass
> [ 1558.056785]  libcrc32c k8temp snd_timer zstd_decompress xxhash i2c_nforce2 
> snd asus_atk0110 soundcore shpchp parport_pc parport ata_generic pata_acpi 
> nouveau video mxm_wmi wmi i2c_algo_bit drm_kms_helper ttm drm serio_raw 
> sata_nv forcedeth
> [ 1558.056819] CPU: 0 PID: 16871 Comm: btrfs-cleaner Tainted: G        W         
> 4.17.19-200.fc28.x86_64 #1
> [ 1558.056821] Hardware name: System manufacturer System Product Name/M2N, 
> BIOS 0902    02/16/2009
> [ 1558.056905] RIP: 0010:__btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
> [ 1558.056908] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010286
> [ 1558.056912] RAX: 0000000000000000 RBX: ffff8b71313f8380 RCX: 
> 0000000000000006
> [ 1558.056914] RDX: 0000000000000007 RSI: 0000000000000086 RDI: 
> ffff8b717fa16930
> [ 1558.056916] RBP: 00000011cb804000 R08: 0000000000000038 R09: 
> 000000000000059a
> [ 1558.056918] R10: 0000000000000000 R11: 0000000000000001 R12: 
> ffff8b715f5d4000
> [ 1558.056920] R13: 00000000fffffffe R14: 0000000000000000 R15: 
> 00000000000005aa
> [ 1558.056924] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
> 0000000000000000
> [ 1558.056927] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1558.056929] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4: 
> 00000000000006f0
> [ 1558.056931] Call Trace:
> [ 1558.057011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> [ 1558.057011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> [ 1558.057011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> [ 1558.057011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> [ 1558.057011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> [ 1558.057011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [ 1558.057011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> [ 1558.057011]  cleaner_kthread+0x129/0x160 [btrfs]
> [ 1558.057011]  kthread+0x112/0x130
> [ 1558.057011]  ? kthread_create_worker_on_cpu+0x70/0x70
> [ 1558.057011]  ret_from_fork+0x35/0x40
> [ 1558.057011] Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 02 0f 92 c0 41 
> 59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 c0 e8 58 33 a0 e7 <0f> 
> 0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d 
> [ 1558.057011] ---[ end trace c4eefd1fdc016ceb ]---
> [ 1558.057734] BTRFS: error (device sda3) in __btrfs_free_extent:7007: 
> errno=-2 No such entry
> [ 1558.057742] BTRFS info (device sda3): forced readonly
> [ 1558.057757] BTRFS: error (device sda3) in btrfs_run_delayed_refs:3076: 
> errno=-2 No such entry

This is the more important part.  The file system has gone read-only due
to a missing extent backref.  This is corruption.

It also means that the subvolume is never going to disappear during this
mount and 'btrfs subvol sync' will wait forever.

-Jeff
-- 
Jeff Mahoney
SUSE Labs



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-04 20:34     ` Jeff Mahoney
@ 2019-01-05 12:30       ` Giuseppe Della Bianca
  2019-01-06 14:12         ` Qu Wenruo
  2019-01-07 22:40         ` Jeff Mahoney
  0 siblings, 2 replies; 20+ messages in thread
From: Giuseppe Della Bianca @ 2019-01-05 12:30 UTC (permalink / raw)
  To: Jeff Mahoney; +Cc: linux-btrfs

In data venerdì 4 gennaio 2019 21:34:03 CET, Jeff Mahoney ha scritto:
]zac[
> > 
> > root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2
> > /
> > tmp/tmp.p9SiQ1GnpV
> > 
> > cat /proc/17133/stack
> > [<0>] hrtimer_nanosleep+0xce/0x1e0
> > [<0>] __x64_sys_nanosleep+0x77/0xa0
> > [<0>] do_syscall_64+0x5b/0x160
> > [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> > [<0>] 0xffffffffffffffff
> 
> Ok, so this is it just sleeping between tree searches.
]zac[
> This part is actually important since we see below that we're searching
> for bytenr 76428623872 which, if present, would be in the cut portion of
> your log.
]zac[

If you want, I can send you the full log (very long).
From what you wrote below it seems to me that you do not need it

]zac]
> 
> This is the more important part.  The file system has gone read-only due
> to a missing extent backref.  This is corruption.

Yes, but this is an autocorruption of btrfs, which occurred (it seems to me), 
during a cleanup after a deleting of a snapshoot of an operating system 
installation (perhaps interrupted by an umount).

> It also means that the subvolume is never going to disappear during this
> mount and 'btrfs subvol sync' will wait forever.
]zac[

In my opinion this is bad.

The infinite wait occurs during the execution of a backup script, so I will 
have to find a bypass even for this problem (the sync was a patch to another 
autocorruption problem).

Gdb



^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-05 12:30       ` Giuseppe Della Bianca
@ 2019-01-06 14:12         ` Qu Wenruo
  2019-01-06 17:57           ` Giuseppe Della Bianca
  2019-01-07 22:40         ` Jeff Mahoney
  1 sibling, 1 reply; 20+ messages in thread
From: Qu Wenruo @ 2019-01-06 14:12 UTC (permalink / raw)
  To: Giuseppe Della Bianca, Jeff Mahoney; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 2050 bytes --]



On 2019/1/5 下午8:30, Giuseppe Della Bianca wrote:
> In data venerdì 4 gennaio 2019 21:34:03 CET, Jeff Mahoney ha scritto:
> ]zac[
>>>
>>> root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2
>>> /
>>> tmp/tmp.p9SiQ1GnpV
>>>
>>> cat /proc/17133/stack
>>> [<0>] hrtimer_nanosleep+0xce/0x1e0
>>> [<0>] __x64_sys_nanosleep+0x77/0xa0
>>> [<0>] do_syscall_64+0x5b/0x160
>>> [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> [<0>] 0xffffffffffffffff
>>
>> Ok, so this is it just sleeping between tree searches.
> ]zac[
>> This part is actually important since we see below that we're searching
>> for bytenr 76428623872 which, if present, would be in the cut portion of
>> your log.
> ]zac[
> 
> If you want, I can send you the full log (very long).
> From what you wrote below it seems to me that you do not need it
> 
> ]zac]
>>
>> This is the more important part.  The file system has gone read-only due
>> to a missing extent backref.  This is corruption.
> 
> Yes, but this is an autocorruption of btrfs, which occurred (it seems to me), 
> during a cleanup after a deleting of a snapshoot of an operating system 
> installation (perhaps interrupted by an umount).

Do you have a full history of the kernel versions?

I'm not sure when the corruption happened.
It's completely possible that some old kernel caused the corruption but
not exposed until now.

Also, would you please run "btrfs check --readonly" on the fs?
It should show all corruption.

And if "btrfs check" shows no corruption, then it's completely a bug in
that given kernel (and even current kernel).

Thanks,
Qu

> 
>> It also means that the subvolume is never going to disappear during this
>> mount and 'btrfs subvol sync' will wait forever.
> ]zac[
> 
> In my opinion this is bad.
> 
> The infinite wait occurs during the execution of a backup script, so I will 
> have to find a bypass even for this problem (the sync was a patch to another 
> autocorruption problem).
> 
> Gdb
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-06 14:12         ` Qu Wenruo
@ 2019-01-06 17:57           ` Giuseppe Della Bianca
  2019-01-06 23:55             ` Qu Wenruo
  0 siblings, 1 reply; 20+ messages in thread
From: Giuseppe Della Bianca @ 2019-01-06 17:57 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: Jeff Mahoney, linux-btrfs

In data domenica 6 gennaio 2019 15:12:38 CET, Qu Wenruo ha scritto:
]zac[
> Do you have a full history of the kernel versions?

I believe from version 4.13.9 to 4.17.7 .
Today I updated everything to fedora 29, kernel 4.19.10 .

> I'm not sure when the corruption happened.
> It's completely possible that some old kernel caused the corruption but
> not exposed until now.

I am quite convinced that it happens during the snapshot delete and the 
subsequent cleanup.
And maybe even the umount is part of the problem.

> Also, would you please run "btrfs check --readonly" on the fs?
> It should show all corruption.
> 
> And if "btrfs check" shows no corruption, then it's completely a bug in
> that given kernel (and even current kernel).

btrfs check reported various corruptions and fixed them.
A this time it seems that the filesystem works fine (it's the first time it 
does not have to be redone from scratch).

> Thanks,
> Qu

No, thanks to you for your work.

Gdb



^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-06 17:57           ` Giuseppe Della Bianca
@ 2019-01-06 23:55             ` Qu Wenruo
       [not found]               ` <CAO6awePqby834dBSgLx5r6onmD9HhGWAfN4bno0zK6pU0QjrEQ@mail.gmail.com>
  0 siblings, 1 reply; 20+ messages in thread
From: Qu Wenruo @ 2019-01-06 23:55 UTC (permalink / raw)
  To: Giuseppe Della Bianca; +Cc: Jeff Mahoney, linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 1439 bytes --]



On 2019/1/7 上午1:57, Giuseppe Della Bianca wrote:
> In data domenica 6 gennaio 2019 15:12:38 CET, Qu Wenruo ha scritto:
> ]zac[
>> Do you have a full history of the kernel versions?
> 
> I believe from version 4.13.9 to 4.17.7 .
> Today I updated everything to fedora 29, kernel 4.19.10 .
> 
>> I'm not sure when the corruption happened.
>> It's completely possible that some old kernel caused the corruption but
>> not exposed until now.
> 
> I am quite convinced that it happens during the snapshot delete and the 
> subsequent cleanup.
> And maybe even the umount is part of the problem.

No, I mean the corruption which finally results the hang was there for a
long time.

It's relatively common that extent tree get corrupted before and some
unfortunately operation touching the corrupted extent tree triggered
some user affecting error.

> 
>> Also, would you please run "btrfs check --readonly" on the fs?
>> It should show all corruption.
>>
>> And if "btrfs check" shows no corruption, then it's completely a bug in
>> that given kernel (and even current kernel).
> 
> btrfs check reported various corruptions and fixed them.

Please paste the output if possible.

Thanks,
Qu

> A this time it seems that the filesystem works fine (it's the first time it 
> does not have to be redone from scratch).
> 
>> Thanks,
>> Qu
> 
> No, thanks to you for your work.
> 
> Gdb
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

[parent not found: <CAO6awePqby834dBSgLx5r6onmD9HhGWAfN4bno0zK6pU0QjrEQ@mail.gmail.com>]

* Fwd: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
       [not found]               ` <CAO6awePqby834dBSgLx5r6onmD9HhGWAfN4bno0zK6pU0QjrEQ@mail.gmail.com>
@ 2019-01-07 12:55                 ` gius db
  2019-01-07 13:31                   ` Qu Wenruo
  0 siblings, 1 reply; 20+ messages in thread
From: gius db @ 2019-01-07 12:55 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs

Il giorno lun 7 gen 2019 alle ore 00:56 Qu Wenruo
<quwenruo.btrfs@gmx.com> ha scritto:
]zac[
> > I am quite convinced that it happens during the snapshot delete and the
> > subsequent cleanup.
> > And maybe even the umount is part of the problem.
>
> No, I mean the corruption which finally results the hang was there for a
> long time.
>
> It's relatively common that extent tree get corrupted before and some
> unfortunately operation touching the corrupted extent tree triggered
> some user affecting error.

Yes, I understand, but the use of filesytem is very specific.

This filesystem and others that have had problems with corruption, are
used only as backups.
So the only operations that are performed are snapshot receive,
snapshot create, snapshot delete.
After the operations are finished, the filesystem is unmounted.

It may just be a coincidence, but the problems of corruptions have
occurred very often after a snapshoot delete.

The other filesystems that are used in a generic way (operating
system, warehouse and data processing, snapshot creating etc.), have
never given problems.

]zac[
> >
> > btrfs check reported various corruptions and fixed them.
>
> Please paste the output if possible.
]zac[

Sorry, I didn't think to save the btrfs check messages.

Gdb

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: Fwd: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-07 12:55                 ` Fwd: " gius db
@ 2019-01-07 13:31                   ` Qu Wenruo
       [not found]                     ` <CAO6aweMu9HUn34406Kkh-UvoDyoJH2ZdGUQx3vdx1Rj955E4KQ@mail.gmail.com>
  0 siblings, 1 reply; 20+ messages in thread
From: Qu Wenruo @ 2019-01-07 13:31 UTC (permalink / raw)
  To: gius db; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 1724 bytes --]



On 2019/1/7 下午8:55, gius db wrote:
> Il giorno lun 7 gen 2019 alle ore 00:56 Qu Wenruo
> <quwenruo.btrfs@gmx.com> ha scritto:
> ]zac[
>>> I am quite convinced that it happens during the snapshot delete and the
>>> subsequent cleanup.
>>> And maybe even the umount is part of the problem.
>>
>> No, I mean the corruption which finally results the hang was there for a
>> long time.
>>
>> It's relatively common that extent tree get corrupted before and some
>> unfortunately operation touching the corrupted extent tree triggered
>> some user affecting error.
> 
> Yes, I understand, but the use of filesytem is very specific.
> 
> This filesystem and others that have had problems with corruption, are
> used only as backups.
> So the only operations that are performed are snapshot receive,
> snapshot create, snapshot delete.
> After the operations are finished, the filesystem is unmounted.
> 
> It may just be a coincidence, but the problems of corruptions have
> occurred very often after a snapshoot delete.

I think this should give us a pretty good clue.

Specific workload, less active usage, and normally no concurrency.

And for the backup usage, you're using relatively new kernel only, right?

Then this should be something taking into consideration for stress test.

Thanks,
Qu

> 
> The other filesystems that are used in a generic way (operating
> system, warehouse and data processing, snapshot creating etc.), have
> never given problems.
> 
> ]zac[
>>>
>>> btrfs check reported various corruptions and fixed them.
>>
>> Please paste the output if possible.
> ]zac[
> 
> Sorry, I didn't think to save the btrfs check messages.
> 
> 
> Gdb
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

[parent not found: <CAO6aweMu9HUn34406Kkh-UvoDyoJH2ZdGUQx3vdx1Rj955E4KQ@mail.gmail.com>]

* Fwd: Fwd: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
       [not found]                     ` <CAO6aweMu9HUn34406Kkh-UvoDyoJH2ZdGUQx3vdx1Rj955E4KQ@mail.gmail.com>
@ 2019-01-07 17:53                       ` gius db
  0 siblings, 0 replies; 20+ messages in thread
From: gius db @ 2019-01-07 17:53 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs

Il giorno lun 7 gen 2019 alle ore 14:31 Qu Wenruo
<quwenruo.btrfs@gmx.com> ha scritto:
]zac[
> >> It's relatively common that extent tree get corrupted before and some
> >> unfortunately operation touching the corrupted extent tree triggered
> >> some user affecting error.
> >
> > Yes, I understand, but the use of filesytem is very specific.
> >
> > This filesystem and others that have had problems with corruption, are
> > used only as backups.
> > So the only operations that are performed are snapshot receive,
> > snapshot create, snapshot delete.
> > After the operations are finished, the filesystem is unmounted.
> >
> > It may just be a coincidence, but the problems of corruptions have
> > occurred very often after a snapshoot delete.
>
> I think this should give us a pretty good clue.
>
> Specific workload, less active usage, and normally no concurrency.
>
> And for the backup usage, you're using relatively new kernel only, right?

Yes, until a few days ago kernel 4.13.9 - 4.17.7 , now kernel 4.19.10 .

> Then this should be something taking into consideration for stress test.
]zac[

I have never been able to reproduce the condition that causes the
problem, currently I wait a few months until the problem decides to
give news of itself.

Gdb

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-05 12:30       ` Giuseppe Della Bianca
  2019-01-06 14:12         ` Qu Wenruo
@ 2019-01-07 22:40         ` Jeff Mahoney
  2019-01-08 21:02           ` Giuseppe Della Bianca
  1 sibling, 1 reply; 20+ messages in thread
From: Jeff Mahoney @ 2019-01-07 22:40 UTC (permalink / raw)
  To: Giuseppe Della Bianca; +Cc: linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 2182 bytes --]

On 1/5/19 7:30 AM, Giuseppe Della Bianca wrote:
> In data venerdì 4 gennaio 2019 21:34:03 CET, Jeff Mahoney ha scritto:
> ]zac[
>>>
>>> root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2
>>> /
>>> tmp/tmp.p9SiQ1GnpV
>>>
>>> cat /proc/17133/stack
>>> [<0>] hrtimer_nanosleep+0xce/0x1e0
>>> [<0>] __x64_sys_nanosleep+0x77/0xa0
>>> [<0>] do_syscall_64+0x5b/0x160
>>> [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> [<0>] 0xffffffffffffffff
>>
>> Ok, so this is it just sleeping between tree searches.
> ]zac[
>> This part is actually important since we see below that we're searching
>> for bytenr 76428623872 which, if present, would be in the cut portion of
>> your log.
> ]zac[
> 
> If you want, I can send you the full log (very long).
> From what you wrote below it seems to me that you do not need it

Please do.  It would be good to see what the state of the extent tree is
there.

> ]zac]
>>
>> This is the more important part.  The file system has gone read-only due
>> to a missing extent backref.  This is corruption.
> 
> Yes, but this is an autocorruption of btrfs, which occurred (it seems to me), 
> during a cleanup after a deleting of a snapshoot of an operating system 
> installation (perhaps interrupted by an umount).
> 
>> It also means that the subvolume is never going to disappear during this
>> mount and 'btrfs subvol sync' will wait forever.
> ]zac[
> 
> In my opinion this is bad.

Agreed.  I was describing what the situation is, not how it should be.

> The infinite wait occurs during the execution of a backup script, so I will 
> have to find a bypass even for this problem (the sync was a patch to another 
> autocorruption problem).

It's something that needs fixing.  The question is how to go about that.
 My first take on it is to have that loop also check whether the file
system is read-only.  That will cover the file system being taken
read-only due to failure, but is also general enough that it'll cover
the case where it can't possibly succeed.  For example, if the file
system is mounted read-only intentionally.

-Jeff
-- 
Jeff Mahoney
SUSE Labs



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-07 22:40         ` Jeff Mahoney
@ 2019-01-08 21:02           ` Giuseppe Della Bianca
  2019-01-08 21:18             ` Jeff Mahoney
  0 siblings, 1 reply; 20+ messages in thread
From: Giuseppe Della Bianca @ 2019-01-08 21:02 UTC (permalink / raw)
  To: Jeff Mahoney; +Cc: linux-btrfs

In data lunedì 7 gennaio 2019 23:40:19 CET, Jeff Mahoney ha scritto:
]zac[
> > If you want, I can send you the full log (very long).
> > From what you wrote below it seems to me that you do not need it
> 
> Please do.  It would be good to see what the state of the extent tree is
> there.


-- Logs begin at Sat 2016-11-26 18:16:29 CET, end at Tue 2019-01-08 21:39:44 
CET. --
gen 01 17:17:32 exnetold.gdb.it kernel: WARNING: CPU: 0 PID: 16871 at fs/
btrfs/extent-tree.c:7001 __btrfs_free_extent.isra.70+0x782/0xb10 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel: Modules linked in: fuse xt_CHECKSUM 
ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns 
nf_conntrack_broadcast xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 
xt_conntrack devlink ip_set nfnetlink ebtable_nat ebtable_broute bridge stp 
llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle 
ip6table_raw ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 
nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw iptable_security 
ebtable_filter ebtables ip6table_filter ip6_tables it87 hwmon_vid sunrpc 
snd_hda_codec_analog snd_hda_codec_generic snd_hda_codec_hdmi btrfs 
powernow_k8 amd64_edac_mod edac_mce_amd snd_hda_intel snd_hda_codec kvm_amd 
xor ccp zstd_compress snd_hda_core snd_hwdep kvm snd_seq snd_seq_device ppdev 
snd_pcm raid6_pq irqbypass
gen 01 17:17:32 exnetold.gdb.it kernel:  libcrc32c k8temp snd_timer 
zstd_decompress xxhash i2c_nforce2 snd asus_atk0110 soundcore shpchp 
parport_pc parport ata_generic pata_acpi nouveau video mxm_wmi wmi 
i2c_algo_bit drm_kms_helper ttm drm serio_raw sata_nv forcedeth
gen 01 17:17:32 exnetold.gdb.it kernel: CPU: 0 PID: 16871 Comm: btrfs-cleaner 
Not tainted 4.17.19-200.fc28.x86_64 #1
gen 01 17:17:32 exnetold.gdb.it kernel: Hardware name: System manufacturer 
System Product Name/M2N, BIOS 0902    02/16/2009
gen 01 17:17:32 exnetold.gdb.it kernel: RIP: 0010:__btrfs_free_extent.isra.
70+0x782/0xb10 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel: RSP: 0018:ffffa63985f8fc30 EFLAGS: 
00010246
gen 01 17:17:32 exnetold.gdb.it kernel: RAX: 00000000fffffffe RBX: 
ffff8b71313f8380 RCX: 0000000000000000
gen 01 17:17:32 exnetold.gdb.it kernel: RDX: 0000000000000000 RSI: 
0000000000000000 RDI: ffff8b711d8ef578
gen 01 17:17:32 exnetold.gdb.it kernel: RBP: 00000011cb804000 R08: 
ffffa63985f8fba4 R09: 0000000000000018
gen 01 17:17:32 exnetold.gdb.it kernel: R10: 0000000000000835 R11: 
0000000000000000 R12: ffff8b715f5d4000
gen 01 17:17:32 exnetold.gdb.it kernel: R13: 00000000fffffffe R14: 
0000000000000000 R15: 00000000000005aa
gen 01 17:17:32 exnetold.gdb.it kernel: FS:  0000000000000000(0000) 
GS:ffff8b717fa00000(0000) knlGS:0000000000000000
gen 01 17:17:32 exnetold.gdb.it kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 
0000000080050033
gen 01 17:17:32 exnetold.gdb.it kernel: CR2: 00007fdfdffffa08 CR3: 
00000000b05b0000 CR4: 00000000000006f0
gen 01 17:17:32 exnetold.gdb.it kernel: Call Trace:
gen 01 17:17:32 exnetold.gdb.it kernel:  __btrfs_run_delayed_refs+0x216/0x10b0 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  ? btrfs_free_tree_block+0x82/0x2c0 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_run_delayed_refs+0x78/0x180 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_should_end_transaction+0x3e/
0x60 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_drop_snapshot+0x3cf/0x820 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  ? btree_submit_bio_start+0x20/0x20 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  
btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  cleaner_kthread+0x129/0x160 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  kthread+0x112/0x130
gen 01 17:17:32 exnetold.gdb.it kernel:  ? 
kthread_create_worker_on_cpu+0x70/0x70
gen 01 17:17:32 exnetold.gdb.it kernel:  ret_from_fork+0x35/0x40
gen 01 17:17:32 exnetold.gdb.it kernel: Code: b8 00 00 00 48 8b 7c 24 18 e8 bb 
b7 ff ff 41 89 c5 58 5a c6 44 24 24 00 45 85 ed 0f 84 97 f9 ff ff 41 83 fd fe 
0f 85 63 fc ff ff <0f> 0b 48 8b 3b e8 84 ca 00 00 4d 89 f8 4c 89 f1 48 89 ea 
ff b4 
gen 01 17:17:32 exnetold.gdb.it kernel: ---[ end trace c4eefd1fdc016cea ]---
gen 01 17:17:32 exnetold.gdb.it kernel: BTRFS info (device sda3): leaf 
62678286336 gen 6388 total ptrs 148 free space 6202 owner 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 0 key (76427296768 169 0) 
itemoff 16250 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 1 key (76427313152 169 0) 
itemoff 16217 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 2 key (76427329536 169 0) 
itemoff 16184 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 3 key (76427345920 169 0) 
itemoff 16151 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 4 key (76427362304 169 0) 
itemoff 16118 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 5 key (76427378688 169 0) 
itemoff 16085 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 6 key (76427395072 169 0) 
itemoff 16025 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 7 key (76427411456 169 0) 
itemoff 15992 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 8 key (76427427840 169 0) 
itemoff 15959 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 9 key (76427444224 169 0) 
itemoff 15926 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 10 key (76427460608 169 
0) itemoff 15893 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 11 key (76427476992 169 
0) itemoff 15860 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 12 key (76427493376 169 
0) itemoff 15827 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 13 key (76427509760 169 
0) itemoff 15794 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 14 key (76427526144 169 
0) itemoff 15761 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 15 key (76427542528 169 
0) itemoff 15728 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 16 key (76427558912 169 
0) itemoff 15668 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 17 key (76427591680 169 
0) itemoff 15635 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 18 key (76427608064 169 
0) itemoff 15602 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 19 key (76427624448 169 
0) itemoff 15569 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76393332736
gen 01 17:17:32 exnetold.gdb.it kernel:         item 20 key (76427640832 169 
0) itemoff 15536 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 21 key (76427673600 169 
0) itemoff 15503 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 22 key (76427689984 169 
0) itemoff 15470 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76600279040
gen 01 17:17:32 exnetold.gdb.it kernel:         item 23 key (76427706368 169 
0) itemoff 15437 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76553961472
gen 01 17:17:32 exnetold.gdb.it kernel:         item 24 key (76427722752 169 
0) itemoff 15404 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 25 key (76427739136 169 
0) itemoff 15371 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 26 key (76427755520 169 
0) itemoff 15338 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 27 key (76427771904 169 
0) itemoff 15305 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 28 key (76427788288 169 
0) itemoff 15272 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 29 key (76427804672 169 
0) itemoff 15239 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 30 key (76427821056 169 
0) itemoff 15206 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 31 key (76427837440 169 
0) itemoff 15146 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 32 key (76427853824 169 
0) itemoff 15113 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 33 key (76427870208 169 
0) itemoff 15080 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 34 key (76427886592 169 
0) itemoff 15020 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 35 key (76427902976 169 
0) itemoff 14987 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 36 key (76427919360 169 
0) itemoff 14954 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76600279040
gen 01 17:17:32 exnetold.gdb.it kernel:         item 37 key (76427935744 169 
0) itemoff 14921 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76601622528
gen 01 17:17:32 exnetold.gdb.it kernel:         item 38 key (76427952128 169 
0) itemoff 14888 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 39 key (76427968512 169 
0) itemoff 14783 itemsize 105
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 9 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85582036992
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 81998725120
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#4: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#5: shared block 
backref parent 76366790656
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#6: shared block 
backref parent 71382368256
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#7: shared block 
backref parent 71219494912
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#8: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 40 key (76427984896 169 
0) itemoff 14750 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 41 key (76428001280 169 
0) itemoff 14717 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 42 key (76428017664 169 
0) itemoff 14684 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 43 key (76428034048 169 
0) itemoff 14651 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 44 key (76428050432 169 
0) itemoff 14618 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:         item 45 key (76428066816 169 
0) itemoff 14558 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 46 key (76428083200 169 
0) itemoff 14525 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 47 key (76428099584 169 
0) itemoff 14465 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 48 key (76428115968 169 
0) itemoff 14405 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 49 key (76428132352 169 
0) itemoff 14345 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 50 key (76428148736 169 
0) itemoff 14285 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 51 key (76428165120 169 
0) itemoff 14225 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 52 key (76428181504 169 
0) itemoff 14165 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 53 key (76428197888 169 
0) itemoff 14105 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 54 key (76428214272 169 
0) itemoff 14045 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 55 key (76428230656 169 
0) itemoff 13985 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 56 key (76428247040 169 
0) itemoff 13925 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 57 key (76428263424 169 
0) itemoff 13865 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 58 key (76428279808 169 
0) itemoff 13805 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 59 key (76428296192 169 
0) itemoff 13745 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 60 key (76428312576 169 
0) itemoff 13685 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 61 key (76428328960 169 
0) itemoff 13625 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 62 key (76428345344 169 
0) itemoff 13565 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 63 key (76428361728 169 
0) itemoff 13505 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 64 key (76428378112 169 
0) itemoff 13445 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel:         item 65 key (76428394496 169 
0) itemoff 13412 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 66 key (76428410880 169 
0) itemoff 13370 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76606570496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 42349903872
gen 01 17:17:32 exnetold.gdb.it kernel:         item 67 key (76428427264 169 
0) itemoff 13337 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 2214 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 361455812608
gen 01 17:17:32 exnetold.gdb.it kernel:         item 68 key (76428443648 169 
0) itemoff 13295 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76606570496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 42349903872
gen 01 17:17:32 exnetold.gdb.it kernel:         item 69 key (76428460032 169 
0) itemoff 13244 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 70 key (76428476416 169 
0) itemoff 13211 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 71 key (76428492800 169 
0) itemoff 13178 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 72 key (76428509184 169 
0) itemoff 13127 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 73 key (76428525568 169 
0) itemoff 13094 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 74 key (76428541952 169 
0) itemoff 13043 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 75 key (76428558336 169 
0) itemoff 13010 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 76 key (76428574720 169 
0) itemoff 12977 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6357 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 91123597312
gen 01 17:17:32 exnetold.gdb.it kernel:         item 77 key (76428591104 169 
0) itemoff 12944 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 78 key (76428607488 169 
0) itemoff 12911 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6357 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 91123597312
gen 01 17:17:32 exnetold.gdb.it kernel:         item 79 key (76428623872 169 
1) itemoff 12869 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6357 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 1451
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: tree block 
backref root 1446
gen 01 17:17:32 exnetold.gdb.it kernel:         item 80 key (76428640256 169 
0) itemoff 12836 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76545802240
gen 01 17:17:32 exnetold.gdb.it kernel:         item 81 key (76428656640 169 
0) itemoff 12803 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6357 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76428623872
gen 01 17:17:32 exnetold.gdb.it kernel:         item 82 key (76428705792 169 
0) itemoff 12770 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 83 key (76428722176 169 
0) itemoff 12737 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 84 key (76428738560 169 
0) itemoff 12704 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 85 key (76428754944 169 
0) itemoff 12671 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76427132928
gen 01 17:17:32 exnetold.gdb.it kernel:         item 86 key (76428771328 169 
0) itemoff 12647 itemsize 24
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 2542 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:         item 87 key (76428771328 182 
128379305984) itemoff 12647 itemsize 0
gen 01 17:17:32 exnetold.gdb.it kernel:                 shared block backref
gen 01 17:17:32 exnetold.gdb.it kernel:         item 88 key (76428787712 169 
0) itemoff 12614 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 7
gen 01 17:17:32 exnetold.gdb.it kernel:         item 89 key (76428804096 169 
0) itemoff 12581 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 90 key (76428820480 169 
0) itemoff 12521 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 91 key (76428836864 169 
0) itemoff 12461 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 92 key (76428853248 169 
0) itemoff 12428 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 93 key (76428869632 169 
0) itemoff 12395 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 94 key (76428886016 169 
0) itemoff 12335 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 95 key (76428902400 169 
0) itemoff 12275 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 96 key (76428918784 169 
0) itemoff 12215 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 97 key (76428935168 169 
0) itemoff 12182 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 98 key (76428951552 169 
0) itemoff 12122 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 99 key (76428967936 169 
0) itemoff 12062 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 100 key (76428984320 169 
0) itemoff 12002 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 101 key (76429000704 169 
0) itemoff 11942 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 102 key (76429017088 169 
0) itemoff 11909 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 103 key (76429033472 169 
0) itemoff 11876 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 104 key (76429049856 169 
0) itemoff 11816 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 105 key (76429066240 169 
0) itemoff 11756 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 106 key (76429082624 169 
0) itemoff 11696 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 107 key (76429099008 169 
0) itemoff 11636 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 108 key (76429115392 169 
0) itemoff 11576 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 109 key (76429131776 169 
0) itemoff 11516 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 110 key (76429148160 169 
0) itemoff 11483 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 111 key (76429164544 169 
0) itemoff 11423 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 112 key (76429180928 169 
0) itemoff 11363 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 113 key (76429197312 169 
0) itemoff 11303 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 114 key (76429213696 169 
0) itemoff 11270 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 115 key (76429230080 169 
0) itemoff 11210 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 116 key (76429246464 169 
0) itemoff 11177 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 117 key (76429262848 169 
0) itemoff 11117 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 118 key (76429279232 169 
0) itemoff 11057 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 119 key (76429295616 169 
0) itemoff 11024 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 2214 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 361455812608
gen 01 17:17:32 exnetold.gdb.it kernel:         item 120 key (76429312000 169 
0) itemoff 10964 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 121 key (76429328384 169 
0) itemoff 10931 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:         item 122 key (76429344768 169 
0) itemoff 10898 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 2214 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 361455812608
gen 01 17:17:32 exnetold.gdb.it kernel:         item 123 key (76429361152 169 
0) itemoff 10847 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498255872
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76427132928
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441969664
gen 01 17:17:32 exnetold.gdb.it kernel:         item 124 key (76429377536 169 
0) itemoff 10814 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 125 key (76429393920 169 
0) itemoff 10763 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 126 key (76429410304 169 
0) itemoff 10712 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 127 key (76429426688 169 
0) itemoff 10661 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76498026496
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76467716096
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 66441887744
gen 01 17:17:32 exnetold.gdb.it kernel:         item 128 key (76429443072 169 
0) itemoff 10628 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6256 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 129 key (76429459456 169 
0) itemoff 10595 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76427132928
gen 01 17:17:32 exnetold.gdb.it kernel:         item 130 key (76429475840 169 
0) itemoff 10553 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76427132928
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 66441969664
gen 01 17:17:32 exnetold.gdb.it kernel:         item 131 key (76429492224 169 
0) itemoff 10520 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 132 key (76429508608 169 
0) itemoff 10478 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6277 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81665769472
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 66383904768
gen 01 17:17:32 exnetold.gdb.it kernel:         item 133 key (76429524992 169 
0) itemoff 10445 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 134 key (76429541376 169 
0) itemoff 10394 itemsize 51
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 3 gen 6278 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81814142976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 66339717120
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 62659117056
gen 01 17:17:32 exnetold.gdb.it kernel:         item 135 key (76429557760 169 
0) itemoff 10361 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 136 key (76429574144 169 
0) itemoff 10328 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 137 key (76429590528 169 
0) itemoff 10295 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6384 
flags 2
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 2
gen 01 17:17:32 exnetold.gdb.it kernel:         item 138 key (76429606912 169 
0) itemoff 10262 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6341 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 85760114688
gen 01 17:17:32 exnetold.gdb.it kernel:         item 139 key (76429623296 169 
0) itemoff 10229 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76425297920
gen 01 17:17:32 exnetold.gdb.it kernel:         item 140 key (76429639680 169 
0) itemoff 10169 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6258 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76644990976
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76605489152
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76590530560
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 71136968704
gen 01 17:17:32 exnetold.gdb.it kernel:         item 141 key (76429656064 169 
0) itemoff 10136 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76425297920
gen 01 17:17:32 exnetold.gdb.it kernel:         item 142 key (76429672448 169 
0) itemoff 10103 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76425297920
gen 01 17:17:32 exnetold.gdb.it kernel:         item 143 key (76429688832 169 
0) itemoff 10070 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76425297920
gen 01 17:17:32 exnetold.gdb.it kernel:         item 144 key (76429705216 169 
0) itemoff 10037 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 76425297920
gen 01 17:17:32 exnetold.gdb.it kernel:         item 145 key (76429721600 169 
1) itemoff 10004 itemsize 33
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 1 gen 6303 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: tree block 
backref root 1434
gen 01 17:17:32 exnetold.gdb.it kernel:         item 146 key (76429737984 169 
0) itemoff 9962 itemsize 42
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 2 gen 6277 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81665769472
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 66383904768
gen 01 17:17:32 exnetold.gdb.it kernel:         item 147 key (76429754368 169 
0) itemoff 9902 itemsize 60
gen 01 17:17:32 exnetold.gdb.it kernel:                 extent refs 4 gen 6256 
flags 258
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#0: shared block 
backref parent 81731174400
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#1: shared block 
backref parent 76512641024
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#2: shared block 
backref parent 76450250752
gen 01 17:17:32 exnetold.gdb.it kernel:                 ref#3: shared block 
backref parent 66397339648
gen 01 17:17:32 exnetold.gdb.it kernel: BTRFS error (device sda3): unable to 
find ref byte nr 76428623872 parent 0 root 1450  owner 1 offset 0
gen 01 17:17:32 exnetold.gdb.it kernel: ------------[ cut here ]------------
gen 01 17:17:32 exnetold.gdb.it kernel: BTRFS: Transaction aborted (error -2)
gen 01 17:17:32 exnetold.gdb.it kernel: WARNING: CPU: 0 PID: 16871 at fs/
btrfs/extent-tree.c:7007 __btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel: Modules linked in: fuse xt_CHECKSUM 
ipt_MASQUERADE nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns 
nf_conntrack_broadcast xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 
xt_conntrack devlink ip_set nfnetlink ebtable_nat ebtable_broute bridge stp 
llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle 
ip6table_raw ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 
nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_raw iptable_security 
ebtable_filter ebtables ip6table_filter ip6_tables it87 hwmon_vid sunrpc 
snd_hda_codec_analog snd_hda_codec_generic snd_hda_codec_hdmi btrfs 
powernow_k8 amd64_edac_mod edac_mce_amd snd_hda_intel snd_hda_codec kvm_amd 
xor ccp zstd_compress snd_hda_core snd_hwdep kvm snd_seq snd_seq_device ppdev 
snd_pcm raid6_pq irqbypass
gen 01 17:17:32 exnetold.gdb.it kernel:  libcrc32c k8temp snd_timer 
zstd_decompress xxhash i2c_nforce2 snd asus_atk0110 soundcore shpchp 
parport_pc parport ata_generic pata_acpi nouveau video mxm_wmi wmi 
i2c_algo_bit drm_kms_helper ttm drm serio_raw sata_nv forcedeth
gen 01 17:17:32 exnetold.gdb.it kernel: CPU: 0 PID: 16871 Comm: btrfs-cleaner 
Tainted: G        W         4.17.19-200.fc28.x86_64 #1
gen 01 17:17:32 exnetold.gdb.it kernel: Hardware name: System manufacturer 
System Product Name/M2N, BIOS 0902    02/16/2009
gen 01 17:17:32 exnetold.gdb.it kernel: RIP: 0010:__btrfs_free_extent.isra.
70+0x7e2/0xb10 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel: RSP: 0018:ffffa63985f8fc30 EFLAGS: 
00010286
gen 01 17:17:32 exnetold.gdb.it kernel: RAX: 0000000000000000 RBX: 
ffff8b71313f8380 RCX: 0000000000000006
gen 01 17:17:32 exnetold.gdb.it kernel: RDX: 0000000000000007 RSI: 
0000000000000086 RDI: ffff8b717fa16930
gen 01 17:17:32 exnetold.gdb.it kernel: RBP: 00000011cb804000 R08: 
0000000000000038 R09: 000000000000059a
gen 01 17:17:32 exnetold.gdb.it kernel: R10: 0000000000000000 R11: 
0000000000000001 R12: ffff8b715f5d4000
gen 01 17:17:32 exnetold.gdb.it kernel: R13: 00000000fffffffe R14: 
0000000000000000 R15: 00000000000005aa
gen 01 17:17:32 exnetold.gdb.it kernel: FS:  0000000000000000(0000) 
GS:ffff8b717fa00000(0000) knlGS:0000000000000000
gen 01 17:17:32 exnetold.gdb.it kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 
0000000080050033
gen 01 17:17:32 exnetold.gdb.it kernel: CR2: 00007fdfdffffa08 CR3: 
00000000b05b0000 CR4: 00000000000006f0
gen 01 17:17:32 exnetold.gdb.it kernel: Call Trace:
gen 01 17:17:32 exnetold.gdb.it kernel:  __btrfs_run_delayed_refs+0x216/0x10b0 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  ? btrfs_free_tree_block+0x82/0x2c0 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_run_delayed_refs+0x78/0x180 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_should_end_transaction+0x3e/
0x60 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  btrfs_drop_snapshot+0x3cf/0x820 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  ? btree_submit_bio_start+0x20/0x20 
[btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  
btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  cleaner_kthread+0x129/0x160 [btrfs]
gen 01 17:17:32 exnetold.gdb.it kernel:  kthread+0x112/0x130
gen 01 17:17:32 exnetold.gdb.it kernel:  ? 
kthread_create_worker_on_cpu+0x70/0x70
gen 01 17:17:32 exnetold.gdb.it kernel:  ret_from_fork+0x35/0x40
gen 01 17:17:32 exnetold.gdb.it kernel: Code: 10 48 8b 40 50 f0 48 0f ba a8 30 
23 00 00 02 0f 92 c0 41 59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 
c0 e8 58 33 a0 e7 <0f> 0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 
28 4d 
gen 01 17:17:32 exnetold.gdb.it kernel: ---[ end trace c4eefd1fdc016ceb ]---
gen 01 17:17:32 exnetold.gdb.it kernel: BTRFS: error (device sda3) in 
__btrfs_free_extent:7007: errno=-2 No such entry
gen 01 17:17:32 exnetold.gdb.it kernel: BTRFS info (device sda3): forced 
readonly
gen 01 17:17:33 exnetold.gdb.it kernel: BTRFS: error (device sda3) in 
btrfs_run_delayed_refs:3076: errno=-2 No such entry


]zac[
> >> It also means that the subvolume is never going to disappear during this
> >> mount and 'btrfs subvol sync' will wait forever.
> > 
> > ]zac[
> > 
> > In my opinion this is bad.
> 
> Agreed.  I was describing what the situation is, not how it should be.

Ok, sorry ( :) ).

]zac[
> It's something that needs fixing.  The question is how to go about that.
>  My first take on it is to have that loop also check whether the file
]zac[

I don't know how btrfs works inside.
I just suppose that somewhere in the code an error state is not handled 
correctly (which can be a file or the whole filesystem not accessible).


Gdb



^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-08 21:02           ` Giuseppe Della Bianca
@ 2019-01-08 21:18             ` Jeff Mahoney
  2019-01-08 21:55               ` Giuseppe Della Bianca
  0 siblings, 1 reply; 20+ messages in thread
From: Jeff Mahoney @ 2019-01-08 21:18 UTC (permalink / raw)
  To: Giuseppe Della Bianca; +Cc: linux-btrfs

[-- Attachment #1.1: Type: text/plain, Size: 1963 bytes --]

On 1/8/19 4:02 PM, Giuseppe Della Bianca wrote:
> In data lunedì 7 gennaio 2019 23:40:19 CET, Jeff Mahoney ha scritto:
> ]zac[
>>> If you want, I can send you the full log (very long).
>>> From what you wrote below it seems to me that you do not need it
>>
>> Please do.  It would be good to see what the state of the extent tree is
>> there.
> 
> 
> -- Logs begin at Sat 2016-11-26 18:16:29 CET, end at Tue 2019-01-08 21:39:44 

Thanks.  As it turns out, since Filipe's identified the fix for this
issue, we don't need it anymore.  If that turns out to be a different
issue, we can revisit the log.

> ]zac[
>>>> It also means that the subvolume is never going to disappear during this
>>>> mount and 'btrfs subvol sync' will wait forever.
>>>
>>> ]zac[
>>>
>>> In my opinion this is bad.
>>
>> Agreed.  I was describing what the situation is, not how it should be.
> 
> Ok, sorry ( :) ).
> 
> ]zac[
>> It's something that needs fixing.  The question is how to go about that.
>>  My first take on it is to have that loop also check whether the file
> ]zac[
> 
> I don't know how btrfs works inside.
> I just suppose that somewhere in the code an error state is not handled 
> correctly (which can be a file or the whole filesystem not accessible).

Kind of, if you look at it under the right light.  The sync subcommand
only waits for the lookup for the subvolume undergoing deletion to not
be there anymore.  There's no error state to be passed back to the
waiter, only the initiator.

The command will hang any time a situation arises that the subvolume
won't go away.  That includes:
1) Read-only file system, intentionally
2) Read-only file system as part of a failure
3) Specifying a subvolume that isn't being deleted

The first two mean that the condition can never be met.  The latter is
something of a special case since the user could remove it at any time.

-Jeff
-- 
Jeff Mahoney
SUSE Labs

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-08 21:18             ` Jeff Mahoney
@ 2019-01-08 21:55               ` Giuseppe Della Bianca
  0 siblings, 0 replies; 20+ messages in thread
From: Giuseppe Della Bianca @ 2019-01-08 21:55 UTC (permalink / raw)
  To: Jeff Mahoney; +Cc: linux-btrfs

In data martedì 8 gennaio 2019 22:18:11 CET, Jeff Mahoney ha scritto:

]zac[
> Thanks.  As it turns out, since Filipe's identified the fix for this
> issue, we don't need it anymore.  If that turns out to be a different
> issue, we can revisit the log.

Ok.

]zac[
> > I don't know how btrfs works inside.
> > I just suppose that somewhere in the code an error state is not handled
> > correctly (which can be a file or the whole filesystem not accessible).
> 
> Kind of, if you look at it under the right light.  The sync subcommand
> only waits for the lookup for the subvolume undergoing deletion to not
> be there anymore.  There's no error state to be passed back to the
> waiter, only the initiator.
]zac[

I understand.
But this is a design error in error handling.
The sync subcommand should also check a flag that the deletion system could 
set in case it is no longer possible to continue the deletion.


Gdb




^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-01 16:37   ` Giuseppe Della Bianca
  2019-01-04 20:34     ` Jeff Mahoney
@ 2019-01-07 23:11     ` Filipe Manana
  2019-01-08 12:14       ` gius db
  1 sibling, 1 reply; 20+ messages in thread
From: Filipe Manana @ 2019-01-07 23:11 UTC (permalink / raw)
  To: Giuseppe Della Bianca; +Cc: Jeff Mahoney, linux-btrfs

On Tue, Jan 1, 2019 at 7:07 PM Giuseppe Della Bianca <giusdbg@gmail.com> wrote:
>
> In data giovedì 9 agosto 2018 20:48:03 CEST, Jeff Mahoney ha scritto:
> > On 8/9/18 11:15 AM, Giuseppe Della Bianca wrote:
> > > Hi.
> > >
> > > My system:
> > > - Fedora 28 x86_64
> > > - kernel-4.17.7-200
> > > - btrfs-progs-4.15.1-1
> > >
> > > The command 'btrfs subvolume sync -s 2 xyz' hangs in this case:
> > >
> > > - Run command 'btrfs subvolume sync -s 2 xyz' .
> > > - After some time the kernel reports an error on the filesystem.
> > >
> > >   (error that existed before the command was launched.)
> > >
> > > - The filesystem goes in read-only mode.
> > > - The command hangs.
> >
> > Can you provide the output of 'dmesg' and the contents of
> > /proc/<pid>/stack where <pid> is the pid of the btrfs command process?
> >
> > -Jeff
>
> I had the problem again.
> This is the information you asked for:
>
> root     17133 17127  0 17:17 ?        00:00:00 btrfs subvolume sync -s 2 /
> tmp/tmp.p9SiQ1GnpV
>
> cat /proc/17133/stack
> [<0>] hrtimer_nanosleep+0xce/0x1e0
> [<0>] __x64_sys_nanosleep+0x77/0xa0
> [<0>] do_syscall_64+0x5b/0x160
> [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [<0>] 0xffffffffffffffff
>
> ...............
> [ 1558.053812] CPU: 0 PID: 16871 Comm: btrfs-cleaner Not tainted
> 4.17.19-200.fc28.x86_64 #1
> [ 1558.053815] Hardware name: System manufacturer System Product Name/M2N,
> BIOS 0902    02/16/2009
> [ 1558.053904] RIP: 0010:__btrfs_free_extent.isra.70+0x782/0xb10 [btrfs]
> [ 1558.053907] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010246
> [ 1558.053912] RAX: 00000000fffffffe RBX: ffff8b71313f8380 RCX:
> 0000000000000000
> [ 1558.053914] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> ffff8b711d8ef578
> [ 1558.053916] RBP: 00000011cb804000 R08: ffffa63985f8fba4 R09:
> 0000000000000018
> [ 1558.053919] R10: 0000000000000835 R11: 0000000000000000 R12:
> ffff8b715f5d4000
> [ 1558.053921] R13: 00000000fffffffe R14: 0000000000000000 R15:
> 00000000000005aa
> [ 1558.053925] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
> 0000000000000000
> [ 1558.053928] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1558.053930] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4:
> 00000000000006f0
> [ 1558.053934] Call Trace:
> [ 1558.054011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> [ 1558.054011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> [ 1558.054011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> [ 1558.054011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> [ 1558.054011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> [ 1558.054011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [ 1558.054011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> [ 1558.054011]  cleaner_kthread+0x129/0x160 [btrfs]
> [ 1558.054011]  kthread+0x112/0x130
> [ 1558.054011]  ? kthread_create_worker_on_cpu+0x70/0x70
> [ 1558.054011]  ret_from_fork+0x35/0x40
> [ 1558.054011] Code: b8 00 00 00 48 8b 7c 24 18 e8 bb b7 ff ff 41 89 c5 58 5a
> c6 44 24 24 00 45 85 ed 0f 84 97 f9 ff ff 41 83 fd fe 0f 85 63 fc ff ff <0f>
> 0b 48 8b 3b e8 84 ca 00 00 4d 89 f8 4c 89 f1 48 89 ea ff b4
> [ 1558.054011] ---[ end trace c4eefd1fdc016cea ]---
> [ 1558.054832] BTRFS info (device sda3): leaf 62678286336 gen 6388 total ptrs
> 148 free space 6202 owner 2
> [ 1558.054837]  item 0 key (76427296768 169 0) itemoff 16250 itemsize 33
> [ 1558.054841]          extent refs 1 gen 6256 flags 258
> [ 1558.054844]          ref#0: shared block backref parent 76512641024
> [ 1558.054850]  item 1 key (76427313152 169 0) itemoff 16217 itemsize 33
> [ 1558.054853]          extent refs 1 gen 6256 flags 258
> [ 1558.054854]          ref#0: shared block backref parent 76512641024
>
> ]zac[
> ...................................
> ]zac[
>
> [ 1558.056504]  item 145 key (76429721600 169 1) itemoff 10004 itemsize 33
> [ 1558.056506]          extent refs 1 gen 6303 flags 258
> [ 1558.056508]          ref#0: tree block backref root 1434
> [ 1558.056512]  item 146 key (76429737984 169 0) itemoff 9962 itemsize 42
> [ 1558.056515]          extent refs 2 gen 6277 flags 258
> [ 1558.056516]          ref#0: shared block backref parent 81665769472
> [ 1558.056519]          ref#1: shared block backref parent 66383904768
> [ 1558.056524]  item 147 key (76429754368 169 0) itemoff 9902 itemsize 60
> [ 1558.056526]          extent refs 4 gen 6256 flags 258
> [ 1558.056528]          ref#0: shared block backref parent 81731174400
> [ 1558.056531]          ref#1: shared block backref parent 76512641024
> [ 1558.056533]          ref#2: shared block backref parent 76450250752
> [ 1558.056536]          ref#3: shared block backref parent 66397339648
> [ 1558.056544] BTRFS error (device sda3): unable to find ref byte nr
> 76428623872 parent 0 root 1450  owner 1 offset 0
> [ 1558.056554] ------------[ cut here ]------------
> [ 1558.056556] BTRFS: Transaction aborted (error -2)
> [ 1558.056718] WARNING: CPU: 0 PID: 16871 at fs/btrfs/extent-tree.c:7007
> __btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
> [ 1558.056720] Modules linked in: fuse xt_CHECKSUM ipt_MASQUERADE
> nf_nat_masquerade_ipv4 tun nf_conntrack_netbios_ns nf_conntrack_broadcast
> xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 xt_conntrack devlink ip_set
> nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat
> nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw
> ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4
> nf_nat nf_conntrack iptable_mangle iptable_raw iptable_security ebtable_filter
> ebtables ip6table_filter ip6_tables it87 hwmon_vid sunrpc snd_hda_codec_analog
> snd_hda_codec_generic snd_hda_codec_hdmi btrfs powernow_k8 amd64_edac_mod
> edac_mce_amd snd_hda_intel snd_hda_codec kvm_amd xor ccp zstd_compress
> snd_hda_core snd_hwdep kvm snd_seq snd_seq_device ppdev snd_pcm raid6_pq
> irqbypass
> [ 1558.056785]  libcrc32c k8temp snd_timer zstd_decompress xxhash i2c_nforce2
> snd asus_atk0110 soundcore shpchp parport_pc parport ata_generic pata_acpi
> nouveau video mxm_wmi wmi i2c_algo_bit drm_kms_helper ttm drm serio_raw
> sata_nv forcedeth
> [ 1558.056819] CPU: 0 PID: 16871 Comm: btrfs-cleaner Tainted: G        W
> 4.17.19-200.fc28.x86_64 #1
> [ 1558.056821] Hardware name: System manufacturer System Product Name/M2N,
> BIOS 0902    02/16/2009
> [ 1558.056905] RIP: 0010:__btrfs_free_extent.isra.70+0x7e2/0xb10 [btrfs]
> [ 1558.056908] RSP: 0018:ffffa63985f8fc30 EFLAGS: 00010286
> [ 1558.056912] RAX: 0000000000000000 RBX: ffff8b71313f8380 RCX:
> 0000000000000006
> [ 1558.056914] RDX: 0000000000000007 RSI: 0000000000000086 RDI:
> ffff8b717fa16930
> [ 1558.056916] RBP: 00000011cb804000 R08: 0000000000000038 R09:
> 000000000000059a
> [ 1558.056918] R10: 0000000000000000 R11: 0000000000000001 R12:
> ffff8b715f5d4000
> [ 1558.056920] R13: 00000000fffffffe R14: 0000000000000000 R15:
> 00000000000005aa
> [ 1558.056924] FS:  0000000000000000(0000) GS:ffff8b717fa00000(0000) knlGS:
> 0000000000000000
> [ 1558.056927] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1558.056929] CR2: 00007fdfdffffa08 CR3: 00000000b05b0000 CR4:
> 00000000000006f0
> [ 1558.056931] Call Trace:
> [ 1558.057011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> [ 1558.057011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> [ 1558.057011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> [ 1558.057011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> [ 1558.057011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> [ 1558.057011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> [ 1558.057011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> [ 1558.057011]  cleaner_kthread+0x129/0x160 [btrfs]
> [ 1558.057011]  kthread+0x112/0x130
> [ 1558.057011]  ? kthread_create_worker_on_cpu+0x70/0x70
> [ 1558.057011]  ret_from_fork+0x35/0x40
> [ 1558.057011] Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 02 0f 92 c0 41
> 59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 c0 e8 58 33 a0 e7 <0f>
> 0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d
> [ 1558.057011] ---[ end trace c4eefd1fdc016ceb ]---
> [ 1558.057734] BTRFS: error (device sda3) in __btrfs_free_extent:7007:
> errno=-2 No such entry
> [ 1558.057742] BTRFS info (device sda3): forced readonly
> [ 1558.057757] BTRFS: error (device sda3) in btrfs_run_delayed_refs:3076:
> errno=-2 No such entry

Very recently, a fix for this went into kernel 5.0-rc1:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0568e82dbe2510fc1fa664f58e5c997d3f1e649e

>
> gdb
>
>


-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-07 23:11     ` Filipe Manana
@ 2019-01-08 12:14       ` gius db
  2019-01-08 12:29         ` Filipe Manana
  0 siblings, 1 reply; 20+ messages in thread
From: gius db @ 2019-01-08 12:14 UTC (permalink / raw)
  To: fdmanana; +Cc: Jeff Mahoney, linux-btrfs

Il giorno mar 8 gen 2019 alle ore 00:11 Filipe Manana
<fdmanana@gmail.com> ha scritto:

]zac[
> > [ 1558.056931] Call Trace:
> > [ 1558.057011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> > [ 1558.057011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> > [ 1558.057011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> > [ 1558.057011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> > [ 1558.057011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> > [ 1558.057011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> > [ 1558.057011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> > [ 1558.057011]  cleaner_kthread+0x129/0x160 [btrfs]
> > [ 1558.057011]  kthread+0x112/0x130
> > [ 1558.057011]  ? kthread_create_worker_on_cpu+0x70/0x70
> > [ 1558.057011]  ret_from_fork+0x35/0x40
> > [ 1558.057011] Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 02 0f 92 c0 41
> > 59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 c0 e8 58 33 a0 e7 <0f>
> > 0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d
> > [ 1558.057011] ---[ end trace c4eefd1fdc016ceb ]---
> > [ 1558.057734] BTRFS: error (device sda3) in __btrfs_free_extent:7007:
> > errno=-2 No such entry
> > [ 1558.057742] BTRFS info (device sda3): forced readonly
> > [ 1558.057757] BTRFS: error (device sda3) in btrfs_run_delayed_refs:3076:
> > errno=-2 No such entry

> Very recently, a fix for this went into kernel 5.0-rc1:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0568e82dbe2510fc1fa664f58e5c997d3f1e649e
]zac[

Thanks for the answer.

If I understand correctly the problem is caused by the concurrence of
the cleanup process (which is started a few seconds after the end of
the snapshot deletion) with the other operations on the filesystem.

It matches what I seem to have noticed (problem that occurs after a
snapshot deletion, occurs randomly due to a race condition, etc..).

P.S. I suppose the 4.x.x kernel will not be corrected.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-08 12:14       ` gius db
@ 2019-01-08 12:29         ` Filipe Manana
  2019-01-08 13:01           ` gius db
  0 siblings, 1 reply; 20+ messages in thread
From: Filipe Manana @ 2019-01-08 12:29 UTC (permalink / raw)
  To: gius db; +Cc: Jeff Mahoney, linux-btrfs

On Tue, Jan 8, 2019 at 12:14 PM gius db <giusdbg@gmail.com> wrote:
>
> Il giorno mar 8 gen 2019 alle ore 00:11 Filipe Manana
> <fdmanana@gmail.com> ha scritto:
>
> ]zac[
> > > [ 1558.056931] Call Trace:
> > > [ 1558.057011]  __btrfs_run_delayed_refs+0x216/0x10b0 [btrfs]
> > > [ 1558.057011]  ? btrfs_free_tree_block+0x82/0x2c0 [btrfs]
> > > [ 1558.057011]  btrfs_run_delayed_refs+0x78/0x180 [btrfs]
> > > [ 1558.057011]  btrfs_should_end_transaction+0x3e/0x60 [btrfs]
> > > [ 1558.057011]  btrfs_drop_snapshot+0x3cf/0x820 [btrfs]
> > > [ 1558.057011]  ? btree_submit_bio_start+0x20/0x20 [btrfs]
> > > [ 1558.057011]  btrfs_clean_one_deleted_snapshot+0xba/0xe0 [btrfs]
> > > [ 1558.057011]  cleaner_kthread+0x129/0x160 [btrfs]
> > > [ 1558.057011]  kthread+0x112/0x130
> > > [ 1558.057011]  ? kthread_create_worker_on_cpu+0x70/0x70
> > > [ 1558.057011]  ret_from_fork+0x35/0x40
> > > [ 1558.057011] Code: 10 48 8b 40 50 f0 48 0f ba a8 30 23 00 00 02 0f 92 c0 41
> > > 59 84 c0 0f 85 cc a4 09 00 44 89 ee 48 c7 c7 98 9e 75 c0 e8 58 33 a0 e7 <0f>
> > > 0b e9 b6 a4 09 00 48 89 df e8 bf 0c ff ff 49 8b 74 24 28 4d
> > > [ 1558.057011] ---[ end trace c4eefd1fdc016ceb ]---
> > > [ 1558.057734] BTRFS: error (device sda3) in __btrfs_free_extent:7007:
> > > errno=-2 No such entry
> > > [ 1558.057742] BTRFS info (device sda3): forced readonly
> > > [ 1558.057757] BTRFS: error (device sda3) in btrfs_run_delayed_refs:3076:
> > > errno=-2 No such entry
>
> > Very recently, a fix for this went into kernel 5.0-rc1:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0568e82dbe2510fc1fa664f58e5c997d3f1e649e
> ]zac[
>
> Thanks for the answer.
>
> If I understand correctly the problem is caused by the concurrence of
> the cleanup process (which is started a few seconds after the end of
> the snapshot deletion) with the other operations on the filesystem.
>
> It matches what I seem to have noticed (problem that occurs after a
> snapshot deletion, occurs randomly due to a race condition, etc..).
>
> P.S. I suppose the 4.x.x kernel will not be corrected.

Why do you suppose that?
The patch has a tag (see the bottom of the changelog) to signal it
should be added to stable kernels 4.4+:

CC: stable@vger.kernel.org # 4.4+



-- 
Filipe David Manana,

“Whether you think you can, or you think you can't — you're right.”

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs.
  2019-01-08 12:29         ` Filipe Manana
@ 2019-01-08 13:01           ` gius db
  0 siblings, 0 replies; 20+ messages in thread
From: gius db @ 2019-01-08 13:01 UTC (permalink / raw)
  To: Filipe Manana; +Cc: linux-btrfs

Il giorno mar 8 gen 2019 alle ore 13:29 Filipe Manana
<fdmanana@gmail.com> ha scritto:

]zac[
> > It matches what I seem to have noticed (problem that occurs after a
> > snapshot deletion, occurs randomly due to a race condition, etc..).
> >
> > P.S. I suppose the 4.x.x kernel will not be corrected.
>
> Why do you suppose that?

My mistake, I suppose ( :) ).

> The patch has a tag (see the bottom of the changelog) to signal it
> should be added to stable kernels 4.4+:
>
> CC: stable@vger.kernel.org # 4.4+

I missed it.
Thanks.

Gdb

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2019-01-08 21:56 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-09 15:15 [COMMAND HANGS] The command 'btrfs subvolume sync -s 2 xyz' can hangs Giuseppe Della Bianca
2018-08-09 18:48 ` Jeff Mahoney
2018-08-10 16:57   ` Giuseppe Della Bianca
2019-01-01 16:37   ` Giuseppe Della Bianca
2019-01-04 20:34     ` Jeff Mahoney
2019-01-05 12:30       ` Giuseppe Della Bianca
2019-01-06 14:12         ` Qu Wenruo
2019-01-06 17:57           ` Giuseppe Della Bianca
2019-01-06 23:55             ` Qu Wenruo
     [not found]               ` <CAO6awePqby834dBSgLx5r6onmD9HhGWAfN4bno0zK6pU0QjrEQ@mail.gmail.com>
2019-01-07 12:55                 ` Fwd: " gius db
2019-01-07 13:31                   ` Qu Wenruo
     [not found]                     ` <CAO6aweMu9HUn34406Kkh-UvoDyoJH2ZdGUQx3vdx1Rj955E4KQ@mail.gmail.com>
2019-01-07 17:53                       ` Fwd: " gius db
2019-01-07 22:40         ` Jeff Mahoney
2019-01-08 21:02           ` Giuseppe Della Bianca
2019-01-08 21:18             ` Jeff Mahoney
2019-01-08 21:55               ` Giuseppe Della Bianca
2019-01-07 23:11     ` Filipe Manana
2019-01-08 12:14       ` gius db
2019-01-08 12:29         ` Filipe Manana
2019-01-08 13:01           ` gius db

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).