All of lore.kernel.org
 help / color / mirror / Atom feed
* filesystem corrupt - error -117
       [not found] <em969af203-04e6-4eff-a115-1129ae853867@frystation>
@ 2021-10-25  8:46 ` Mia
  2021-10-25 10:53   ` Qu Wenruo
  0 siblings, 1 reply; 21+ messages in thread
From: Mia @ 2021-10-25  8:46 UTC (permalink / raw)
  To: linux-btrfs

Hello,
I need support since my root filesystem just went readonly :(

[641955.981560] BTRFS error (device sda3): tree block 342685007872 owner 
7 already locked by pid=8099, extent tree corruption detected
[641955.983724] ------------[ cut here ]------------
[641955.983731] BTRFS: Transaction aborted (error -117)
[641955.984030] WARNING: CPU: 0 PID: 8099 at fs/btrfs/inode.c:3131 
btrfs_finish_ordered_io+0x6b4/0x7a0 [btrfs]
[641955.984031] Modules linked in: nfsv3 nfs_acl rpcsec_gss_krb5 
auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache dm_mod veth 
xfrm_user xfrm_algo br_netfilter ip6t_REJECT nf_reject_ipv6 ip6_tables 
nft_chain_route_ipv6 nft_chain_nat_ipv6 ip6t_MASQUERADE nf_nat_ipv6 
nf_log_ipv6 xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment 
xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat 
nft_chain_route_ipv4 aufs(OE) xt_CT nft_counter nfnetlink_log xt_NFLOG 
nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic 
nf_conntrack_snmp nf_nat_sip nf_nat_pptp xt_CHECKSUM nf_nat_proto_gre 
nf_nat_irc ipt_MASQUERADE nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp 
nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp xt_tcpudp 
nft_compat nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre 
nf_conntrack_netlink
[641955.984048]  nf_conntrack_netbios_ns nf_conntrack_broadcast 
nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_chain_nat_ipv4 
nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 
wireguard(E) ip6_udp_tunnel udp_tunnel nf_tables nfnetlink overlay 
bridge stp llc fuse ext4 crc16 mbcache jbd2 fscrypto ecb 
crct10dif_pclmul crc32_pclmul joydev ghash_clmulni_intel serio_raw 
virtio_balloon pcspkr sg virtio_console qemu_fw_cfg evdev sunrpc 
ip_tables x_tables autofs4 btrfs xor zstd_decompress zstd_compress 
xxhash raid6_pq libcrc32c crc32c_generic hid_generic usbhid hid sr_mod 
cdrom sd_mod ata_generic virtio_net net_failover virtio_scsi failover 
crc32c_intel bochs_drm ttm aesni_intel aes_x86_64 drm_kms_helper 
crypto_simd cryptd glue_helper ata_piix psmouse libata uhci_hcd drm 
ehci_hcd scsi_mod
[641955.984073]  usbcore virtio_pci i2c_piix4 virtio_ring floppy virtio 
usb_common button
[641955.984078] CPU: 0 PID: 8099 Comm: kworker/u8:1 Tainted: G           
OE     4.19.0-17-amd64 #1 Debian 4.19.194-3
[641955.984079] Hardware name: netcup KVM Server, BIOS RS 2000 G9 
06/01/2021
[641955.984096] Workqueue: btrfs-endio-write btrfs_endio_write_helper 
[btrfs]
[641955.984114] RIP: 0010:btrfs_finish_ordered_io+0x6b4/0x7a0 [btrfs]
[641955.984115] Code: 49 8b 44 24 50 f0 48 0f ba a8 00 17 00 00 02 72 1b 
41 83 fd fb 0f 84 82 94 07 00 44 89 ee 48 c7 c7 b8 c6 61 c0 e8 f5 8e 97 
e0 <0f> 0b 44 89 e9 ba 3b 0c 00 00 48 c7 c6 d0 01 61 c0 4c 89 e7 e8 19
[641955.984116] RSP: 0018:ffff97e883197dd0 EFLAGS: 00010286
[641955.984117] RAX: 0000000000000000 RBX: ffff8b19259ae0a0 RCX: 
0000000000000006
[641955.984117] RDX: 0000000000000007 RSI: 0000000000000086 RDI: 
ffff8b1c2fa166b0
[641955.984118] RBP: ffff8b19259ade80 R08: 0000000000000a78 R09: 
0000000000000004
[641955.984118] R10: 0000000000000000 R11: 0000000000000001 R12: 
ffff8b1c2bc7e548
[641955.984119] R13: 00000000ffffff8b R14: ffff8b1a3f0e1800 R15: 
ffff8b19358df520
[641955.984121] FS:  0000000000000000(0000) GS:ffff8b1c2fa00000(0000) 
knlGS:0000000000000000
[641955.984122] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[641955.984122] CR2: 00007f9c537966a8 CR3: 0000000427bcc000 CR4: 
0000000000340ef0
[641955.984124] Call Trace:
[641955.984180]  normal_work_helper+0xba/0x380 [btrfs]
[641955.984266]  process_one_work+0x1a7/0x3a0
[641955.984287]  worker_thread+0x30/0x390
[641955.984288]  ? create_worker+0x1a0/0x1a0
[641955.984290]  kthread+0x112/0x130
[641955.984302]  ? kthread_bind+0x30/0x30
[641955.984350]  ret_from_fork+0x22/0x40
[641955.984369] ---[ end trace cc28f56e39ff5127 ]---
[641955.984377] BTRFS: error (device sda3) in 
btrfs_finish_ordered_io:3131: errno=-117 unknown
[641955.986629] BTRFS info (device sda3): forced readonly
[641955.986738] BTRFS error (device sda3): pending csums is 4096


root@rx1 ~ # btrfs fi show
Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
     Total devices 1 FS bytes used 189.12GiB
     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3

root@rx1 ~ # btrfs fi df /
Data, single: total=194.89GiB, used=187.46GiB
System, single: total=32.00MiB, used=48.00KiB
Metadata, single: total=4.16GiB, used=1.65GiB
GlobalReserve, single: total=380.45MiB, used=0.00B

root@rx1 ~ # btrfs --version                                             
                                                                          
                                                :(
btrfs-progs v4.20.1


root@rx1 ~ # uname -a
Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64 
GNU/Linux

Hope someone can help.
Regrads
Mia


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-25  8:46 ` filesystem corrupt - error -117 Mia
@ 2021-10-25 10:53   ` Qu Wenruo
  2021-10-25 10:55     ` Qu Wenruo
       [not found]     ` <69109d24-efa7-b9d1-e1df-c79b3989e7bf@rx2.rx-server.de>
  0 siblings, 2 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-25 10:53 UTC (permalink / raw)
  To: Mia, linux-btrfs



On 2021/10/25 16:46, Mia wrote:
> Hello,
> I need support since my root filesystem just went readonly :(
>
> [641955.981560] BTRFS error (device sda3): tree block 342685007872 owner
> 7 already locked by pid=8099, extent tree corruption detected

This line explains itself.

Your extent tree is no corrupted, thus it allocated a new tree block
which is in fact already hold by other tree.

This means your metadata is no longer protected properly by COW.

"btrfs check" is highly recommended to expose the root cause.

>
> root@rx1 ~ # btrfs fi show
> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>      Total devices 1 FS bytes used 189.12GiB
>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>
> root@rx1 ~ # btrfs fi df /
> Data, single: total=194.89GiB, used=187.46GiB
> System, single: total=32.00MiB, used=48.00KiB
> Metadata, single: total=4.16GiB, used=1.65GiB
> GlobalReserve, single: total=380.45MiB, used=0.00B
>
> root@rx1 ~ # btrfs --version
>                                                 :(
> btrfs-progs v4.20.1
>
>
> root@rx1 ~ # uname -a
> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
> GNU/Linux

This is a little old for btrfs, but I don't think that's the cause.

Thanks,
Qu

>
> Hope someone can help.
> Regrads
> Mia
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-25 10:53   ` Qu Wenruo
@ 2021-10-25 10:55     ` Qu Wenruo
       [not found]     ` <69109d24-efa7-b9d1-e1df-c79b3989e7bf@rx2.rx-server.de>
  1 sibling, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-25 10:55 UTC (permalink / raw)
  To: Mia, linux-btrfs



On 2021/10/25 18:53, Qu Wenruo wrote:
>
>
> On 2021/10/25 16:46, Mia wrote:
>> Hello,
>> I need support since my root filesystem just went readonly :(
>>
>> [641955.981560] BTRFS error (device sda3): tree block 342685007872 owner
>> 7 already locked by pid=8099, extent tree corruption detected
>
> This line explains itself.
>
> Your extent tree is no corrupted, thus it allocated a new tree block

I missed the "w" for the word "now"...

> which is in fact already hold by other tree.
>
> This means your metadata is no longer protected properly by COW.
>
> "btrfs check" is highly recommended to expose the root cause.
>
>>
>> root@rx1 ~ # btrfs fi show
>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>      Total devices 1 FS bytes used 189.12GiB
>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>
>> root@rx1 ~ # btrfs fi df /
>> Data, single: total=194.89GiB, used=187.46GiB
>> System, single: total=32.00MiB, used=48.00KiB
>> Metadata, single: total=4.16GiB, used=1.65GiB
>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>
>> root@rx1 ~ # btrfs --version
>>                                                 :(
>> btrfs-progs v4.20.1
>>
>>
>> root@rx1 ~ # uname -a
>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
>> GNU/Linux
>
> This is a little old for btrfs, but I don't think that's the cause.
>
> Thanks,
> Qu
>
>>
>> Hope someone can help.
>> Regrads
>> Mia
>>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]     ` <69109d24-efa7-b9d1-e1df-c79b3989e7bf@rx2.rx-server.de>
@ 2021-10-25 11:13       ` Mia
  2021-10-25 11:14         ` Qu Wenruo
  0 siblings, 1 reply; 21+ messages in thread
From: Mia @ 2021-10-25 11:13 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs

Hi Qu,

thanks for your response.
Here the output of btrfs check: 
https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2

Thanks,
Mia

------ Originalnachricht ------
Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
Gesendet: 25.10.2021 12:55:46
Betreff: Re: filesystem corrupt - error -117

>
>
>On 2021/10/25 18:53, Qu Wenruo wrote:
>>
>>
>>On 2021/10/25 16:46, Mia wrote:
>>>Hello,
>>>I need support since my root filesystem just went readonly :(
>>>
>>>[641955.981560] BTRFS error (device sda3): tree block 342685007872 owner
>>>7 already locked by pid=8099, extent tree corruption detected
>>
>>This line explains itself.
>>
>>Your extent tree is no corrupted, thus it allocated a new tree block
>
>I missed the "w" for the word "now"...
>
>>which is in fact already hold by other tree.
>>
>>This means your metadata is no longer protected properly by COW.
>>
>>"btrfs check" is highly recommended to expose the root cause.
>>
>>>
>>>root@rx1 ~ # btrfs fi show
>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>      Total devices 1 FS bytes used 189.12GiB
>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>
>>>root@rx1 ~ # btrfs fi df /
>>>Data, single: total=194.89GiB, used=187.46GiB
>>>System, single: total=32.00MiB, used=48.00KiB
>>>Metadata, single: total=4.16GiB, used=1.65GiB
>>>GlobalReserve, single: total=380.45MiB, used=0.00B
>>>
>>>root@rx1 ~ # btrfs --version
>>>                                                 :(
>>>btrfs-progs v4.20.1
>>>
>>>
>>>root@rx1 ~ # uname -a
>>>Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
>>>GNU/Linux
>>
>>This is a little old for btrfs, but I don't think that's the cause.
>>
>>Thanks,
>>Qu
>>
>>>
>>>Hope someone can help.
>>>Regrads
>>>Mia
>>>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-25 11:13       ` Mia
@ 2021-10-25 11:14         ` Qu Wenruo
  2021-10-25 11:18           ` Qu Wenruo
       [not found]           ` <884d76d1-5836-9a91-a39b-41c37441e020@rx2.rx-server.de>
  0 siblings, 2 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-25 11:14 UTC (permalink / raw)
  To: Mia, Qu Wenruo, linux-btrfs



On 2021/10/25 19:13, Mia wrote:
> Hi Qu,
> 
> thanks for your response.
> Here the output of btrfs check: 
> https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2

Unfortunately it's not full, and it's using an old btrfs-progs which can 
cause false alert.

Please use latest btrfs-progs v5.14.2 to re-check.

Thanks,
Qu
> 
> Thanks,
> Mia
> 
> ------ Originalnachricht ------
> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
> Gesendet: 25.10.2021 12:55:46
> Betreff: Re: filesystem corrupt - error -117
> 
>>
>>
>> On 2021/10/25 18:53, Qu Wenruo wrote:
>>>
>>>
>>> On 2021/10/25 16:46, Mia wrote:
>>>> Hello,
>>>> I need support since my root filesystem just went readonly :(
>>>>
>>>> [641955.981560] BTRFS error (device sda3): tree block 342685007872 
>>>> owner
>>>> 7 already locked by pid=8099, extent tree corruption detected
>>>
>>> This line explains itself.
>>>
>>> Your extent tree is no corrupted, thus it allocated a new tree block
>>
>> I missed the "w" for the word "now"...
>>
>>> which is in fact already hold by other tree.
>>>
>>> This means your metadata is no longer protected properly by COW.
>>>
>>> "btrfs check" is highly recommended to expose the root cause.
>>>
>>>>
>>>> root@rx1 ~ # btrfs fi show
>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>
>>>> root@rx1 ~ # btrfs fi df /
>>>> Data, single: total=194.89GiB, used=187.46GiB
>>>> System, single: total=32.00MiB, used=48.00KiB
>>>> Metadata, single: total=4.16GiB, used=1.65GiB
>>>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>
>>>> root@rx1 ~ # btrfs --version
>>>>                                                 :(
>>>> btrfs-progs v4.20.1
>>>>
>>>>
>>>> root@rx1 ~ # uname -a
>>>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
>>>> GNU/Linux
>>>
>>> This is a little old for btrfs, but I don't think that's the cause.
>>>
>>> Thanks,
>>> Qu
>>>
>>>>
>>>> Hope someone can help.
>>>> Regrads
>>>> Mia
>>>>
> 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-25 11:14         ` Qu Wenruo
@ 2021-10-25 11:18           ` Qu Wenruo
       [not found]           ` <884d76d1-5836-9a91-a39b-41c37441e020@rx2.rx-server.de>
  1 sibling, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-25 11:18 UTC (permalink / raw)
  To: Qu Wenruo, Mia, linux-btrfs



On 2021/10/25 19:14, Qu Wenruo wrote:
>
>
> On 2021/10/25 19:13, Mia wrote:
>> Hi Qu,
>>
>> thanks for your response.
>> Here the output of btrfs check:
>> https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>
> Unfortunately it's not full, and it's using an old btrfs-progs which can
> cause false alert.

My bad, gist is folding the output.

It shows no corruption for the extent tree, thus I guess the transaction
abort has prevented COW from being broken.

>
> Please use latest btrfs-progs v5.14.2 to re-check.

In that case, a newer btrfs-progs is only going to remove the false alerts.

Any clue on the workload causing the abort?

For now, I can only recommend to use newer kernel (v5.10+ I guess?) to
see if you can reproduce the problem.

Thanks,
Qu

>
> Thanks,
> Qu
>>
>> Thanks,
>> Mia
>>
>> ------ Originalnachricht ------
>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>> An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>> Gesendet: 25.10.2021 12:55:46
>> Betreff: Re: filesystem corrupt - error -117
>>
>>>
>>>
>>> On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>
>>>>
>>>> On 2021/10/25 16:46, Mia wrote:
>>>>> Hello,
>>>>> I need support since my root filesystem just went readonly :(
>>>>>
>>>>> [641955.981560] BTRFS error (device sda3): tree block 342685007872
>>>>> owner
>>>>> 7 already locked by pid=8099, extent tree corruption detected
>>>>
>>>> This line explains itself.
>>>>
>>>> Your extent tree is no corrupted, thus it allocated a new tree block
>>>
>>> I missed the "w" for the word "now"...
>>>
>>>> which is in fact already hold by other tree.
>>>>
>>>> This means your metadata is no longer protected properly by COW.
>>>>
>>>> "btrfs check" is highly recommended to expose the root cause.
>>>>
>>>>>
>>>>> root@rx1 ~ # btrfs fi show
>>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>
>>>>> root@rx1 ~ # btrfs fi df /
>>>>> Data, single: total=194.89GiB, used=187.46GiB
>>>>> System, single: total=32.00MiB, used=48.00KiB
>>>>> Metadata, single: total=4.16GiB, used=1.65GiB
>>>>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>
>>>>> root@rx1 ~ # btrfs --version
>>>>>                                                 :(
>>>>> btrfs-progs v4.20.1
>>>>>
>>>>>
>>>>> root@rx1 ~ # uname -a
>>>>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
>>>>> GNU/Linux
>>>>
>>>> This is a little old for btrfs, but I don't think that's the cause.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>>
>>>>> Hope someone can help.
>>>>> Regrads
>>>>> Mia
>>>>>
>>
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]           ` <884d76d1-5836-9a91-a39b-41c37441e020@rx2.rx-server.de>
@ 2021-10-25 17:09             ` Mia
  2021-10-25 22:45               ` Qu Wenruo
       [not found]               ` <3ce1dd17-b574-abe3-d6cc-eb16f00117cc@rx2.rx-server.de>
  0 siblings, 2 replies; 21+ messages in thread
From: Mia @ 2021-10-25 17:09 UTC (permalink / raw)
  To: Qu Wenruo, Qu Wenruo, linux-btrfs

Hi Qu,

sorry for the late reply. I tried the btrfs check again with arch live 
cd:

root@archiso ~ # uname -a
Linux archiso 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13 
+0000 x86_64 GNU/Linux
root@archiso ~ # btrfs --version
btrfs-progs v5.14.2

https://gist.github.com/lynara/12dcfff870260b6bc35b9d1137921fc4

I'm still getting many errors.
Sorry I currently don't know what caused this. I suspect it might be 
Seafile since I'm now having a currupted library there.

Should I use --repair?

Regards
Mia

------ Originalnachricht ------
Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
An: "Qu Wenruo" <wqu@suse.com>; "Mia" <9speysdx24@kr33.de>; 
linux-btrfs@vger.kernel.org
Gesendet: 25.10.2021 13:18:54
Betreff: Re: filesystem corrupt - error -117

>
>
>On 2021/10/25 19:14, Qu Wenruo wrote:
>>
>>
>>On 2021/10/25 19:13, Mia wrote:
>>>Hi Qu,
>>>
>>>thanks for your response.
>>>Here the output of btrfs check:
>>>https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>>
>>Unfortunately it's not full, and it's using an old btrfs-progs which can
>>cause false alert.
>
>My bad, gist is folding the output.
>
>It shows no corruption for the extent tree, thus I guess the transaction
>abort has prevented COW from being broken.
>
>>
>>Please use latest btrfs-progs v5.14.2 to re-check.
>
>In that case, a newer btrfs-progs is only going to remove the false alerts.
>
>Any clue on the workload causing the abort?
>
>For now, I can only recommend to use newer kernel (v5.10+ I guess?) to
>see if you can reproduce the problem.
>
>Thanks,
>Qu
>
>>
>>Thanks,
>>Qu
>>>
>>>Thanks,
>>>Mia
>>>
>>>------ Originalnachricht ------
>>>Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>>>Gesendet: 25.10.2021 12:55:46
>>>Betreff: Re: filesystem corrupt - error -117
>>>
>>>>
>>>>
>>>>On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>>On 2021/10/25 16:46, Mia wrote:
>>>>>>Hello,
>>>>>>I need support since my root filesystem just went readonly :(
>>>>>>
>>>>>>[641955.981560] BTRFS error (device sda3): tree block 342685007872
>>>>>>owner
>>>>>>7 already locked by pid=8099, extent tree corruption detected
>>>>>
>>>>>This line explains itself.
>>>>>
>>>>>Your extent tree is no corrupted, thus it allocated a new tree block
>>>>
>>>>I missed the "w" for the word "now"...
>>>>
>>>>>which is in fact already hold by other tree.
>>>>>
>>>>>This means your metadata is no longer protected properly by COW.
>>>>>
>>>>>"btrfs check" is highly recommended to expose the root cause.
>>>>>
>>>>>>
>>>>>>root@rx1 ~ # btrfs fi show
>>>>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>
>>>>>>root@rx1 ~ # btrfs fi df /
>>>>>>Data, single: total=194.89GiB, used=187.46GiB
>>>>>>System, single: total=32.00MiB, used=48.00KiB
>>>>>>Metadata, single: total=4.16GiB, used=1.65GiB
>>>>>>GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>>
>>>>>>root@rx1 ~ # btrfs --version
>>>>>>                                                 :(
>>>>>>btrfs-progs v4.20.1
>>>>>>
>>>>>>
>>>>>>root@rx1 ~ # uname -a
>>>>>>Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
>>>>>>GNU/Linux
>>>>>
>>>>>This is a little old for btrfs, but I don't think that's the cause.
>>>>>
>>>>>Thanks,
>>>>>Qu
>>>>>
>>>>>>
>>>>>>Hope someone can help.
>>>>>>Regrads
>>>>>>Mia
>>>>>>
>>>
>>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-25 17:09             ` Mia
@ 2021-10-25 22:45               ` Qu Wenruo
       [not found]               ` <3ce1dd17-b574-abe3-d6cc-eb16f00117cc@rx2.rx-server.de>
  1 sibling, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-25 22:45 UTC (permalink / raw)
  To: Mia, Qu Wenruo, linux-btrfs



On 2021/10/26 01:09, Mia wrote:
> Hi Qu,
>
> sorry for the late reply. I tried the btrfs check again with arch live cd:
>
> root@archiso ~ # uname -a
> Linux archiso 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13
> +0000 x86_64 GNU/Linux
> root@archiso ~ # btrfs --version
> btrfs-progs v5.14.2
>
> https://gist.github.com/lynara/12dcfff870260b6bc35b9d1137921fc4

OK, so the metadata problem is really there, but it shouldn't affect
your fs right now, unless you want to mount it with 64K page size.

And for the new error (inline file extent too large), it may cause
problems, but under most cases, kernel can handle it without problem.
>
> I'm still getting many errors.
> Sorry I currently don't know what caused this. I suspect it might be
> Seafile since I'm now having a currupted library there.
>
> Should I use --repair?

No, --repair won't help in this case.

In fact, your fs is fine, no on-disk metadata problem yet.

For your case, I can only recommend to use newer kernel to have better
sanity check.
Meanwhile I would also recommend to run a memtest to ensure it's not
some memory problem causing the bug.

Thanks,
Qu

>
> Regards
> Mia
>
> ------ Originalnachricht ------
> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> An: "Qu Wenruo" <wqu@suse.com>; "Mia" <9speysdx24@kr33.de>;
> linux-btrfs@vger.kernel.org
> Gesendet: 25.10.2021 13:18:54
> Betreff: Re: filesystem corrupt - error -117
>
>>
>>
>> On 2021/10/25 19:14, Qu Wenruo wrote:
>>>
>>>
>>> On 2021/10/25 19:13, Mia wrote:
>>>> Hi Qu,
>>>>
>>>> thanks for your response.
>>>> Here the output of btrfs check:
>>>> https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>>>
>>> Unfortunately it's not full, and it's using an old btrfs-progs which can
>>> cause false alert.
>>
>> My bad, gist is folding the output.
>>
>> It shows no corruption for the extent tree, thus I guess the transaction
>> abort has prevented COW from being broken.
>>
>>>
>>> Please use latest btrfs-progs v5.14.2 to re-check.
>>
>> In that case, a newer btrfs-progs is only going to remove the false
>> alerts.
>>
>> Any clue on the workload causing the abort?
>>
>> For now, I can only recommend to use newer kernel (v5.10+ I guess?) to
>> see if you can reproduce the problem.
>>
>> Thanks,
>> Qu
>>
>>>
>>> Thanks,
>>> Qu
>>>>
>>>> Thanks,
>>>> Mia
>>>>
>>>> ------ Originalnachricht ------
>>>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>> An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>>>> Gesendet: 25.10.2021 12:55:46
>>>> Betreff: Re: filesystem corrupt - error -117
>>>>
>>>>>
>>>>>
>>>>> On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>>>
>>>>>>
>>>>>> On 2021/10/25 16:46, Mia wrote:
>>>>>>> Hello,
>>>>>>> I need support since my root filesystem just went readonly :(
>>>>>>>
>>>>>>> [641955.981560] BTRFS error (device sda3): tree block 342685007872
>>>>>>> owner
>>>>>>> 7 already locked by pid=8099, extent tree corruption detected
>>>>>>
>>>>>> This line explains itself.
>>>>>>
>>>>>> Your extent tree is no corrupted, thus it allocated a new tree block
>>>>>
>>>>> I missed the "w" for the word "now"...
>>>>>
>>>>>> which is in fact already hold by other tree.
>>>>>>
>>>>>> This means your metadata is no longer protected properly by COW.
>>>>>>
>>>>>> "btrfs check" is highly recommended to expose the root cause.
>>>>>>
>>>>>>>
>>>>>>> root@rx1 ~ # btrfs fi show
>>>>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>>
>>>>>>> root@rx1 ~ # btrfs fi df /
>>>>>>> Data, single: total=194.89GiB, used=187.46GiB
>>>>>>> System, single: total=32.00MiB, used=48.00KiB
>>>>>>> Metadata, single: total=4.16GiB, used=1.65GiB
>>>>>>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>>>
>>>>>>> root@rx1 ~ # btrfs --version
>>>>>>>                                                 :(
>>>>>>> btrfs-progs v4.20.1
>>>>>>>
>>>>>>>
>>>>>>> root@rx1 ~ # uname -a
>>>>>>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18)
>>>>>>> x86_64
>>>>>>> GNU/Linux
>>>>>>
>>>>>> This is a little old for btrfs, but I don't think that's the cause.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>
>>>>>>>
>>>>>>> Hope someone can help.
>>>>>>> Regrads
>>>>>>> Mia
>>>>>>>
>>>>
>>>
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]               ` <3ce1dd17-b574-abe3-d6cc-eb16f00117cc@rx2.rx-server.de>
@ 2021-10-26  6:03                 ` Mia
  2021-10-26  7:24                   ` Qu Wenruo
                                     ` (2 more replies)
  0 siblings, 3 replies; 21+ messages in thread
From: Mia @ 2021-10-26  6:03 UTC (permalink / raw)
  To: Qu Wenruo, Qu Wenruo, linux-btrfs

Hi Qu,

thanks for clarification.
So I should just ignore these errors for now?
What about these ones, you haven't mentioned:
bad metadata [342605463552, 342605479936) crossing stripe boundary

Problem with updating is that this is currently still Debian 10 and a 
production environment and I don't know when it is possible to upgrade 
because of dependencies.

Regards
Mia

------ Originalnachricht ------
Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
An: "Mia" <9speysdx24@kr33.de>; "Qu Wenruo" <wqu@suse.com>; 
linux-btrfs@vger.kernel.org
Gesendet: 26.10.2021 00:45:18
Betreff: Re: filesystem corrupt - error -117

>
>
>On 2021/10/26 01:09, Mia wrote:
>>Hi Qu,
>>
>>sorry for the late reply. I tried the btrfs check again with arch live cd:
>>
>>root@archiso ~ # uname -a
>>Linux archiso 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13
>>+0000 x86_64 GNU/Linux
>>root@archiso ~ # btrfs --version
>>btrfs-progs v5.14.2
>>
>>https://gist.github.com/lynara/12dcfff870260b6bc35b9d1137921fc4
>
>OK, so the metadata problem is really there, but it shouldn't affect
>your fs right now, unless you want to mount it with 64K page size.
>
>And for the new error (inline file extent too large), it may cause
>problems, but under most cases, kernel can handle it without problem.
>>
>>I'm still getting many errors.
>>Sorry I currently don't know what caused this. I suspect it might be
>>Seafile since I'm now having a currupted library there.
>>
>>Should I use --repair?
>
>No, --repair won't help in this case.
>
>In fact, your fs is fine, no on-disk metadata problem yet.
>
>For your case, I can only recommend to use newer kernel to have better
>sanity check.
>Meanwhile I would also recommend to run a memtest to ensure it's not
>some memory problem causing the bug.
>
>Thanks,
>Qu
>
>>
>>Regards
>>Mia
>>
>>------ Originalnachricht ------
>>Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>An: "Qu Wenruo" <wqu@suse.com>; "Mia" <9speysdx24@kr33.de>;
>>linux-btrfs@vger.kernel.org
>>Gesendet: 25.10.2021 13:18:54
>>Betreff: Re: filesystem corrupt - error -117
>>
>>>
>>>
>>>On 2021/10/25 19:14, Qu Wenruo wrote:
>>>>
>>>>
>>>>On 2021/10/25 19:13, Mia wrote:
>>>>>Hi Qu,
>>>>>
>>>>>thanks for your response.
>>>>>Here the output of btrfs check:
>>>>>https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>>>>
>>>>Unfortunately it's not full, and it's using an old btrfs-progs which can
>>>>cause false alert.
>>>
>>>My bad, gist is folding the output.
>>>
>>>It shows no corruption for the extent tree, thus I guess the transaction
>>>abort has prevented COW from being broken.
>>>
>>>>
>>>>Please use latest btrfs-progs v5.14.2 to re-check.
>>>
>>>In that case, a newer btrfs-progs is only going to remove the false
>>>alerts.
>>>
>>>Any clue on the workload causing the abort?
>>>
>>>For now, I can only recommend to use newer kernel (v5.10+ I guess?) to
>>>see if you can reproduce the problem.
>>>
>>>Thanks,
>>>Qu
>>>
>>>>
>>>>Thanks,
>>>>Qu
>>>>>
>>>>>Thanks,
>>>>>Mia
>>>>>
>>>>>------ Originalnachricht ------
>>>>>Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>>>An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>>>>>Gesendet: 25.10.2021 12:55:46
>>>>>Betreff: Re: filesystem corrupt - error -117
>>>>>
>>>>>>
>>>>>>
>>>>>>On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>>>>
>>>>>>>
>>>>>>>On 2021/10/25 16:46, Mia wrote:
>>>>>>>>Hello,
>>>>>>>>I need support since my root filesystem just went readonly :(
>>>>>>>>
>>>>>>>>[641955.981560] BTRFS error (device sda3): tree block 342685007872
>>>>>>>>owner
>>>>>>>>7 already locked by pid=8099, extent tree corruption detected
>>>>>>>
>>>>>>>This line explains itself.
>>>>>>>
>>>>>>>Your extent tree is no corrupted, thus it allocated a new tree block
>>>>>>
>>>>>>I missed the "w" for the word "now"...
>>>>>>
>>>>>>>which is in fact already hold by other tree.
>>>>>>>
>>>>>>>This means your metadata is no longer protected properly by COW.
>>>>>>>
>>>>>>>"btrfs check" is highly recommended to expose the root cause.
>>>>>>>
>>>>>>>>
>>>>>>>>root@rx1 ~ # btrfs fi show
>>>>>>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>>>
>>>>>>>>root@rx1 ~ # btrfs fi df /
>>>>>>>>Data, single: total=194.89GiB, used=187.46GiB
>>>>>>>>System, single: total=32.00MiB, used=48.00KiB
>>>>>>>>Metadata, single: total=4.16GiB, used=1.65GiB
>>>>>>>>GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>>>>
>>>>>>>>root@rx1 ~ # btrfs --version
>>>>>>>>                                                 :(
>>>>>>>>btrfs-progs v4.20.1
>>>>>>>>
>>>>>>>>
>>>>>>>>root@rx1 ~ # uname -a
>>>>>>>>Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18)
>>>>>>>>x86_64
>>>>>>>>GNU/Linux
>>>>>>>
>>>>>>>This is a little old for btrfs, but I don't think that's the cause.
>>>>>>>
>>>>>>>Thanks,
>>>>>>>Qu
>>>>>>>
>>>>>>>>
>>>>>>>>Hope someone can help.
>>>>>>>>Regrads
>>>>>>>>Mia
>>>>>>>>
>>>>>
>>>>
>>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-26  6:03                 ` Mia
@ 2021-10-26  7:24                   ` Qu Wenruo
  2021-10-26 11:18                     ` Robert Krig
  2021-10-26 14:12                   ` Patrik Lundquist
       [not found]                   ` <CAA7pwKOrgt6syr5C3X1+bC14QXZEJ+8HTMZruBBPBT574zNkkQ@rx2.rx-server.de>
  2 siblings, 1 reply; 21+ messages in thread
From: Qu Wenruo @ 2021-10-26  7:24 UTC (permalink / raw)
  To: Mia, Qu Wenruo, linux-btrfs



On 2021/10/26 14:03, Mia wrote:
> Hi Qu,
>
> thanks for clarification.
> So I should just ignore these errors for now?

Yes, none of them is going to cause any direct problems.

> What about these ones, you haven't mentioned:
> bad metadata [342605463552, 342605479936) crossing stripe boundary

This is the same, it just means it crosses 64K boundary, which is not
supported for the incoming subpage support (using 4K page size on 64K
page size systems).

>
> Problem with updating is that this is currently still Debian 10 and a
> production environment and I don't know when it is possible to upgrade
> because of dependencies.

OK, understood the situation now.

Then I can't provide much helper as I'm not familiar with Debian...

If not reproducible so far, I can only recommend for a memtest to rule
out memory bitflip, which could also cause the bug.

Thanks,
Qu
>
> Regards
> Mia
>
> ------ Originalnachricht ------
> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> An: "Mia" <9speysdx24@kr33.de>; "Qu Wenruo" <wqu@suse.com>;
> linux-btrfs@vger.kernel.org
> Gesendet: 26.10.2021 00:45:18
> Betreff: Re: filesystem corrupt - error -117
>
>>
>>
>> On 2021/10/26 01:09, Mia wrote:
>>> Hi Qu,
>>>
>>> sorry for the late reply. I tried the btrfs check again with arch
>>> live cd:
>>>
>>> root@archiso ~ # uname -a
>>> Linux archiso 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13
>>> +0000 x86_64 GNU/Linux
>>> root@archiso ~ # btrfs --version
>>> btrfs-progs v5.14.2
>>>
>>> https://gist.github.com/lynara/12dcfff870260b6bc35b9d1137921fc4
>>
>> OK, so the metadata problem is really there, but it shouldn't affect
>> your fs right now, unless you want to mount it with 64K page size.
>>
>> And for the new error (inline file extent too large), it may cause
>> problems, but under most cases, kernel can handle it without problem.
>>>
>>> I'm still getting many errors.
>>> Sorry I currently don't know what caused this. I suspect it might be
>>> Seafile since I'm now having a currupted library there.
>>>
>>> Should I use --repair?
>>
>> No, --repair won't help in this case.
>>
>> In fact, your fs is fine, no on-disk metadata problem yet.
>>
>> For your case, I can only recommend to use newer kernel to have better
>> sanity check.
>> Meanwhile I would also recommend to run a memtest to ensure it's not
>> some memory problem causing the bug.
>>
>> Thanks,
>> Qu
>>
>>>
>>> Regards
>>> Mia
>>>
>>> ------ Originalnachricht ------
>>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>> An: "Qu Wenruo" <wqu@suse.com>; "Mia" <9speysdx24@kr33.de>;
>>> linux-btrfs@vger.kernel.org
>>> Gesendet: 25.10.2021 13:18:54
>>> Betreff: Re: filesystem corrupt - error -117
>>>
>>>>
>>>>
>>>> On 2021/10/25 19:14, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>> On 2021/10/25 19:13, Mia wrote:
>>>>>> Hi Qu,
>>>>>>
>>>>>> thanks for your response.
>>>>>> Here the output of btrfs check:
>>>>>> https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>>>>>
>>>>> Unfortunately it's not full, and it's using an old btrfs-progs
>>>>> which can
>>>>> cause false alert.
>>>>
>>>> My bad, gist is folding the output.
>>>>
>>>> It shows no corruption for the extent tree, thus I guess the
>>>> transaction
>>>> abort has prevented COW from being broken.
>>>>
>>>>>
>>>>> Please use latest btrfs-progs v5.14.2 to re-check.
>>>>
>>>> In that case, a newer btrfs-progs is only going to remove the false
>>>> alerts.
>>>>
>>>> Any clue on the workload causing the abort?
>>>>
>>>> For now, I can only recommend to use newer kernel (v5.10+ I guess?) to
>>>> see if you can reproduce the problem.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>>
>>>>>> Thanks,
>>>>>> Mia
>>>>>>
>>>>>> ------ Originalnachricht ------
>>>>>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>>>> An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>>>>>> Gesendet: 25.10.2021 12:55:46
>>>>>> Betreff: Re: filesystem corrupt - error -117
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2021/10/25 16:46, Mia wrote:
>>>>>>>>> Hello,
>>>>>>>>> I need support since my root filesystem just went readonly :(
>>>>>>>>>
>>>>>>>>> [641955.981560] BTRFS error (device sda3): tree block 342685007872
>>>>>>>>> owner
>>>>>>>>> 7 already locked by pid=8099, extent tree corruption detected
>>>>>>>>
>>>>>>>> This line explains itself.
>>>>>>>>
>>>>>>>> Your extent tree is no corrupted, thus it allocated a new tree
>>>>>>>> block
>>>>>>>
>>>>>>> I missed the "w" for the word "now"...
>>>>>>>
>>>>>>>> which is in fact already hold by other tree.
>>>>>>>>
>>>>>>>> This means your metadata is no longer protected properly by COW.
>>>>>>>>
>>>>>>>> "btrfs check" is highly recommended to expose the root cause.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> root@rx1 ~ # btrfs fi show
>>>>>>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>>>>
>>>>>>>>> root@rx1 ~ # btrfs fi df /
>>>>>>>>> Data, single: total=194.89GiB, used=187.46GiB
>>>>>>>>> System, single: total=32.00MiB, used=48.00KiB
>>>>>>>>> Metadata, single: total=4.16GiB, used=1.65GiB
>>>>>>>>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>>>>>
>>>>>>>>> root@rx1 ~ # btrfs --version
>>>>>>>>>                                                 :(
>>>>>>>>> btrfs-progs v4.20.1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> root@rx1 ~ # uname -a
>>>>>>>>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18)
>>>>>>>>> x86_64
>>>>>>>>> GNU/Linux
>>>>>>>>
>>>>>>>> This is a little old for btrfs, but I don't think that's the cause.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Qu
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hope someone can help.
>>>>>>>>> Regrads
>>>>>>>>> Mia
>>>>>>>>>
>>>>>>
>>>>>
>>>
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-26  7:24                   ` Qu Wenruo
@ 2021-10-26 11:18                     ` Robert Krig
  0 siblings, 0 replies; 21+ messages in thread
From: Robert Krig @ 2021-10-26 11:18 UTC (permalink / raw)
  To: linux-btrfs


[-- Attachment #1.1.1: Type: text/plain, Size: 6567 bytes --]

If you are on Debian 10, you can enable backports and install both a 
kernel and btrfs-progs from backports. This would of course require a 
reboot. Apart from that, this shouldn't conflict with any other 
dependencies, unless you have something that explicitly relies on the 
installed kernel version.



On 26.10.21 09:24, Qu Wenruo wrote:
>
>
> On 2021/10/26 14:03, Mia wrote:
>> Hi Qu,
>>
>> thanks for clarification.
>> So I should just ignore these errors for now?
>
> Yes, none of them is going to cause any direct problems.
>
>> What about these ones, you haven't mentioned:
>> bad metadata [342605463552, 342605479936) crossing stripe boundary
>
> This is the same, it just means it crosses 64K boundary, which is not
> supported for the incoming subpage support (using 4K page size on 64K
> page size systems).
>
>>
>> Problem with updating is that this is currently still Debian 10 and a
>> production environment and I don't know when it is possible to upgrade
>> because of dependencies.
>
> OK, understood the situation now.
>
> Then I can't provide much helper as I'm not familiar with Debian...
>
> If not reproducible so far, I can only recommend for a memtest to rule
> out memory bitflip, which could also cause the bug.
>
> Thanks,
> Qu
>>
>> Regards
>> Mia
>>
>> ------ Originalnachricht ------
>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>> An: "Mia" <9speysdx24@kr33.de>; "Qu Wenruo" <wqu@suse.com>;
>> linux-btrfs@vger.kernel.org
>> Gesendet: 26.10.2021 00:45:18
>> Betreff: Re: filesystem corrupt - error -117
>>
>>>
>>>
>>> On 2021/10/26 01:09, Mia wrote:
>>>> Hi Qu,
>>>>
>>>> sorry for the late reply. I tried the btrfs check again with arch
>>>> live cd:
>>>>
>>>> root@archiso ~ # uname -a
>>>> Linux archiso 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13
>>>> +0000 x86_64 GNU/Linux
>>>> root@archiso ~ # btrfs --version
>>>> btrfs-progs v5.14.2
>>>>
>>>> https://gist.github.com/lynara/12dcfff870260b6bc35b9d1137921fc4
>>>
>>> OK, so the metadata problem is really there, but it shouldn't affect
>>> your fs right now, unless you want to mount it with 64K page size.
>>>
>>> And for the new error (inline file extent too large), it may cause
>>> problems, but under most cases, kernel can handle it without problem.
>>>>
>>>> I'm still getting many errors.
>>>> Sorry I currently don't know what caused this. I suspect it might be
>>>> Seafile since I'm now having a currupted library there.
>>>>
>>>> Should I use --repair?
>>>
>>> No, --repair won't help in this case.
>>>
>>> In fact, your fs is fine, no on-disk metadata problem yet.
>>>
>>> For your case, I can only recommend to use newer kernel to have better
>>> sanity check.
>>> Meanwhile I would also recommend to run a memtest to ensure it's not
>>> some memory problem causing the bug.
>>>
>>> Thanks,
>>> Qu
>>>
>>>>
>>>> Regards
>>>> Mia
>>>>
>>>> ------ Originalnachricht ------
>>>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>> An: "Qu Wenruo" <wqu@suse.com>; "Mia" <9speysdx24@kr33.de>;
>>>> linux-btrfs@vger.kernel.org
>>>> Gesendet: 25.10.2021 13:18:54
>>>> Betreff: Re: filesystem corrupt - error -117
>>>>
>>>>>
>>>>>
>>>>> On 2021/10/25 19:14, Qu Wenruo wrote:
>>>>>>
>>>>>>
>>>>>> On 2021/10/25 19:13, Mia wrote:
>>>>>>> Hi Qu,
>>>>>>>
>>>>>>> thanks for your response.
>>>>>>> Here the output of btrfs check:
>>>>>>> https://gist.github.com/lynara/1c613f7ec9448600f643a59d22c1efb2
>>>>>>
>>>>>> Unfortunately it's not full, and it's using an old btrfs-progs
>>>>>> which can
>>>>>> cause false alert.
>>>>>
>>>>> My bad, gist is folding the output.
>>>>>
>>>>> It shows no corruption for the extent tree, thus I guess the
>>>>> transaction
>>>>> abort has prevented COW from being broken.
>>>>>
>>>>>>
>>>>>> Please use latest btrfs-progs v5.14.2 to re-check.
>>>>>
>>>>> In that case, a newer btrfs-progs is only going to remove the false
>>>>> alerts.
>>>>>
>>>>> Any clue on the workload causing the abort?
>>>>>
>>>>> For now, I can only recommend to use newer kernel (v5.10+ I 
>>>>> guess?) to
>>>>> see if you can reproduce the problem.
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mia
>>>>>>>
>>>>>>> ------ Originalnachricht ------
>>>>>>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>>>>> An: "Mia" <9speysdx24@kr33.de>; linux-btrfs@vger.kernel.org
>>>>>>> Gesendet: 25.10.2021 12:55:46
>>>>>>> Betreff: Re: filesystem corrupt - error -117
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2021/10/25 18:53, Qu Wenruo wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 2021/10/25 16:46, Mia wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> I need support since my root filesystem just went readonly :(
>>>>>>>>>>
>>>>>>>>>> [641955.981560] BTRFS error (device sda3): tree block 
>>>>>>>>>> 342685007872
>>>>>>>>>> owner
>>>>>>>>>> 7 already locked by pid=8099, extent tree corruption detected
>>>>>>>>>
>>>>>>>>> This line explains itself.
>>>>>>>>>
>>>>>>>>> Your extent tree is no corrupted, thus it allocated a new tree
>>>>>>>>> block
>>>>>>>>
>>>>>>>> I missed the "w" for the word "now"...
>>>>>>>>
>>>>>>>>> which is in fact already hold by other tree.
>>>>>>>>>
>>>>>>>>> This means your metadata is no longer protected properly by COW.
>>>>>>>>>
>>>>>>>>> "btrfs check" is highly recommended to expose the root cause.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> root@rx1 ~ # btrfs fi show
>>>>>>>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>>>>>      Total devices 1 FS bytes used 189.12GiB
>>>>>>>>>>      devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>>>>>
>>>>>>>>>> root@rx1 ~ # btrfs fi df /
>>>>>>>>>> Data, single: total=194.89GiB, used=187.46GiB
>>>>>>>>>> System, single: total=32.00MiB, used=48.00KiB
>>>>>>>>>> Metadata, single: total=4.16GiB, used=1.65GiB
>>>>>>>>>> GlobalReserve, single: total=380.45MiB, used=0.00B
>>>>>>>>>>
>>>>>>>>>> root@rx1 ~ # btrfs --version
>>>>>>>>>> :(
>>>>>>>>>> btrfs-progs v4.20.1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> root@rx1 ~ # uname -a
>>>>>>>>>> Linux rx1 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18)
>>>>>>>>>> x86_64
>>>>>>>>>> GNU/Linux
>>>>>>>>>
>>>>>>>>> This is a little old for btrfs, but I don't think that's the 
>>>>>>>>> cause.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Qu
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hope someone can help.
>>>>>>>>>> Regrads
>>>>>>>>>> Mia
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 25205 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-26  6:03                 ` Mia
  2021-10-26  7:24                   ` Qu Wenruo
@ 2021-10-26 14:12                   ` Patrik Lundquist
       [not found]                   ` <CAA7pwKOrgt6syr5C3X1+bC14QXZEJ+8HTMZruBBPBT574zNkkQ@rx2.rx-server.de>
  2 siblings, 0 replies; 21+ messages in thread
From: Patrik Lundquist @ 2021-10-26 14:12 UTC (permalink / raw)
  To: Mia; +Cc: Linux Btrfs

On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>
> Problem with updating is that this is currently still Debian 10 and a
> production environment and I don't know when it is possible to upgrade
> because of dependencies.

Maybe you can install the buster-backports kernel which currently is 5.10.70?

https://backports.debian.org/Instructions/

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                   ` <CAA7pwKOrgt6syr5C3X1+bC14QXZEJ+8HTMZruBBPBT574zNkkQ@rx2.rx-server.de>
@ 2021-10-26 17:28                     ` Mia
  2021-10-27  2:49                       ` Qu Wenruo
       [not found]                     ` <emb611c0ff-705d-4c01-b50f-320f962f39fb@frystation>
  1 sibling, 1 reply; 21+ messages in thread
From: Mia @ 2021-10-26 17:28 UTC (permalink / raw)
  To: Patrik Lundquist, Qu Wenruo; +Cc: Linux Btrfs

Hi Patrik,

good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian 
5.10.46-4~bpo10+1 (2021-08-07) now.

Hi Qu,
regarding the memtest. This is a virtual machine, I have no access to 
the host system.
I don't know if a memtest inside the vm would bring legit results?

Regards
Mia

------ Originalnachricht ------
Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
An: "Mia" <9speysdx24@kr33.de>
Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
Gesendet: 26.10.2021 16:12:45
Betreff: Re: filesystem corrupt - error -117

>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>
>>  Problem with updating is that this is currently still Debian 10 and a
>>  production environment and I don't know when it is possible to upgrade
>>  because of dependencies.
>
>Maybe you can install the buster-backports kernel which currently is 5.10.70?
>
>https://backports.debian.org/Instructions/


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-10-26 17:28                     ` Mia
@ 2021-10-27  2:49                       ` Qu Wenruo
  0 siblings, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-10-27  2:49 UTC (permalink / raw)
  To: Mia, Patrik Lundquist, Qu Wenruo; +Cc: Linux Btrfs



On 2021/10/27 01:28, Mia wrote:
> Hi Patrik,
> 
> good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian 
> 5.10.46-4~bpo10+1 (2021-08-07) now.
> 
> Hi Qu,
> regarding the memtest. This is a virtual machine, I have no access to 
> the host system.
> I don't know if a memtest inside the vm would bring legit results?

If the host is using ECC memory (IIRC most VPS are already doing that), 
then no need to bother the memory bitflip possibility.

Thanks,
Qu

> 
> Regards
> Mia
> 
> ------ Originalnachricht ------
> Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
> An: "Mia" <9speysdx24@kr33.de>
> Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
> Gesendet: 26.10.2021 16:12:45
> Betreff: Re: filesystem corrupt - error -117
> 
>> On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>
>>>  Problem with updating is that this is currently still Debian 10 and a
>>>  production environment and I don't know when it is possible to upgrade
>>>  because of dependencies.
>>
>> Maybe you can install the buster-backports kernel which currently is 
>> 5.10.70?
>>
>> https://backports.debian.org/Instructions/
> 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                     ` <emb611c0ff-705d-4c01-b50f-320f962f39fb@frystation>
@ 2021-12-11  8:20                       ` Mia
       [not found]                       ` <embddc7343-8fdf-4be8-87d8-644e20ea86c0@frystation>
  1 sibling, 0 replies; 21+ messages in thread
From: Mia @ 2021-12-11  8:20 UTC (permalink / raw)
  To: Patrik Lundquist, Qu Wenruo, linux-btrfs

Hello,

after the switching to the newer kernel I had no further problems until 
now.
It happened again.

[1653860.040458] BTRFS error (device sda3): tree first key mismatch 
detected, bytenr=543379832832 parent_transid=1349283 key 
expected=(468977471488,168,16384) has=(468983328768,168,61440)
[1653860.041095] ------------[ cut here ]------------
[1653860.041098] BTRFS: Transaction aborted (error -117)
[1653860.041289] WARNING: CPU: 2 PID: 219 at fs/btrfs/extent-tree.c:2148 
btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
[1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT 
nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter 
xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment xt_hashlimit 
xt_conntrack xt_addrtype xt_mark xt_nat xt_CT nfnetlink_log xt_NFLOG 
nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic 
nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 
nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane 
nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp 
nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast 
nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter 
nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack 
nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard 
libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64 
ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64 
curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic 
overlay bridge stp llc fuse ext4
[1653860.041392]  amd_energy crc32_pclmul crc16 mbcache 
ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd 
glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw 
pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs 
blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic usbhid 
hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic ata_generic 
virtio_net net_failover virtio_scsi failover bochs_drm drm_vram_helper 
drm_ttm_helper ttm drm_kms_helper cec ata_piix uhci_hcd drm libata 
ehci_hcd usbcore crct10dif_pclmul crct10dif_common virtio_pci 
crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod i2c_piix4 
floppy button
[1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted: G        
      L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
[1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9 
06/01/2021
[1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
[1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83 
f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac 12 
4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7 c6 
a0
[1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
[1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX: 
0000000000000027
[1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI: 
ffff9d54efd18a08
[1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09: 
c0000000ffffefff
[1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12: 
ffff9d51f0cda800
[1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15: 
0000000000000cca
[1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000) 
knlGS:0000000000000000
[1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4: 
0000000000350ee0
[1653860.041583] Call Trace:
[1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
[1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
[1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
[1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
[1653860.041817]  kthread+0x116/0x130
[1653860.041821]  ? __kthread_cancel_work+0x40/0x40
[1653860.041826]  ret_from_fork+0x22/0x30
[1653860.041831] ---[ end trace af79c203a5452514 ]---
[1653860.041835] BTRFS: error (device sda3) in 
btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
[1653860.042111] BTRFS info (device sda3): forced readonly

root@rx1 ~ # btrfs fi show
Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
     Total devices 1 FS bytes used 186.78GiB
     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3

root@rx1 ~ # btrfs fi df /                                               
                                                                          
                                                :(
Data, single: total=194.89GiB, used=185.16GiB
System, single: total=32.00MiB, used=48.00KiB
Metadata, single: total=4.16GiB, used=1.62GiB
GlobalReserve, single: total=385.84MiB, used=0.00B

root@rx1 ~ # btrfs --version
btrfs-progs v5.10.1

root@rx1 ~ # uname -a
Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1 
(2021-10-10) x86_64 GNU/Linux

Regards
Mia

------ Originalnachricht ------
Von: "Mia" <9speysdx24@kr33.de>
An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo" 
<quwenruo.btrfs@gmx.com>
Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
Gesendet: 26.10.2021 19:28:14
Betreff: Re: filesystem corrupt - error -117

>Hi Patrik,
>
>good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian 5.10.46-4~bpo10+1 (2021-08-07) now.
>
>Hi Qu,
>regarding the memtest. This is a virtual machine, I have no access to the host system.
>I don't know if a memtest inside the vm would bring legit results?
>
>Regards
>Mia
>
>------ Originalnachricht ------
>Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>An: "Mia" <9speysdx24@kr33.de>
>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>Gesendet: 26.10.2021 16:12:45
>Betreff: Re: filesystem corrupt - error -117
>
>>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>
>>>  Problem with updating is that this is currently still Debian 10 and a
>>>  production environment and I don't know when it is possible to upgrade
>>>  because of dependencies.
>>
>>Maybe you can install the buster-backports kernel which currently is 5.10.70?
>>
>>https://backports.debian.org/Instructions/


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                       ` <embddc7343-8fdf-4be8-87d8-644e20ea86c0@frystation>
@ 2021-12-11  8:28                         ` Mia
  2021-12-11  8:39                           ` Qu Wenruo
       [not found]                           ` <29fcb603-506f-d721-5214-2870ce2f8773@rx2.rx-server.de>
  0 siblings, 2 replies; 21+ messages in thread
From: Mia @ 2021-12-11  8:28 UTC (permalink / raw)
  To: Patrik Lundquist, Qu Wenruo, linux-btrfs

Hi,

just an addition, I'm now unable to mount and boot the server.

https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing

Regards
Mia

------ Originalnachricht ------
Von: "Mia" <9speysdx24@kr33.de>
An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo" 
<quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
Gesendet: 11.12.2021 09:20:39
Betreff: Re: filesystem corrupt - error -117

>Hello,
>
>after the switching to the newer kernel I had no further problems until now.
>It happened again.
>
>[1653860.040458] BTRFS error (device sda3): tree first key mismatch detected, bytenr=543379832832 parent_transid=1349283 key expected=(468977471488,168,16384) has=(468983328768,168,61440)
>[1653860.041095] ------------[ cut here ]------------
>[1653860.041098] BTRFS: Transaction aborted (error -117)
>[1653860.041289] WARNING: CPU: 2 PID: 219 at fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>[1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64 curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic overlay bridge stp llc fuse ext4
>[1653860.041392]  amd_energy crc32_pclmul crc16 mbcache ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic ata_generic virtio_net net_failover virtio_scsi failover bochs_drm drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod i2c_piix4 floppy button
>[1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted: G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>[1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9 06/01/2021
>[1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>[1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83 f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac 12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7 c6 a0
>[1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>[1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX: 0000000000000027
>[1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI: ffff9d54efd18a08
>[1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09: c0000000ffffefff
>[1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12: ffff9d51f0cda800
>[1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15: 0000000000000cca
>[1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000) knlGS:0000000000000000
>[1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>[1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4: 0000000000350ee0
>[1653860.041583] Call Trace:
>[1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>[1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>[1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>[1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>[1653860.041817]  kthread+0x116/0x130
>[1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>[1653860.041826]  ret_from_fork+0x22/0x30
>[1653860.041831] ---[ end trace af79c203a5452514 ]---
>[1653860.041835] BTRFS: error (device sda3) in btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>[1653860.042111] BTRFS info (device sda3): forced readonly
>
>root@rx1 ~ # btrfs fi show
>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>     Total devices 1 FS bytes used 186.78GiB
>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>
>root@rx1 ~ # btrfs fi df /                                                                                                                                                                       :(
>Data, single: total=194.89GiB, used=185.16GiB
>System, single: total=32.00MiB, used=48.00KiB
>Metadata, single: total=4.16GiB, used=1.62GiB
>GlobalReserve, single: total=385.84MiB, used=0.00B
>
>root@rx1 ~ # btrfs --version
>btrfs-progs v5.10.1
>
>root@rx1 ~ # uname -a
>Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1 (2021-10-10) x86_64 GNU/Linux
>
>Regards
>Mia
>
>------ Originalnachricht ------
>Von: "Mia" <9speysdx24@kr33.de>
>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>Gesendet: 26.10.2021 19:28:14
>Betreff: Re: filesystem corrupt - error -117
>
>>Hi Patrik,
>>
>>good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian 5.10.46-4~bpo10+1 (2021-08-07) now.
>>
>>Hi Qu,
>>regarding the memtest. This is a virtual machine, I have no access to the host system.
>>I don't know if a memtest inside the vm would bring legit results?
>>
>>Regards
>>Mia
>>
>>------ Originalnachricht ------
>>Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>An: "Mia" <9speysdx24@kr33.de>
>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>Gesendet: 26.10.2021 16:12:45
>>Betreff: Re: filesystem corrupt - error -117
>>
>>>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>
>>>>  Problem with updating is that this is currently still Debian 10 and a
>>>>  production environment and I don't know when it is possible to upgrade
>>>>  because of dependencies.
>>>
>>>Maybe you can install the buster-backports kernel which currently is 5.10.70?
>>>
>>>https://backports.debian.org/Instructions/


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-12-11  8:28                         ` Mia
@ 2021-12-11  8:39                           ` Qu Wenruo
       [not found]                           ` <29fcb603-506f-d721-5214-2870ce2f8773@rx2.rx-server.de>
  1 sibling, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-12-11  8:39 UTC (permalink / raw)
  To: Mia, Patrik Lundquist, linux-btrfs, Filipe Manana



On 2021/12/11 16:28, Mia wrote:
> Hi,
>
> just an addition, I'm now unable to mount and boot the server.
>
> https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing

This is just log tree, thus btrfs-rescue zero-log can easily repair that.

This bug looks like something fixed by Filipe, but I'm not familiar
enough with log tree code.

Add Filipe to this thread.

Thanks,
Qu

>
>
> Regards
> Mia
>
> ------ Originalnachricht ------
> Von: "Mia" <9speysdx24@kr33.de>
> An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
> <quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
> Gesendet: 11.12.2021 09:20:39
> Betreff: Re: filesystem corrupt - error -117
>
>> Hello,
>>
>> after the switching to the newer kernel I had no further problems
>> until now.
>> It happened again.
>>
>> [1653860.040458] BTRFS error (device sda3): tree first key mismatch
>> detected, bytenr=543379832832 parent_transid=1349283 key
>> expected=(468977471488,168,16384) has=(468983328768,168,61440)
>> [1653860.041095] ------------[ cut here ]------------
>> [1653860.041098] BTRFS: Transaction aborted (error -117)
>> [1653860.041289] WARNING: CPU: 2 PID: 219 at
>> fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>> [1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT
>> nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter
>> xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment
>> xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT
>> nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp
>> nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc
>> nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda
>> nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp
>> nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>> nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter
>> nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack
>> nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard
>> libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64
>> ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64
>> curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic
>> overlay bridge stp llc fuse ext4
>> [1653860.041392]  amd_energy crc32_pclmul crc16 mbcache
>> ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd
>> glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw
>> pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs
>> blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic
>> usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic
>> ata_generic virtio_net net_failover virtio_scsi failover bochs_drm
>> drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix
>> uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common
>> virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod
>> i2c_piix4 floppy button
>> [1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted:
>> G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>> [1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9
>> 06/01/2021
>> [1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>> [1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83
>> f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac
>> 12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7
>> c6 a0
>> [1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>> [1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX:
>> 0000000000000027
>> [1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI:
>> ffff9d54efd18a08
>> [1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09:
>> c0000000ffffefff
>> [1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12:
>> ffff9d51f0cda800
>> [1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15:
>> 0000000000000cca
>> [1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000)
>> knlGS:0000000000000000
>> [1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4:
>> 0000000000350ee0
>> [1653860.041583] Call Trace:
>> [1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>> [1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>> [1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>> [1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>> [1653860.041817]  kthread+0x116/0x130
>> [1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>> [1653860.041826]  ret_from_fork+0x22/0x30
>> [1653860.041831] ---[ end trace af79c203a5452514 ]---
>> [1653860.041835] BTRFS: error (device sda3) in
>> btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>> [1653860.042111] BTRFS info (device sda3): forced readonly
>>
>> root@rx1 ~ # btrfs fi show
>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>     Total devices 1 FS bytes used 186.78GiB
>>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>
>> root@rx1 ~ # btrfs fi df
>> /
>> :(
>> Data, single: total=194.89GiB, used=185.16GiB
>> System, single: total=32.00MiB, used=48.00KiB
>> Metadata, single: total=4.16GiB, used=1.62GiB
>> GlobalReserve, single: total=385.84MiB, used=0.00B
>>
>> root@rx1 ~ # btrfs --version
>> btrfs-progs v5.10.1
>>
>> root@rx1 ~ # uname -a
>> Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1
>> (2021-10-10) x86_64 GNU/Linux
>>
>> Regards
>> Mia
>>
>> ------ Originalnachricht ------
>> Von: "Mia" <9speysdx24@kr33.de>
>> An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>> <quwenruo.btrfs@gmx.com>
>> Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>> Gesendet: 26.10.2021 19:28:14
>> Betreff: Re: filesystem corrupt - error -117
>>
>>> Hi Patrik,
>>>
>>> good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian
>>> 5.10.46-4~bpo10+1 (2021-08-07) now.
>>>
>>> Hi Qu,
>>> regarding the memtest. This is a virtual machine, I have no access to
>>> the host system.
>>> I don't know if a memtest inside the vm would bring legit results?
>>>
>>> Regards
>>> Mia
>>>
>>> ------ Originalnachricht ------
>>> Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>> An: "Mia" <9speysdx24@kr33.de>
>>> Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>> Gesendet: 26.10.2021 16:12:45
>>> Betreff: Re: filesystem corrupt - error -117
>>>
>>>> On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>>
>>>>>  Problem with updating is that this is currently still Debian 10 and a
>>>>>  production environment and I don't know when it is possible to
>>>>> upgrade
>>>>>  because of dependencies.
>>>>
>>>> Maybe you can install the buster-backports kernel which currently is
>>>> 5.10.70?
>>>>
>>>> https://backports.debian.org/Instructions/
>

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                           ` <29fcb603-506f-d721-5214-2870ce2f8773@rx2.rx-server.de>
@ 2021-12-11  9:10                             ` Mia
       [not found]                             ` <em7854e1bc-eb3d-43a3-abc7-c6ed3e1a167a@frystation>
  1 sibling, 0 replies; 21+ messages in thread
From: Mia @ 2021-12-11  9:10 UTC (permalink / raw)
  To: Qu Wenruo, Patrik Lundquist, linux-btrfs, Filipe Manana

Hi Qu,

thanks for the quick reply.
As suggested I did a btrfs rescue zero-log /dev/sda3
and I was able to mount and than boot the server again.

Are there any further steps I can do do verify overall integrity?
Since my trust in btrfs isn't that great anymore.

Thanks again for your help!
Mia

------ Originalnachricht ------
Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
An: "Mia" <9speysdx24@kr33.de>; "Patrik Lundquist" 
<patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe 
Manana" <fdmanana@kernel.org>
Gesendet: 11.12.2021 09:39:01
Betreff: Re: filesystem corrupt - error -117

>
>
>On 2021/12/11 16:28, Mia wrote:
>>Hi,
>>
>>just an addition, I'm now unable to mount and boot the server.
>>
>>https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing
>
>This is just log tree, thus btrfs-rescue zero-log can easily repair that.
>
>This bug looks like something fixed by Filipe, but I'm not familiar
>enough with log tree code.
>
>Add Filipe to this thread.
>
>Thanks,
>Qu
>
>>
>>
>>Regards
>>Mia
>>
>>------ Originalnachricht ------
>>Von: "Mia" <9speysdx24@kr33.de>
>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>><quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
>>Gesendet: 11.12.2021 09:20:39
>>Betreff: Re: filesystem corrupt - error -117
>>
>>>Hello,
>>>
>>>after the switching to the newer kernel I had no further problems
>>>until now.
>>>It happened again.
>>>
>>>[1653860.040458] BTRFS error (device sda3): tree first key mismatch
>>>detected, bytenr=543379832832 parent_transid=1349283 key
>>>expected=(468977471488,168,16384) has=(468983328768,168,61440)
>>>[1653860.041095] ------------[ cut here ]------------
>>>[1653860.041098] BTRFS: Transaction aborted (error -117)
>>>[1653860.041289] WARNING: CPU: 2 PID: 219 at
>>>fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>[1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT
>>>nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter
>>>xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment
>>>xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT
>>>nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp
>>>nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc
>>>nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda
>>>nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp
>>>nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>>>nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter
>>>nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack
>>>nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard
>>>libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64
>>>ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64
>>>curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic
>>>overlay bridge stp llc fuse ext4
>>>[1653860.041392]  amd_energy crc32_pclmul crc16 mbcache
>>>ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd
>>>glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw
>>>pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs
>>>blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic
>>>usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic
>>>ata_generic virtio_net net_failover virtio_scsi failover bochs_drm
>>>drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix
>>>uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common
>>>virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod
>>>i2c_piix4 floppy button
>>>[1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted:
>>>G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>>>[1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9
>>>06/01/2021
>>>[1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>[1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83
>>>f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac
>>>12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7
>>>c6 a0
>>>[1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>>>[1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX:
>>>0000000000000027
>>>[1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI:
>>>ffff9d54efd18a08
>>>[1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09:
>>>c0000000ffffefff
>>>[1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12:
>>>ffff9d51f0cda800
>>>[1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15:
>>>0000000000000cca
>>>[1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000)
>>>knlGS:0000000000000000
>>>[1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>[1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4:
>>>0000000000350ee0
>>>[1653860.041583] Call Trace:
>>>[1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>>>[1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>>>[1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>>>[1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>>>[1653860.041817]  kthread+0x116/0x130
>>>[1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>>>[1653860.041826]  ret_from_fork+0x22/0x30
>>>[1653860.041831] ---[ end trace af79c203a5452514 ]---
>>>[1653860.041835] BTRFS: error (device sda3) in
>>>btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>>>[1653860.042111] BTRFS info (device sda3): forced readonly
>>>
>>>root@rx1 ~ # btrfs fi show
>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>     Total devices 1 FS bytes used 186.78GiB
>>>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>
>>>root@rx1 ~ # btrfs fi df
>>>/
>>>:(
>>>Data, single: total=194.89GiB, used=185.16GiB
>>>System, single: total=32.00MiB, used=48.00KiB
>>>Metadata, single: total=4.16GiB, used=1.62GiB
>>>GlobalReserve, single: total=385.84MiB, used=0.00B
>>>
>>>root@rx1 ~ # btrfs --version
>>>btrfs-progs v5.10.1
>>>
>>>root@rx1 ~ # uname -a
>>>Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1
>>>(2021-10-10) x86_64 GNU/Linux
>>>
>>>Regards
>>>Mia
>>>
>>>------ Originalnachricht ------
>>>Von: "Mia" <9speysdx24@kr33.de>
>>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>><quwenruo.btrfs@gmx.com>
>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>Gesendet: 26.10.2021 19:28:14
>>>Betreff: Re: filesystem corrupt - error -117
>>>
>>>>Hi Patrik,
>>>>
>>>>good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian
>>>>5.10.46-4~bpo10+1 (2021-08-07) now.
>>>>
>>>>Hi Qu,
>>>>regarding the memtest. This is a virtual machine, I have no access to
>>>>the host system.
>>>>I don't know if a memtest inside the vm would bring legit results?
>>>>
>>>>Regards
>>>>Mia
>>>>
>>>>------ Originalnachricht ------
>>>>Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>>>An: "Mia" <9speysdx24@kr33.de>
>>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>Gesendet: 26.10.2021 16:12:45
>>>>Betreff: Re: filesystem corrupt - error -117
>>>>
>>>>>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>>>
>>>>>>  Problem with updating is that this is currently still Debian 10 and a
>>>>>>  production environment and I don't know when it is possible to
>>>>>>upgrade
>>>>>>  because of dependencies.
>>>>>
>>>>>Maybe you can install the buster-backports kernel which currently is
>>>>>5.10.70?
>>>>>
>>>>>https://backports.debian.org/Instructions/
>>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                             ` <em7854e1bc-eb3d-43a3-abc7-c6ed3e1a167a@frystation>
@ 2021-12-11  9:12                               ` Mia
  2021-12-11  9:19                                 ` Qu Wenruo
       [not found]                                 ` <114dbcbb-c0ad-6ffd-f9ff-7aff031d03b5@rx2.rx-server.de>
  0 siblings, 2 replies; 21+ messages in thread
From: Mia @ 2021-12-11  9:12 UTC (permalink / raw)
  To: Qu Wenruo, Patrik Lundquist, linux-btrfs, Filipe Manana

Never mind, filesytem went RO again a few minutes after boot. :(

[   10.759194] BTRFS error (device sda3): tree first key mismatch 
detected, bytenr=543379832832 parent_transid=1349283 key 
expected=(468977471488,168,16384) has=(468983328768,168,61440)
[   10.762568] BTRFS error (device sda3): tree first key mismatch 
detected, bytenr=543379832832 parent_transid=1349283 key 
expected=(468977471488,168,16384) has=(468983328768,168,61440)
[   10.764642] ------------[ cut here ]------------
[   10.764644] BTRFS: Transaction aborted (error -117)
[   10.764766] WARNING: CPU: 2 PID: 2097 at fs/btrfs/extent-tree.c:2148 
btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
[   10.764767] Modules linked in: xfrm_user xfrm_algo ip6t_REJECT 
nf_reject_ipv6 ip6_tables nf_log_ipv6 br_netfilter xt_recent ipt_REJECT 
nf_reject_ipv4 xt_multiport xt_comment xt_hashlimit xt_conntrack 
xt_addrtype xt_mark xt_nat xt_CT nfnetlink_log xt_NFLOG nf_log_ipv4 
nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp 
nf_nat_sip nf_nat_pptp nft_counter nf_nat_irc nf_nat_h323 nf_nat_ftp 
nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane 
nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp 
nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast 
nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_CHECKSUM 
nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 
nf_defrag_ipv4 xt_tcpudp nft_compat wireguard libchacha20poly1305 
nf_tables chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel 
nfnetlink libblake2s blake2s_x86_64 curve25519_x86_64 
libcurve25519_generic libchacha libblake2s_generic overlay bridge stp 
llc fuse amd_energy crc32_pclmul
[   10.764814]  ext4 ghash_clmulni_intel crc16 mbcache aesni_intel jbd2 
libaes crypto_simd cryptd virtio_console glue_helper evdev sg joydev 
virtio_balloon serio_raw pcspkr qemu_fw_cfg sunrpc ip_tables x_tables 
autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_generic 
hid_generic usbhid hid sr_mod sd_mod cdrom t10_pi crc_t10dif 
crct10dif_generic ata_generic virtio_net net_failover virtio_scsi 
failover ata_piix bochs_drm drm_vram_helper drm_ttm_helper ttm libata 
psmouse drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel 
uhci_hcd scsi_mod cec ehci_hcd i2c_piix4 drm usbcore virtio_pci 
virtio_ring virtio usb_common floppy button
[   10.764867] CPU: 2 PID: 2097 Comm: dockerd Not tainted 
5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
[   10.764868] Hardware name: netcup KVM Server, BIOS RS 2000 G9 
06/01/2021
[   10.764887] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
[   10.764889] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83 f8 
fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 ed 84 c0 89 04 24 e8 ac 32 cc 
f8 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7 c6 a0
[   10.764890] RSP: 0018:ffffb84940f0fbe0 EFLAGS: 00010282
[   10.764892] RAX: 0000000000000000 RBX: ffff9fef80ec9958 RCX: 
0000000000000027
[   10.764893] RDX: 0000000000000027 RSI: ffff9ff2afd18a00 RDI: 
ffff9ff2afd18a08
[   10.764893] RBP: ffff9fef80ec9958 R08: 0000000000000000 R09: 
c0000000ffffefff
[   10.764894] R10: 0000000000000001 R11: ffffb84940f0f9e8 R12: 
ffff9fef89d63200
[   10.764895] R13: ffff9fef82ca1e30 R14: ffff9fef864e39c0 R15: 
000000000000d20c
[   10.764899] FS:  00007fe7cde3b700(0000) GS:ffff9ff2afd00000(0000) 
knlGS:0000000000000000
[   10.764900] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   10.764901] CR2: 000055c1340bad98 CR3: 0000000105cc4000 CR4: 
0000000000350ee0
[   10.764904] Call Trace:
[   10.764929]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
[   10.764949]  ? btrfs_record_root_in_trans+0x56/0x60 [btrfs]
[   10.764970]  ? start_transaction+0xd3/0x580 [btrfs]
[   10.764993]  btrfs_mksubvol+0x2b0/0x440 [btrfs]
[   10.765014]  btrfs_mksnapshot+0x75/0xa0 [btrfs]
[   10.765033]  __btrfs_ioctl_snap_create+0x167/0x170 [btrfs]
[   10.765051]  btrfs_ioctl_snap_create_v2+0x111/0x140 [btrfs]
[   10.765069]  btrfs_ioctl+0xbc4/0x2f70 [btrfs]
[   10.765085]  ? __x64_sys_ioctl+0x84/0xc0
[   10.765087]  __x64_sys_ioctl+0x84/0xc0
[   10.765112]  do_syscall_64+0x33/0x80
[   10.765127]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   10.765139] RIP: 0033:0x5555e122e5db
[   10.765141] Code: fa ff eb bf e8 46 3c fa ff e9 61 ff ff ff cc e8 1b 
01 fa ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 
05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30
[   10.765142] RSP: 002b:000000c000eaa908 EFLAGS: 00000216 ORIG_RAX: 
0000000000000010
[   10.765144] RAX: ffffffffffffffda RBX: 000000c000063000 RCX: 
00005555e122e5db
[   10.765145] RDX: 000000c000fa1000 RSI: 0000000050009417 RDI: 
0000000000000018
[   10.765145] RBP: 000000c000eaa9a8 R08: 000000c000ad4c00 R09: 
0000000000000000
[   10.765146] R10: 0000000000000000 R11: 0000000000000216 R12: 
00000000000001ce
[   10.765147] R13: 00000000000001cd R14: 0000000000000200 R15: 
ffffffffffffffff
[   10.765150] ---[ end trace 5f3f8ae993a39924 ]---
[   10.765153] BTRFS: error (device sda3) in 
btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
[   10.766038] BTRFS info (device sda3): forced readonly


------ Originalnachricht ------
Von: "Mia" <9speysdx24@kr33.de>
An: "Qu Wenruo" <quwenruo.btrfs@gmx.com>; "Patrik Lundquist" 
<patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe 
Manana" <fdmanana@kernel.org>
Gesendet: 11.12.2021 10:10:25
Betreff: Re: filesystem corrupt - error -117

>Hi Qu,
>
>thanks for the quick reply.
>As suggested I did a btrfs rescue zero-log /dev/sda3
>and I was able to mount and than boot the server again.
>
>Are there any further steps I can do do verify overall integrity?
>Since my trust in btrfs isn't that great anymore.
>
>Thanks again for your help!
>Mia
>
>------ Originalnachricht ------
>Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>An: "Mia" <9speysdx24@kr33.de>; "Patrik Lundquist" <patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe Manana" <fdmanana@kernel.org>
>Gesendet: 11.12.2021 09:39:01
>Betreff: Re: filesystem corrupt - error -117
>
>>
>>
>>On 2021/12/11 16:28, Mia wrote:
>>>Hi,
>>>
>>>just an addition, I'm now unable to mount and boot the server.
>>>
>>>https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing
>>
>>This is just log tree, thus btrfs-rescue zero-log can easily repair that.
>>
>>This bug looks like something fixed by Filipe, but I'm not familiar
>>enough with log tree code.
>>
>>Add Filipe to this thread.
>>
>>Thanks,
>>Qu
>>
>>>
>>>
>>>Regards
>>>Mia
>>>
>>>------ Originalnachricht ------
>>>Von: "Mia" <9speysdx24@kr33.de>
>>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>><quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
>>>Gesendet: 11.12.2021 09:20:39
>>>Betreff: Re: filesystem corrupt - error -117
>>>
>>>>Hello,
>>>>
>>>>after the switching to the newer kernel I had no further problems
>>>>until now.
>>>>It happened again.
>>>>
>>>>[1653860.040458] BTRFS error (device sda3): tree first key mismatch
>>>>detected, bytenr=543379832832 parent_transid=1349283 key
>>>>expected=(468977471488,168,16384) has=(468983328768,168,61440)
>>>>[1653860.041095] ------------[ cut here ]------------
>>>>[1653860.041098] BTRFS: Transaction aborted (error -117)
>>>>[1653860.041289] WARNING: CPU: 2 PID: 219 at
>>>>fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>[1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT
>>>>nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter
>>>>xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment
>>>>xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT
>>>>nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp
>>>>nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc
>>>>nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda
>>>>nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp
>>>>nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>>>>nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter
>>>>nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack
>>>>nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard
>>>>libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64
>>>>ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64
>>>>curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic
>>>>overlay bridge stp llc fuse ext4
>>>>[1653860.041392]  amd_energy crc32_pclmul crc16 mbcache
>>>>ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd
>>>>glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw
>>>>pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs
>>>>blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic
>>>>usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic
>>>>ata_generic virtio_net net_failover virtio_scsi failover bochs_drm
>>>>drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix
>>>>uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common
>>>>virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod
>>>>i2c_piix4 floppy button
>>>>[1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted:
>>>>G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>>>>[1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9
>>>>06/01/2021
>>>>[1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>[1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83
>>>>f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac
>>>>12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7
>>>>c6 a0
>>>>[1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>>>>[1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX:
>>>>0000000000000027
>>>>[1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI:
>>>>ffff9d54efd18a08
>>>>[1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09:
>>>>c0000000ffffefff
>>>>[1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12:
>>>>ffff9d51f0cda800
>>>>[1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15:
>>>>0000000000000cca
>>>>[1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000)
>>>>knlGS:0000000000000000
>>>>[1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>[1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4:
>>>>0000000000350ee0
>>>>[1653860.041583] Call Trace:
>>>>[1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>>>>[1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>>>>[1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>>>>[1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>>>>[1653860.041817]  kthread+0x116/0x130
>>>>[1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>>>>[1653860.041826]  ret_from_fork+0x22/0x30
>>>>[1653860.041831] ---[ end trace af79c203a5452514 ]---
>>>>[1653860.041835] BTRFS: error (device sda3) in
>>>>btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>>>>[1653860.042111] BTRFS info (device sda3): forced readonly
>>>>
>>>>root@rx1 ~ # btrfs fi show
>>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>     Total devices 1 FS bytes used 186.78GiB
>>>>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>
>>>>root@rx1 ~ # btrfs fi df
>>>>/
>>>>:(
>>>>Data, single: total=194.89GiB, used=185.16GiB
>>>>System, single: total=32.00MiB, used=48.00KiB
>>>>Metadata, single: total=4.16GiB, used=1.62GiB
>>>>GlobalReserve, single: total=385.84MiB, used=0.00B
>>>>
>>>>root@rx1 ~ # btrfs --version
>>>>btrfs-progs v5.10.1
>>>>
>>>>root@rx1 ~ # uname -a
>>>>Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1
>>>>(2021-10-10) x86_64 GNU/Linux
>>>>
>>>>Regards
>>>>Mia
>>>>
>>>>------ Originalnachricht ------
>>>>Von: "Mia" <9speysdx24@kr33.de>
>>>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>>><quwenruo.btrfs@gmx.com>
>>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>Gesendet: 26.10.2021 19:28:14
>>>>Betreff: Re: filesystem corrupt - error -117
>>>>
>>>>>Hi Patrik,
>>>>>
>>>>>good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian
>>>>>5.10.46-4~bpo10+1 (2021-08-07) now.
>>>>>
>>>>>Hi Qu,
>>>>>regarding the memtest. This is a virtual machine, I have no access to
>>>>>the host system.
>>>>>I don't know if a memtest inside the vm would bring legit results?
>>>>>
>>>>>Regards
>>>>>Mia
>>>>>
>>>>>------ Originalnachricht ------
>>>>>Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>>>>An: "Mia" <9speysdx24@kr33.de>
>>>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>>Gesendet: 26.10.2021 16:12:45
>>>>>Betreff: Re: filesystem corrupt - error -117
>>>>>
>>>>>>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>>>>
>>>>>>>  Problem with updating is that this is currently still Debian 10 and a
>>>>>>>  production environment and I don't know when it is possible to
>>>>>>>upgrade
>>>>>>>  because of dependencies.
>>>>>>
>>>>>>Maybe you can install the buster-backports kernel which currently is
>>>>>>5.10.70?
>>>>>>
>>>>>>https://backports.debian.org/Instructions/
>>>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
  2021-12-11  9:12                               ` Mia
@ 2021-12-11  9:19                                 ` Qu Wenruo
       [not found]                                 ` <114dbcbb-c0ad-6ffd-f9ff-7aff031d03b5@rx2.rx-server.de>
  1 sibling, 0 replies; 21+ messages in thread
From: Qu Wenruo @ 2021-12-11  9:19 UTC (permalink / raw)
  To: Mia, Qu Wenruo, Patrik Lundquist, linux-btrfs, Filipe Manana



On 2021/12/11 17:12, Mia wrote:
> Never mind, filesytem went RO again a few minutes after boot. :(
> 
> [   10.759194] BTRFS error (device sda3): tree first key mismatch 
> detected, bytenr=543379832832 parent_transid=1349283 key 
> expected=(468977471488,168,16384) has=(468983328768,168,61440)
> [   10.762568] BTRFS error (device sda3): tree first key mismatch 
> detected, bytenr=543379832832 parent_transid=1349283 key 
> expected=(468977471488,168,16384) has=(468983328768,168,61440)

This is from extent tree, and it looks like there is some corruption in 
the extent tree already.

Btrfs check is recommended to verify if the corruption is already 
committed to disk.

I don't have any good idea on this problem any more, if it's really a 
corruption in extent tree, it's not going to end up well...

Thanks,
Qu

> [   10.764642] ------------[ cut here ]------------
> [   10.764644] BTRFS: Transaction aborted (error -117)
> [   10.764766] WARNING: CPU: 2 PID: 2097 at fs/btrfs/extent-tree.c:2148 
> btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
> [   10.764767] Modules linked in: xfrm_user xfrm_algo ip6t_REJECT 
> nf_reject_ipv6 ip6_tables nf_log_ipv6 br_netfilter xt_recent ipt_REJECT 
> nf_reject_ipv4 xt_multiport xt_comment xt_hashlimit xt_conntrack 
> xt_addrtype xt_mark xt_nat xt_CT nfnetlink_log xt_NFLOG nf_log_ipv4 
> nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp 
> nf_nat_sip nf_nat_pptp nft_counter nf_nat_irc nf_nat_h323 nf_nat_ftp 
> nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane 
> nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp 
> nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast 
> nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_CHECKSUM 
> nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 
> nf_defrag_ipv4 xt_tcpudp nft_compat wireguard libchacha20poly1305 
> nf_tables chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel 
> nfnetlink libblake2s blake2s_x86_64 curve25519_x86_64 
> libcurve25519_generic libchacha libblake2s_generic overlay bridge stp 
> llc fuse amd_energy crc32_pclmul
> [   10.764814]  ext4 ghash_clmulni_intel crc16 mbcache aesni_intel jbd2 
> libaes crypto_simd cryptd virtio_console glue_helper evdev sg joydev 
> virtio_balloon serio_raw pcspkr qemu_fw_cfg sunrpc ip_tables x_tables 
> autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_generic 
> hid_generic usbhid hid sr_mod sd_mod cdrom t10_pi crc_t10dif 
> crct10dif_generic ata_generic virtio_net net_failover virtio_scsi 
> failover ata_piix bochs_drm drm_vram_helper drm_ttm_helper ttm libata 
> psmouse drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel 
> uhci_hcd scsi_mod cec ehci_hcd i2c_piix4 drm usbcore virtio_pci 
> virtio_ring virtio usb_common floppy button
> [   10.764867] CPU: 2 PID: 2097 Comm: dockerd Not tainted 
> 5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
> [   10.764868] Hardware name: netcup KVM Server, BIOS RS 2000 G9 06/01/2021
> [   10.764887] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
> [   10.764889] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83 f8 
> fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 ed 84 c0 89 04 24 e8 ac 32 cc 
> f8 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7 c6 a0
> [   10.764890] RSP: 0018:ffffb84940f0fbe0 EFLAGS: 00010282
> [   10.764892] RAX: 0000000000000000 RBX: ffff9fef80ec9958 RCX: 
> 0000000000000027
> [   10.764893] RDX: 0000000000000027 RSI: ffff9ff2afd18a00 RDI: 
> ffff9ff2afd18a08
> [   10.764893] RBP: ffff9fef80ec9958 R08: 0000000000000000 R09: 
> c0000000ffffefff
> [   10.764894] R10: 0000000000000001 R11: ffffb84940f0f9e8 R12: 
> ffff9fef89d63200
> [   10.764895] R13: ffff9fef82ca1e30 R14: ffff9fef864e39c0 R15: 
> 000000000000d20c
> [   10.764899] FS:  00007fe7cde3b700(0000) GS:ffff9ff2afd00000(0000) 
> knlGS:0000000000000000
> [   10.764900] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   10.764901] CR2: 000055c1340bad98 CR3: 0000000105cc4000 CR4: 
> 0000000000350ee0
> [   10.764904] Call Trace:
> [   10.764929]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
> [   10.764949]  ? btrfs_record_root_in_trans+0x56/0x60 [btrfs]
> [   10.764970]  ? start_transaction+0xd3/0x580 [btrfs]
> [   10.764993]  btrfs_mksubvol+0x2b0/0x440 [btrfs]
> [   10.765014]  btrfs_mksnapshot+0x75/0xa0 [btrfs]
> [   10.765033]  __btrfs_ioctl_snap_create+0x167/0x170 [btrfs]
> [   10.765051]  btrfs_ioctl_snap_create_v2+0x111/0x140 [btrfs]
> [   10.765069]  btrfs_ioctl+0xbc4/0x2f70 [btrfs]
> [   10.765085]  ? __x64_sys_ioctl+0x84/0xc0
> [   10.765087]  __x64_sys_ioctl+0x84/0xc0
> [   10.765112]  do_syscall_64+0x33/0x80
> [   10.765127]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [   10.765139] RIP: 0033:0x5555e122e5db
> [   10.765141] Code: fa ff eb bf e8 46 3c fa ff e9 61 ff ff ff cc e8 1b 
> 01 fa ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 
> 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30
> [   10.765142] RSP: 002b:000000c000eaa908 EFLAGS: 00000216 ORIG_RAX: 
> 0000000000000010
> [   10.765144] RAX: ffffffffffffffda RBX: 000000c000063000 RCX: 
> 00005555e122e5db
> [   10.765145] RDX: 000000c000fa1000 RSI: 0000000050009417 RDI: 
> 0000000000000018
> [   10.765145] RBP: 000000c000eaa9a8 R08: 000000c000ad4c00 R09: 
> 0000000000000000
> [   10.765146] R10: 0000000000000000 R11: 0000000000000216 R12: 
> 00000000000001ce
> [   10.765147] R13: 00000000000001cd R14: 0000000000000200 R15: 
> ffffffffffffffff
> [   10.765150] ---[ end trace 5f3f8ae993a39924 ]---
> [   10.765153] BTRFS: error (device sda3) in 
> btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
> [   10.766038] BTRFS info (device sda3): forced readonly
> 
> 
> ------ Originalnachricht ------
> Von: "Mia" <9speysdx24@kr33.de>
> An: "Qu Wenruo" <quwenruo.btrfs@gmx.com>; "Patrik Lundquist" 
> <patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe 
> Manana" <fdmanana@kernel.org>
> Gesendet: 11.12.2021 10:10:25
> Betreff: Re: filesystem corrupt - error -117
> 
>> Hi Qu,
>>
>> thanks for the quick reply.
>> As suggested I did a btrfs rescue zero-log /dev/sda3
>> and I was able to mount and than boot the server again.
>>
>> Are there any further steps I can do do verify overall integrity?
>> Since my trust in btrfs isn't that great anymore.
>>
>> Thanks again for your help!
>> Mia
>>
>> ------ Originalnachricht ------
>> Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>> An: "Mia" <9speysdx24@kr33.de>; "Patrik Lundquist" 
>> <patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe 
>> Manana" <fdmanana@kernel.org>
>> Gesendet: 11.12.2021 09:39:01
>> Betreff: Re: filesystem corrupt - error -117
>>
>>>
>>>
>>> On 2021/12/11 16:28, Mia wrote:
>>>> Hi,
>>>>
>>>> just an addition, I'm now unable to mount and boot the server.
>>>>
>>>> https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing 
>>>>
>>>
>>> This is just log tree, thus btrfs-rescue zero-log can easily repair 
>>> that.
>>>
>>> This bug looks like something fixed by Filipe, but I'm not familiar
>>> enough with log tree code.
>>>
>>> Add Filipe to this thread.
>>>
>>> Thanks,
>>> Qu
>>>
>>>>
>>>>
>>>> Regards
>>>> Mia
>>>>
>>>> ------ Originalnachricht ------
>>>> Von: "Mia" <9speysdx24@kr33.de>
>>>> An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>>> <quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
>>>> Gesendet: 11.12.2021 09:20:39
>>>> Betreff: Re: filesystem corrupt - error -117
>>>>
>>>>> Hello,
>>>>>
>>>>> after the switching to the newer kernel I had no further problems
>>>>> until now.
>>>>> It happened again.
>>>>>
>>>>> [1653860.040458] BTRFS error (device sda3): tree first key mismatch
>>>>> detected, bytenr=543379832832 parent_transid=1349283 key
>>>>> expected=(468977471488,168,16384) has=(468983328768,168,61440)
>>>>> [1653860.041095] ------------[ cut here ]------------
>>>>> [1653860.041098] BTRFS: Transaction aborted (error -117)
>>>>> [1653860.041289] WARNING: CPU: 2 PID: 219 at
>>>>> fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>> [1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT
>>>>> nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter
>>>>> xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment
>>>>> xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT
>>>>> nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp
>>>>> nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc
>>>>> nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda
>>>>> nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp
>>>>> nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>>>>> nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter
>>>>> nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack
>>>>> nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard
>>>>> libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64
>>>>> ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64
>>>>> curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic
>>>>> overlay bridge stp llc fuse ext4
>>>>> [1653860.041392]  amd_energy crc32_pclmul crc16 mbcache
>>>>> ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd
>>>>> glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw
>>>>> pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs
>>>>> blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic
>>>>> usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic
>>>>> ata_generic virtio_net net_failover virtio_scsi failover bochs_drm
>>>>> drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix
>>>>> uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common
>>>>> virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod
>>>>> i2c_piix4 floppy button
>>>>> [1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted:
>>>>> G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>>>>> [1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9
>>>>> 06/01/2021
>>>>> [1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>> [1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83
>>>>> f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac
>>>>> 12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7
>>>>> c6 a0
>>>>> [1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>>>>> [1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX:
>>>>> 0000000000000027
>>>>> [1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI:
>>>>> ffff9d54efd18a08
>>>>> [1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09:
>>>>> c0000000ffffefff
>>>>> [1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12:
>>>>> ffff9d51f0cda800
>>>>> [1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15:
>>>>> 0000000000000cca
>>>>> [1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000)
>>>>> knlGS:0000000000000000
>>>>> [1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>> [1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4:
>>>>> 0000000000350ee0
>>>>> [1653860.041583] Call Trace:
>>>>> [1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>>>>> [1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>>>>> [1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>>>>> [1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>>>>> [1653860.041817]  kthread+0x116/0x130
>>>>> [1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>>>>> [1653860.041826]  ret_from_fork+0x22/0x30
>>>>> [1653860.041831] ---[ end trace af79c203a5452514 ]---
>>>>> [1653860.041835] BTRFS: error (device sda3) in
>>>>> btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>>>>> [1653860.042111] BTRFS info (device sda3): forced readonly
>>>>>
>>>>> root@rx1 ~ # btrfs fi show
>>>>> Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>     Total devices 1 FS bytes used 186.78GiB
>>>>>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>
>>>>> root@rx1 ~ # btrfs fi df
>>>>> /
>>>>> :(
>>>>> Data, single: total=194.89GiB, used=185.16GiB
>>>>> System, single: total=32.00MiB, used=48.00KiB
>>>>> Metadata, single: total=4.16GiB, used=1.62GiB
>>>>> GlobalReserve, single: total=385.84MiB, used=0.00B
>>>>>
>>>>> root@rx1 ~ # btrfs --version
>>>>> btrfs-progs v5.10.1
>>>>>
>>>>> root@rx1 ~ # uname -a
>>>>> Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1
>>>>> (2021-10-10) x86_64 GNU/Linux
>>>>>
>>>>> Regards
>>>>> Mia
>>>>>
>>>>> ------ Originalnachricht ------
>>>>> Von: "Mia" <9speysdx24@kr33.de>
>>>>> An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>>>> <quwenruo.btrfs@gmx.com>
>>>>> Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>> Gesendet: 26.10.2021 19:28:14
>>>>> Betreff: Re: filesystem corrupt - error -117
>>>>>
>>>>>> Hi Patrik,
>>>>>>
>>>>>> good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian
>>>>>> 5.10.46-4~bpo10+1 (2021-08-07) now.
>>>>>>
>>>>>> Hi Qu,
>>>>>> regarding the memtest. This is a virtual machine, I have no access to
>>>>>> the host system.
>>>>>> I don't know if a memtest inside the vm would bring legit results?
>>>>>>
>>>>>> Regards
>>>>>> Mia
>>>>>>
>>>>>> ------ Originalnachricht ------
>>>>>> Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>>>>> An: "Mia" <9speysdx24@kr33.de>
>>>>>> Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>>> Gesendet: 26.10.2021 16:12:45
>>>>>> Betreff: Re: filesystem corrupt - error -117
>>>>>>
>>>>>>> On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>>>>>
>>>>>>>>  Problem with updating is that this is currently still Debian 10 
>>>>>>>> and a
>>>>>>>>  production environment and I don't know when it is possible to
>>>>>>>> upgrade
>>>>>>>>  because of dependencies.
>>>>>>>
>>>>>>> Maybe you can install the buster-backports kernel which currently is
>>>>>>> 5.10.70?
>>>>>>>
>>>>>>> https://backports.debian.org/Instructions/
>>>>
> 


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: filesystem corrupt - error -117
       [not found]                                 ` <114dbcbb-c0ad-6ffd-f9ff-7aff031d03b5@rx2.rx-server.de>
@ 2021-12-11  9:34                                   ` Mia
  0 siblings, 0 replies; 21+ messages in thread
From: Mia @ 2021-12-11  9:34 UTC (permalink / raw)
  To: Qu Wenruo, Qu Wenruo, Patrik Lundquist, linux-btrfs, Filipe Manana

Okay that doesn't sound good.
Since I got it mounted RO at the moment I'm rsycning all the data to 
another server.
Hope this goes well.

Thanks
Mia

------ Originalnachricht ------
Von: "Qu Wenruo" <wqu@suse.com>
An: "Mia" <9speysdx24@kr33.de>; "Qu Wenruo" <quwenruo.btrfs@gmx.com>; 
"Patrik Lundquist" <patrik.lundquist@gmail.com>; 
linux-btrfs@vger.kernel.org; "Filipe Manana" <fdmanana@kernel.org>
Gesendet: 11.12.2021 10:19:03
Betreff: Re: filesystem corrupt - error -117

>
>
>On 2021/12/11 17:12, Mia wrote:
>>Never mind, filesytem went RO again a few minutes after boot. :(
>>
>>[   10.759194] BTRFS error (device sda3): tree first key mismatch detected, bytenr=543379832832 parent_transid=1349283 key expected=(468977471488,168,16384) has=(468983328768,168,61440)
>>[   10.762568] BTRFS error (device sda3): tree first key mismatch detected, bytenr=543379832832 parent_transid=1349283 key expected=(468977471488,168,16384) has=(468983328768,168,61440)
>
>This is from extent tree, and it looks like there is some corruption in the extent tree already.
>
>Btrfs check is recommended to verify if the corruption is already committed to disk.
>
>I don't have any good idea on this problem any more, if it's really a corruption in extent tree, it's not going to end up well...
>
>Thanks,
>Qu
>
>>[   10.764642] ------------[ cut here ]------------
>>[   10.764644] BTRFS: Transaction aborted (error -117)
>>[   10.764766] WARNING: CPU: 2 PID: 2097 at fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>[   10.764767] Modules linked in: xfrm_user xfrm_algo ip6t_REJECT nf_reject_ipv6 ip6_tables nf_log_ipv6 br_netfilter xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nft_counter nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_CHECKSUM nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard libchacha20poly1305 nf_tables chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel nfnetlink libblake2s blake2s_x86_64 curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic overlay bridge stp llc fuse amd_energy crc32_pclmul
>>[   10.764814]  ext4 ghash_clmulni_intel crc16 mbcache aesni_intel jbd2 libaes crypto_simd cryptd virtio_console glue_helper evdev sg joydev virtio_balloon serio_raw pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic usbhid hid sr_mod sd_mod cdrom t10_pi crc_t10dif crct10dif_generic ata_generic virtio_net net_failover virtio_scsi failover ata_piix bochs_drm drm_vram_helper drm_ttm_helper ttm libata psmouse drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel uhci_hcd scsi_mod cec ehci_hcd i2c_piix4 drm usbcore virtio_pci virtio_ring virtio usb_common floppy button
>>[   10.764867] CPU: 2 PID: 2097 Comm: dockerd Not tainted 5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>>[   10.764868] Hardware name: netcup KVM Server, BIOS RS 2000 G9 06/01/2021
>>[   10.764887] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>[   10.764889] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83 f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 ed 84 c0 89 04 24 e8 ac 32 cc f8 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7 c6 a0
>>[   10.764890] RSP: 0018:ffffb84940f0fbe0 EFLAGS: 00010282
>>[   10.764892] RAX: 0000000000000000 RBX: ffff9fef80ec9958 RCX: 0000000000000027
>>[   10.764893] RDX: 0000000000000027 RSI: ffff9ff2afd18a00 RDI: ffff9ff2afd18a08
>>[   10.764893] RBP: ffff9fef80ec9958 R08: 0000000000000000 R09: c0000000ffffefff
>>[   10.764894] R10: 0000000000000001 R11: ffffb84940f0f9e8 R12: ffff9fef89d63200
>>[   10.764895] R13: ffff9fef82ca1e30 R14: ffff9fef864e39c0 R15: 000000000000d20c
>>[   10.764899] FS:  00007fe7cde3b700(0000) GS:ffff9ff2afd00000(0000) knlGS:0000000000000000
>>[   10.764900] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>[   10.764901] CR2: 000055c1340bad98 CR3: 0000000105cc4000 CR4: 0000000000350ee0
>>[   10.764904] Call Trace:
>>[   10.764929]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>>[   10.764949]  ? btrfs_record_root_in_trans+0x56/0x60 [btrfs]
>>[   10.764970]  ? start_transaction+0xd3/0x580 [btrfs]
>>[   10.764993]  btrfs_mksubvol+0x2b0/0x440 [btrfs]
>>[   10.765014]  btrfs_mksnapshot+0x75/0xa0 [btrfs]
>>[   10.765033]  __btrfs_ioctl_snap_create+0x167/0x170 [btrfs]
>>[   10.765051]  btrfs_ioctl_snap_create_v2+0x111/0x140 [btrfs]
>>[   10.765069]  btrfs_ioctl+0xbc4/0x2f70 [btrfs]
>>[   10.765085]  ? __x64_sys_ioctl+0x84/0xc0
>>[   10.765087]  __x64_sys_ioctl+0x84/0xc0
>>[   10.765112]  do_syscall_64+0x33/0x80
>>[   10.765127]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>[   10.765139] RIP: 0033:0x5555e122e5db
>>[   10.765141] Code: fa ff eb bf e8 46 3c fa ff e9 61 ff ff ff cc e8 1b 01 fa ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30
>>[   10.765142] RSP: 002b:000000c000eaa908 EFLAGS: 00000216 ORIG_RAX: 0000000000000010
>>[   10.765144] RAX: ffffffffffffffda RBX: 000000c000063000 RCX: 00005555e122e5db
>>[   10.765145] RDX: 000000c000fa1000 RSI: 0000000050009417 RDI: 0000000000000018
>>[   10.765145] RBP: 000000c000eaa9a8 R08: 000000c000ad4c00 R09: 0000000000000000
>>[   10.765146] R10: 0000000000000000 R11: 0000000000000216 R12: 00000000000001ce
>>[   10.765147] R13: 00000000000001cd R14: 0000000000000200 R15: ffffffffffffffff
>>[   10.765150] ---[ end trace 5f3f8ae993a39924 ]---
>>[   10.765153] BTRFS: error (device sda3) in btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>>[   10.766038] BTRFS info (device sda3): forced readonly
>>
>>
>>------ Originalnachricht ------
>>Von: "Mia" <9speysdx24@kr33.de>
>>An: "Qu Wenruo" <quwenruo.btrfs@gmx.com>; "Patrik Lundquist" <patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe Manana" <fdmanana@kernel.org>
>>Gesendet: 11.12.2021 10:10:25
>>Betreff: Re: filesystem corrupt - error -117
>>
>>>Hi Qu,
>>>
>>>thanks for the quick reply.
>>>As suggested I did a btrfs rescue zero-log /dev/sda3
>>>and I was able to mount and than boot the server again.
>>>
>>>Are there any further steps I can do do verify overall integrity?
>>>Since my trust in btrfs isn't that great anymore.
>>>
>>>Thanks again for your help!
>>>Mia
>>>
>>>------ Originalnachricht ------
>>>Von: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>>>An: "Mia" <9speysdx24@kr33.de>; "Patrik Lundquist" <patrik.lundquist@gmail.com>; linux-btrfs@vger.kernel.org; "Filipe Manana" <fdmanana@kernel.org>
>>>Gesendet: 11.12.2021 09:39:01
>>>Betreff: Re: filesystem corrupt - error -117
>>>
>>>>
>>>>
>>>>On 2021/12/11 16:28, Mia wrote:
>>>>>Hi,
>>>>>
>>>>>just an addition, I'm now unable to mount and boot the server.
>>>>>
>>>>>https://drive.google.com/file/d/1QfqCR7oqKbaTxokCGWVmqNCR5vMul7Xy/view?usp=sharing
>>>>
>>>>This is just log tree, thus btrfs-rescue zero-log can easily repair that.
>>>>
>>>>This bug looks like something fixed by Filipe, but I'm not familiar
>>>>enough with log tree code.
>>>>
>>>>Add Filipe to this thread.
>>>>
>>>>Thanks,
>>>>Qu
>>>>
>>>>>
>>>>>
>>>>>Regards
>>>>>Mia
>>>>>
>>>>>------ Originalnachricht ------
>>>>>Von: "Mia" <9speysdx24@kr33.de>
>>>>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>>>><quwenruo.btrfs@gmx.com>; linux-btrfs@vger.kernel.org
>>>>>Gesendet: 11.12.2021 09:20:39
>>>>>Betreff: Re: filesystem corrupt - error -117
>>>>>
>>>>>>Hello,
>>>>>>
>>>>>>after the switching to the newer kernel I had no further problems
>>>>>>until now.
>>>>>>It happened again.
>>>>>>
>>>>>>[1653860.040458] BTRFS error (device sda3): tree first key mismatch
>>>>>>detected, bytenr=543379832832 parent_transid=1349283 key
>>>>>>expected=(468977471488,168,16384) has=(468983328768,168,61440)
>>>>>>[1653860.041095] ------------[ cut here ]------------
>>>>>>[1653860.041098] BTRFS: Transaction aborted (error -117)
>>>>>>[1653860.041289] WARNING: CPU: 2 PID: 219 at
>>>>>>fs/btrfs/extent-tree.c:2148 btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>>>[1653860.041291] Modules linked in: dm_mod veth ip6t_REJECT
>>>>>>nf_reject_ipv6 ip6_tables nf_log_ipv6 xfrm_user xfrm_algo br_netfilter
>>>>>>xt_recent ipt_REJECT nf_reject_ipv4 xt_multiport xt_comment
>>>>>>xt_hashlimit xt_conntrack xt_addrtype xt_mark xt_nat xt_CT
>>>>>>nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp
>>>>>>nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_irc
>>>>>>nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda
>>>>>>nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp
>>>>>>nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast
>>>>>>nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nft_counter
>>>>>>nft_chain_nat xt_CHECKSUM xt_MASQUERADE nf_nat nf_conntrack
>>>>>>nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp nft_compat wireguard
>>>>>>libchacha20poly1305 chacha_x86_64 nf_tables poly1305_x86_64
>>>>>>ip6_udp_tunnel udp_tunnel libblake2s nfnetlink blake2s_x86_64
>>>>>>curve25519_x86_64 libcurve25519_generic libchacha libblake2s_generic
>>>>>>overlay bridge stp llc fuse ext4
>>>>>>[1653860.041392]  amd_energy crc32_pclmul crc16 mbcache
>>>>>>ghash_clmulni_intel jbd2 aesni_intel libaes crypto_simd cryptd
>>>>>>glue_helper joydev sg virtio_console virtio_balloon evdev serio_raw
>>>>>>pcspkr qemu_fw_cfg sunrpc ip_tables x_tables autofs4 btrfs
>>>>>>blake2b_generic xor raid6_pq libcrc32c crc32c_generic hid_generic
>>>>>>usbhid hid sd_mod sr_mod cdrom t10_pi crc_t10dif crct10dif_generic
>>>>>>ata_generic virtio_net net_failover virtio_scsi failover bochs_drm
>>>>>>drm_vram_helper drm_ttm_helper ttm drm_kms_helper cec ata_piix
>>>>>>uhci_hcd drm libata ehci_hcd usbcore crct10dif_pclmul crct10dif_common
>>>>>>virtio_pci crc32c_intel virtio_ring psmouse virtio usb_common scsi_mod
>>>>>>i2c_piix4 floppy button
>>>>>>[1653860.041483] CPU: 2 PID: 219 Comm: btrfs-transacti Tainted:
>>>>>>G             L    5.10.0-0.bpo.9-amd64 #1 Debian 5.10.70-1~bpo10+1
>>>>>>[1653860.041484] Hardware name: netcup KVM Server, BIOS RS 2000 G9
>>>>>>06/01/2021
>>>>>>[1653860.041530] RIP: 0010:btrfs_run_delayed_refs+0x1a6/0x1f0 [btrfs]
>>>>>>[1653860.041535] Code: 8b 55 50 f0 48 0f ba aa 40 0a 00 00 02 72 20 83
>>>>>>f8 fb 74 39 83 f8 e2 74 34 89 c6 48 c7 c7 68 0d 86 c0 89 04 24 e8 ac
>>>>>>12 4b e7 <0f> 0b 8b 04 24 89 c1 ba 64 08 00 00 48 89 ef 89 04 24 48 c7
>>>>>>c6 a0
>>>>>>[1653860.041538] RSP: 0018:ffffa9ea8040fe10 EFLAGS: 00010282
>>>>>>[1653860.041541] RAX: 0000000000000000 RBX: ffff9d51c5fb9f70 RCX:
>>>>>>0000000000000027
>>>>>>[1653860.041543] RDX: 0000000000000027 RSI: ffff9d54efd18a00 RDI:
>>>>>>ffff9d54efd18a08
>>>>>>[1653860.041546] RBP: ffff9d51c5fb9f70 R08: 0000000000000000 R09:
>>>>>>c0000000ffffefff
>>>>>>[1653860.041548] R10: 0000000000000001 R11: ffffa9ea8040fc18 R12:
>>>>>>ffff9d51f0cda800
>>>>>>[1653860.041550] R13: ffff9d51e183b460 R14: ffff9d51e183b340 R15:
>>>>>>0000000000000cca
>>>>>>[1653860.041556] FS:  0000000000000000(0000) GS:ffff9d54efd00000(0000)
>>>>>>knlGS:0000000000000000
>>>>>>[1653860.041558] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>>>>[1653860.041561] CR2: 00007fd3ecdd1000 CR3: 0000000102288000 CR4:
>>>>>>0000000000350ee0
>>>>>>[1653860.041583] Call Trace:
>>>>>>[1653860.041675]  btrfs_commit_transaction+0x57/0xb70 [btrfs]
>>>>>>[1653860.041722]  ? start_transaction+0xd3/0x580 [btrfs]
>>>>>>[1653860.041767]  transaction_kthread+0x15d/0x180 [btrfs]
>>>>>>[1653860.041810]  ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
>>>>>>[1653860.041817]  kthread+0x116/0x130
>>>>>>[1653860.041821]  ? __kthread_cancel_work+0x40/0x40
>>>>>>[1653860.041826]  ret_from_fork+0x22/0x30
>>>>>>[1653860.041831] ---[ end trace af79c203a5452514 ]---
>>>>>>[1653860.041835] BTRFS: error (device sda3) in
>>>>>>btrfs_run_delayed_refs:2148: errno=-117 Filesystem corrupted
>>>>>>[1653860.042111] BTRFS info (device sda3): forced readonly
>>>>>>
>>>>>>root@rx1 ~ # btrfs fi show
>>>>>>Label: none  uuid: 21306973-6bf3-4877-9543-633d472dcb46
>>>>>>     Total devices 1 FS bytes used 186.78GiB
>>>>>>     devid    1 size 319.00GiB used 199.08GiB path /dev/sda3
>>>>>>
>>>>>>root@rx1 ~ # btrfs fi df
>>>>>>/
>>>>>>:(
>>>>>>Data, single: total=194.89GiB, used=185.16GiB
>>>>>>System, single: total=32.00MiB, used=48.00KiB
>>>>>>Metadata, single: total=4.16GiB, used=1.62GiB
>>>>>>GlobalReserve, single: total=385.84MiB, used=0.00B
>>>>>>
>>>>>>root@rx1 ~ # btrfs --version
>>>>>>btrfs-progs v5.10.1
>>>>>>
>>>>>>root@rx1 ~ # uname -a
>>>>>>Linux rx1 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1
>>>>>>(2021-10-10) x86_64 GNU/Linux
>>>>>>
>>>>>>Regards
>>>>>>Mia
>>>>>>
>>>>>>------ Originalnachricht ------
>>>>>>Von: "Mia" <9speysdx24@kr33.de>
>>>>>>An: "Patrik Lundquist" <patrik.lundquist@gmail.com>; "Qu Wenruo"
>>>>>><quwenruo.btrfs@gmx.com>
>>>>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>>>Gesendet: 26.10.2021 19:28:14
>>>>>>Betreff: Re: filesystem corrupt - error -117
>>>>>>
>>>>>>>Hi Patrik,
>>>>>>>
>>>>>>>good suggestion, I'm on 5.10.0-0.bpo.8-amd64 #1 SMP Debian
>>>>>>>5.10.46-4~bpo10+1 (2021-08-07) now.
>>>>>>>
>>>>>>>Hi Qu,
>>>>>>>regarding the memtest. This is a virtual machine, I have no access to
>>>>>>>the host system.
>>>>>>>I don't know if a memtest inside the vm would bring legit results?
>>>>>>>
>>>>>>>Regards
>>>>>>>Mia
>>>>>>>
>>>>>>>------ Originalnachricht ------
>>>>>>>Von: "Patrik Lundquist" <patrik.lundquist@gmail.com>
>>>>>>>An: "Mia" <9speysdx24@kr33.de>
>>>>>>>Cc: "Linux Btrfs" <linux-btrfs@vger.kernel.org>
>>>>>>>Gesendet: 26.10.2021 16:12:45
>>>>>>>Betreff: Re: filesystem corrupt - error -117
>>>>>>>
>>>>>>>>On Tue, 26 Oct 2021 at 09:15, Mia <9speysdx24@kr33.de> wrote:
>>>>>>>>>
>>>>>>>>>  Problem with updating is that this is currently still Debian 10 and a
>>>>>>>>>  production environment and I don't know when it is possible to
>>>>>>>>>upgrade
>>>>>>>>>  because of dependencies.
>>>>>>>>
>>>>>>>>Maybe you can install the buster-backports kernel which currently is
>>>>>>>>5.10.70?
>>>>>>>>
>>>>>>>>https://backports.debian.org/Instructions/
>>>>>
>>
>


^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2021-12-11  9:34 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <em969af203-04e6-4eff-a115-1129ae853867@frystation>
2021-10-25  8:46 ` filesystem corrupt - error -117 Mia
2021-10-25 10:53   ` Qu Wenruo
2021-10-25 10:55     ` Qu Wenruo
     [not found]     ` <69109d24-efa7-b9d1-e1df-c79b3989e7bf@rx2.rx-server.de>
2021-10-25 11:13       ` Mia
2021-10-25 11:14         ` Qu Wenruo
2021-10-25 11:18           ` Qu Wenruo
     [not found]           ` <884d76d1-5836-9a91-a39b-41c37441e020@rx2.rx-server.de>
2021-10-25 17:09             ` Mia
2021-10-25 22:45               ` Qu Wenruo
     [not found]               ` <3ce1dd17-b574-abe3-d6cc-eb16f00117cc@rx2.rx-server.de>
2021-10-26  6:03                 ` Mia
2021-10-26  7:24                   ` Qu Wenruo
2021-10-26 11:18                     ` Robert Krig
2021-10-26 14:12                   ` Patrik Lundquist
     [not found]                   ` <CAA7pwKOrgt6syr5C3X1+bC14QXZEJ+8HTMZruBBPBT574zNkkQ@rx2.rx-server.de>
2021-10-26 17:28                     ` Mia
2021-10-27  2:49                       ` Qu Wenruo
     [not found]                     ` <emb611c0ff-705d-4c01-b50f-320f962f39fb@frystation>
2021-12-11  8:20                       ` Mia
     [not found]                       ` <embddc7343-8fdf-4be8-87d8-644e20ea86c0@frystation>
2021-12-11  8:28                         ` Mia
2021-12-11  8:39                           ` Qu Wenruo
     [not found]                           ` <29fcb603-506f-d721-5214-2870ce2f8773@rx2.rx-server.de>
2021-12-11  9:10                             ` Mia
     [not found]                             ` <em7854e1bc-eb3d-43a3-abc7-c6ed3e1a167a@frystation>
2021-12-11  9:12                               ` Mia
2021-12-11  9:19                                 ` Qu Wenruo
     [not found]                                 ` <114dbcbb-c0ad-6ffd-f9ff-7aff031d03b5@rx2.rx-server.de>
2021-12-11  9:34                                   ` Mia

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.