* Tree-Checker-Error on vServer-Directory
@ 2020-05-20 15:54 joerg.ebertz
2020-05-20 22:50 ` Qu Wenruo
0 siblings, 1 reply; 3+ messages in thread
From: joerg.ebertz @ 2020-05-20 15:54 UTC (permalink / raw)
To: linux-btrfs
Hallo,
https://btrfs.wiki.kernel.org/index.php/Tree-checker suggested that I
ask here for help.
When I run a Linux-Kernel 5.7.0-rc2, compiled with pretty much
default configuration, and try to run
ls /vservers
I get the error message:
ls: cannot access '/vservers': Input/output error
From dmesg I get:
[912737.466820] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[912737.466825] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[912737.467190] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[912737.467194] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
With kernel 5.6.9 I get:
[ 520.285953] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.285958] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.286234] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.286238] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.771733] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.771746] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.772075] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.772079] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.778309] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.778314] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.778656] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.778661] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.783287] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.783292] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.783655] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.783659] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.788400] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.788404] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.788745] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.788749] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
[ 520.791619] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
[ 520.791624] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
With the default kernel of Debian “Stable” = “10” = “Buster”, which is
presently 4.19.0-9 this works flawlessly. When I booted my computer
with grml 2018.12 ( https://grml.org/ ) with a 4.19.8 kernel, I didn_t
have any problems either, “btrfs check” and “btrfs scrub” didn't find
any errors.
Has there been a change to tree-checker, that has a problem with
something vServer ( http://linux-vserver.org ) does? I don't use
vServer any more, since Debian stopped supporting it, I only keep those
files for reference.
I don't think that matters, but I have three SSDs in a RAID0 Array,
on top of which there is a LUKS container which contains a LVM2
which in turn has the BTRFS file system in one of it's LVs.
Is there any other information You need?
Greeting,
Joerg
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Tree-Checker-Error on vServer-Directory
2020-05-20 15:54 Tree-Checker-Error on vServer-Directory joerg.ebertz
@ 2020-05-20 22:50 ` Qu Wenruo
2020-05-21 11:27 ` Jörg Ebertz
0 siblings, 1 reply; 3+ messages in thread
From: Qu Wenruo @ 2020-05-20 22:50 UTC (permalink / raw)
To: joerg.ebertz, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 5524 bytes --]
On 2020/5/20 下午11:54, joerg.ebertz@gmx.net wrote:
> Hallo,
>
> https://btrfs.wiki.kernel.org/index.php/Tree-checker suggested that I
> ask here for help.
>
> When I run a Linux-Kernel 5.7.0-rc2, compiled with pretty much
> default configuration, and try to run
>
> ls /vservers
>
> I get the error message:
>
> ls: cannot access '/vservers': Input/output error
>
> From dmesg I get:
>
> [912737.466820] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [912737.466825] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [912737.467190] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [912737.467194] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
The problem is exactly what it said, the flag 0x200000 is not something
registered in btrfs.
In btrfs, valid inode flags mask is 0x800008ff. the 0x20000 is
definitely not the case.
>
> With kernel 5.6.9 I get:
>
> [ 520.285953] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.285958] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.286234] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.286238] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.771733] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.771746] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.772075] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.772079] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.778309] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.778314] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.778656] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.778661] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.783287] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.783292] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.783655] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.783659] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.788400] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.788404] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.788745] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.788749] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
> [ 520.791619] BTRFS critical (device dm-2): corrupt leaf: root=5 block=350945280 slot=85 ino=55495, unknown flags detected: 0x2000000
> [ 520.791624] BTRFS error (device dm-2): block=350945280 read time tree block corruption detected
>
> With the default kernel of Debian “Stable” = “10” = “Buster”, which is
> presently 4.19.0-9 this works flawlessly. When I booted my computer
> with grml 2018.12 ( https://grml.org/ ) with a 4.19.8 kernel, I didn_t
> have any problems either, “btrfs check” and “btrfs scrub” didn't find
> any errors.
Btrfs check hasn't add such restrict check yet, and btrfs scrub doesn't
check the tree contents.
Furthermore, from the bit, it looks like a memory bitflip, and it is
definitely caused in older kernels which doesn't has such comprehensive
check.
If it's a memory bitflip, it's highly recommended to run a full memtest
to ensure your memory is OK.
Such hardware problem is really hard to detect, and would definitely
cause unexpected behavior.
>
> Has there been a change to tree-checker, that has a problem with
> something vServer ( http://linux-vserver.org ) does? I don't use
> vServer any more, since Debian stopped supporting it, I only keep those
> files for reference.
Yes, in v5.2 tree-checker added extra check on inode item, in commit
496245cac57e ("btrfs: tree-checker: Verify inode item").
>
> I don't think that matters, but I have three SSDs in a RAID0 Array,
> on top of which there is a LUKS container which contains a LVM2
> which in turn has the BTRFS file system in one of it's LVs.
>
> Is there any other information You need?
No extra info needed. We'll add btrfs check support for it soon.
To recover your data, it can be done by removing the offending inodes
with older kernel.
The inode can be located using the inode number provided in the dmesg.
Thanks,
Qu
>
> Greeting,
> Joerg
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Tree-Checker-Error on vServer-Directory
2020-05-20 22:50 ` Qu Wenruo
@ 2020-05-21 11:27 ` Jörg Ebertz
0 siblings, 0 replies; 3+ messages in thread
From: Jörg Ebertz @ 2020-05-21 11:27 UTC (permalink / raw)
To: linux-btrfs
On 2020-05-21, Qu Wenruo wrote:
> Btrfs check hasn't add such restrict check yet, and btrfs scrub doesn't
> check the tree contents.
>
> Furthermore, from the bit, it looks like a memory bitflip, and it is
> definitely caused in older kernels which doesn't has such comprehensive
> check.
Running rsync on the entire file system should trigger tree-checker, if
there are other damaged inodes, right?
> If it's a memory bitflip, it's highly recommended to run a full memtest
> to ensure your memory is OK.
I'll do that.
> To recover your data, it can be done by removing the offending inodes
> with older kernel.
>
> The inode can be located using the inode number provided in the dmesg.
sudo find / -inum 55495 -print
gave me
/vserver
To remove the inode, I run
sudo mkdir /vserver2
sudo cp -a --reflink /vserver /vserver2
sudo rm /vserver
sudo mv /vserver2 /vserver
correct?
Joerg
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-05-21 11:27 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-20 15:54 Tree-Checker-Error on vServer-Directory joerg.ebertz
2020-05-20 22:50 ` Qu Wenruo
2020-05-21 11:27 ` Jörg Ebertz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.