[BUG] BTRFS critical corrupt leaf - bisected to 496245cac57e

[Alexander Wetzel <alexander.wetzel@web.de>]

Hello,

After updating one of my VMs from 5.1.16 to 5.2 btrfs is acting up strange:
The system is using btrfs as root (also for /boot) and has compression 
enabled. (It's a gentoo virtual machine and not using an initrd.) 
Rebooting the system into 5.2 is able to bring up openssh, but not other 
services (like e.g. postfix).

Redirecting dmesg to a file also failed. The file was created but empty, 
even when checked immediately. Rebooting the system into the old kernel 
fully restores functionality and a btrfs srub shows no errors..

When running a bad kernel the system is partially ro, but since I'm 
using selinux in strict mode it could also by caused by selinux unable 
to read some data.

Here how a reboot via ssh looks for a broken kernel:
xar /home/alex # shutdown -r now
shutdown: warning: cannot open /var/run/shutdown.pid

But deleting the "bad" kernel and running "grub-mkconfig -o 
/boot/grub/grub.cfg" works as it should and the system is using the 
previous kernel... So it still can write some things...

I've bisected the problem to 496245cac57e (btrfs: tree-checker: Verify 
inode item)

filtering for btrfs and removing duplicate lines just shows three uniq 
error messages:
  BTRFS critical (device vda3): corrupt leaf: root=300 block=8645398528 
slot=4 ino=259223, invalid inode generation: has 139737289170944 expect 
[0, 1425224]
  BTRFS critical (device vda3): corrupt leaf: root=300 block=8645398528 
slot=4 ino=259223, invalid inode generation: has 139737289170944 expect 
[0, 1425225]
  BTRFS critical (device vda3): corrupt leaf: root=300 block=8645398528 
slot=4 ino=259223, invalid inode generation: has 139737289170944 expect 
[0, 1425227]
  BTRFS error (device vda3): block=8645398528 read time tree block 
corruption detected

All there errors are only there with commit 496245cac57e, booting a 
kernel without the patch after that just works normally again.

I tried to reproduce the issue transferring the fs with btrfs 
send/receive to another system. I was able to mount the migrated Fs with 
a 5.2 kernel and also btrfs scub was ok...

I tried to revert the commit but a simple revert is not working. So I've 
just verified that a kernel build on 496245cac57e shows the symptoms 
while using the commit before that is fine.

I've not tried more, so I still can reproduce the issue when needed and 
gather more data. (The system is now back running 5.1.16)

Now I guess I could have some corruption only detected/triggered with 
the patch and btrfs check may fix it... shall I try that next?

Here the dmesg from the affected system (the first few lines still in 
the log buffer when I checked):
[    8.963796] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    8.963799] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    8.967487] audit: type=1400 audit(1563023702.540:19): avc:  denied 
{ write } for  pid=2154 comm="cp" name="localtime" dev="vda3" 
ino=1061039 scontext=system_u:system_r:initrc_t 
tcontext=system_u:object_r:locale_t tclass=file permissive=0
[    9.023608] audit: type=1400 audit(1563023702.590:20): avc:  denied 
{ mounton } for  pid=2194 comm="mount" path="/chroot/dns/run/named" 
dev="vda3" ino=1061002 scontext=system_u:system_r:mount_t 
tcontext=system_u:object_r:named_var_run_t tclass=dir permissive=0
[    9.038235] audit: type=1400 audit(1563023702.610:21): avc:  denied 
{ getattr } for  pid=2199 comm="start-stop-daem" path="pid:[4026531836]" 
dev="nsfs" ino=4026531836 scontext=system_u:system_r:initrc_t 
tcontext=system_u:object_r:nsfs_t tclass=file permissive=0
[    9.100897] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.100900] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    9.137974] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.137976] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    9.138095] audit: type=1400 audit(1563023702.710:22): avc:  denied 
{ getattr } for  pid=2237 comm="start-stop-daem" path="pid:[4026531836]" 
dev="nsfs" ino=4026531836 scontext=system_u:system_r:initrc_t 
tcontext=system_u:object_r:nsfs_t tclass=file permissive=0
[    9.138477] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.138480] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    9.161866] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.161868] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    9.170228] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.170230] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected
[    9.214491] BTRFS critical (device vda3): corrupt leaf: root=300 
block=8645398528 slot=4 ino=259223, invalid inode generation: has 
139737289170944 expect [0, 1425224]
[    9.214494] BTRFS error (device vda3): block=8645398528 read time 
tree block corruption detected


Alexander