On 2020/2/5 下午6:18, Chiung-Ming Huang wrote:
> Hi everyone
> 
> It's a long story. I try to describe it shortly. My btrfs RAID1
> includes 5 HDDs, 10Tx2, 1Tx1, 2Tx1 and 3Tx1. They all based on bcache
> (1Tx1 SSD as cache) and luks. I tried to reorder it to ` Luks -->
> Bcache --> SSD --> HDD` with only one layer of luks on bcache. But I
> failed because of power-off accidentally. Please help me to fix it.
> Thanks.
> 
> 1. OS: Ubuntu 18.04
> 
> 2. $ uname -a
> Linux rescue 5.3.0-26-generic #28~18.04.1-Ubuntu SMP Wed Dec 18
> 16:40:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
...
> [Wed Feb  5 17:09:04 2020] BTRFS error (device bcache2): tree level
> mismatch detected, bytenr=19499133206528 level expected=0 has=2
> [Wed Feb  5 17:09:04 2020] BTRFS error (device bcache2): tree level
> mismatch detected, bytenr=19499133206528 level expected=0 has=2
> [Wed Feb  5 17:09:04 2020] BTRFS warning (device bcache2): iterating
> uuid_tree failed -117
> btrfs fi df /
> 
> ------------------ dmesg part 2/2 ------------------
> 
> [Wed Feb  5 17:09:36 2020] BTRFS error (device bcache2): tree block
> 14963956514816 owner 3 already locked by pid=3187, extent tree
> corruption detected

This shows the problem. Your extent tree is corrupted.

I don't believe the lower storage stack is involved.

Full histroy of the fs please (from mkfs to current stage)


...
> 
> 7. $ btrfs check -p /dev/bcache4
> Opening filesystem to check...
> Checking filesystem on /dev/bcache4
> UUID: 0b79cf54-c424-40ed-adca-bd66b38ad57a
> Error: could not find extent items for root 257(0:00:00 elapsed, 1199
> items checked)
> [1/7] checking root items                      (0:00:00 elapsed, 7748
> items checked)
> ERROR: failed to repair root items: No such file or directory

Have you tried btrfs check --repair then mount?
Is that mentioned dmesg the first time you hit, not something after
btrfs check --repair?

And `btrfs check` without --repair please, that's the most important
info to evaluate how to fix it (if possible).

Thanks,
Qu

> 
> 8. $ btrfs scrub start -B -R /mnt
> The status is aborted because the file system was forcely re-mounted readonly.
> 
> 9. $ lsblk -o NAME,SIZE,TYPE,FSTYPE
> sda                     931.5G disk  bcache
> └─bcache0               931.5G disk  crypto_LUKS
>   └─disk-1t                931.5G crypt btrfs
> sdb                     232.9G disk
> └─sdb6                     10G part  crypto_LUKS
>   └─rescue                 10G crypt btrfs
> sdc                       2.7T disk  crypto_LUKS
> └─disk-3t             2.7T crypt bcache
>   └─bcache3               2.7T disk  btrfs
> sdd                       9.1T disk  crypto_LUKS
> └─disk-10t           9.1T crypt bcache
>   └─bcache2               9.1T disk  btrfs
> sde                       1.8T disk  bcache
> └─bcache1                 1.8T disk  crypto_LUKS
>   └─disk-2t          1.8T crypt btrfs
> sdf                       9.1T disk  crypto_LUKS
> └─disk-10t          9.1T crypt bcache
>   └─bcache4               9.1T disk  btrfs
> nvme0n1                 953.9G disk
> └─nvme0n1p1               636G part  crypto_LUKS
>   └─cache                 636G crypt bcache
>     ├─bcache0           931.5G disk  crypto_LUKS
>     │ └─disk-1t           931.5G crypt btrfs
>     ├─bcache1             1.8T disk  crypto_LUKS
>     │ └─disk-2t          1.8T crypt btrfs
>     ├─bcache2             9.1T disk  btrfs
>     ├─bcache3             2.7T disk  btrfs
>     └─bcache4             9.1T disk  btrfs
> 
> 
> Regards,
> Chiung-Ming Huang
>