failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Joseph Spagnol]

Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.

# mount -t btrfs /dev/sda4 /mnt/
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.

# btrfs-find-root /dev/sda4
Couldn't read tree root
Superblock thinks the generation is 432440
Superblock thinks the level is 1
Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
.......

# btrfsck -b -p /dev/sda4
Opening filesystem to check...
checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0
ERROR: failed to read block groups: Input/output error
ERROR: cannot open file system

Here are some more details;
# uname -a
Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
# btrfs --version
btrfs-progs v5.16.2
# btrfs fi show
Label: 'OLDDATA'  uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
        Total devices 1 FS bytes used 34.20GiB
        devid    1 size 50.00GiB used 37.52GiB path /dev/sda3
Label: 'OPENSUSE'  uuid: c3632d30-a117-43ef-8993-88f1933f6676
        Total devices 1 FS bytes used 24.60GiB
        devid    1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
Label: 'DATA'  uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
        Total devices 1 FS bytes used 118.67GiB
        devid    1 size 200.00GiB used 122.02GiB path /dev/sda4
# btrfs fi df /dev/sda4
ERROR: not a directory: /dev/sda4
# btrfs fi df /data/
ERROR: not a btrfs filesystem: /data/
## dmesg.log ##
[65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
[65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
[65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
[65500.890771] BTRFS info (device sda4): disabling disk space caching
[65500.890773] BTRFS info (device sda4): force clearing of disk cache
[65500.890775] BTRFS info (device sda4): has skinny extents
[65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
[65500.893593] BTRFS warning (device sda4): failed to read tree root
[65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
[65500.893856] BTRFS warning (device sda4): failed to read tree root
[65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
[65500.908111] BTRFS error (device sda4): failed to read block groups: -5
[65500.963167] BTRFS error (device sda4): open_ctree failed

P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed

Is there any way I could rebuild the tree?

Thanks in advance
Joseph

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Qu Wenruo]

On 2022/3/22 20:53, Joseph Spagnol wrote:
> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>
> # mount -t btrfs /dev/sda4 /mnt/
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
> # btrfs-find-root /dev/sda4
> Couldn't read tree root
> Superblock thinks the generation is 432440
> Superblock thinks the level is 1
> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> .......
>
> # btrfsck -b -p /dev/sda4
> Opening filesystem to check...
> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0

Some range is completely wiped out.
And that wiped out range is in extent tree.


There are several two theories for it:

- Some discard related bug
   It can be the firmware of disk, or btrfs itself.
   Some range got wiped out even we're still needing it.

- Some missing writes
   The write should reach disk but didn't.
   This means the barrier is not working.
   In that case, disk firmware may be the problem.


> ERROR: failed to read block groups: Input/output error
> ERROR: cannot open file system
>
> Here are some more details;
> # uname -a
> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
> # btrfs --version
> btrfs-progs v5.16.2
> # btrfs fi show
> Label: 'OLDDATA'  uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
>          Total devices 1 FS bytes used 34.20GiB
>          devid    1 size 50.00GiB used 37.52GiB path /dev/sda3
> Label: 'OPENSUSE'  uuid: c3632d30-a117-43ef-8993-88f1933f6676
>          Total devices 1 FS bytes used 24.60GiB
>          devid    1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
> Label: 'DATA'  uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
>          Total devices 1 FS bytes used 118.67GiB
>          devid    1 size 200.00GiB used 122.02GiB path /dev/sda4
> # btrfs fi df /dev/sda4
> ERROR: not a directory: /dev/sda4
> # btrfs fi df /data/
> ERROR: not a btrfs filesystem: /data/
> ## dmesg.log ##
> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
> [65500.890771] BTRFS info (device sda4): disabling disk space caching
> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
> [65500.890775] BTRFS info (device sda4): has skinny extents
> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [65500.893593] BTRFS warning (device sda4): failed to read tree root
> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [65500.893856] BTRFS warning (device sda4): failed to read tree root
> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
> [65500.963167] BTRFS error (device sda4): open_ctree failed
>
> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed

There are already at least two tree blocks got wiped.

I won't be surprised if there are more.

For now, only data salvage can be even attempted.

Using newer enough kernel (like from openSUSE tumbleweed), then mount
with -o rescue=all,ro to see if it can be mounted.

That's more or less the same as btrfs-restore, but more convenient to
copy things out.

Thanks,
Qu
>
> Is there any way I could rebuild the tree?
>
> Thanks in advance
> Joseph
>
>

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Joseph Spagnol]

Hello, thanks for the quick response.
unfortunately the ro mount from a more recent kernel does not work as well
 
# uname -r
5.16.11-1-default
# mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.

Am not sure this can help but this btrfs partition become like this after a sudden system freeze.

Is there a possibility to check an fix the partition size?
I believe it could be an issue with the actual size of the partition/partitions.

Sent: Tuesday, March 22, 2022 at 2:05 PM
From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
To: "Joseph Spagnol" <joseph.spagnol@programmer.net>, linux-btrfs@vger.kernel.org
Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

On 2022/3/22 20:53, Joseph Spagnol wrote:
> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>
> # mount -t btrfs /dev/sda4 /mnt/
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
> # btrfs-find-root /dev/sda4
> Couldn't read tree root
> Superblock thinks the generation is 432440
> Superblock thinks the level is 1
> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
> .......
>
> # btrfsck -b -p /dev/sda4
> Opening filesystem to check...
> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0

Some range is completely wiped out.
And that wiped out range is in extent tree.


There are several two theories for it:

- Some discard related bug
It can be the firmware of disk, or btrfs itself.
Some range got wiped out even we're still needing it.

- Some missing writes
The write should reach disk but didn't.
This means the barrier is not working.
In that case, disk firmware may be the problem.


> ERROR: failed to read block groups: Input/output error
> ERROR: cannot open file system
>
> Here are some more details;
> # uname -a
> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
> # btrfs --version
> btrfs-progs v5.16.2
> # btrfs fi show
> Label: 'OLDDATA' uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
> Total devices 1 FS bytes used 34.20GiB
> devid 1 size 50.00GiB used 37.52GiB path /dev/sda3
> Label: 'OPENSUSE' uuid: c3632d30-a117-43ef-8993-88f1933f6676
> Total devices 1 FS bytes used 24.60GiB
> devid 1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
> Label: 'DATA' uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
> Total devices 1 FS bytes used 118.67GiB
> devid 1 size 200.00GiB used 122.02GiB path /dev/sda4
> # btrfs fi df /dev/sda4
> ERROR: not a directory: /dev/sda4
> # btrfs fi df /data/
> ERROR: not a btrfs filesystem: /data/
> ## dmesg.log ##
> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
> [65500.890771] BTRFS info (device sda4): disabling disk space caching
> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
> [65500.890775] BTRFS info (device sda4): has skinny extents
> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [65500.893593] BTRFS warning (device sda4): failed to read tree root
> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [65500.893856] BTRFS warning (device sda4): failed to read tree root
> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
> [65500.963167] BTRFS error (device sda4): open_ctree failed
>
> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed

There are already at least two tree blocks got wiped.

I won't be surprised if there are more.

For now, only data salvage can be even attempted.

Using newer enough kernel (like from openSUSE tumbleweed), then mount
with -o rescue=all,ro to see if it can be mounted.

That's more or less the same as btrfs-restore, but more convenient to
copy things out.

Thanks,
Qu
>
> Is there any way I could rebuild the tree?
>
> Thanks in advance
> Joseph
>
>

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Qu Wenruo]

On 2022/3/23 01:17, Joseph Spagnol wrote:
> Hello, thanks for the quick response.
> unfortunately the ro mount from a more recent kernel does not work as well
>
> # uname -r
> 5.16.11-1-default
> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.

Dmesg please.

But I guess there are more errors on the critical trees, thus it failed.

>
> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.

Without dmesg of that incident, pretty hard to say.

>
> Is there a possibility to check an fix the partition size?
> I believe it could be an issue with the actual size of the partition/partitions.

Not sure what you mean here.

Did you changed the partition size without using btrfs device resize?

Thanks,
Qu

>
> Sent: Tuesday, March 22, 2022 at 2:05 PM
> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>, linux-btrfs@vger.kernel.org
> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>
> On 2022/3/22 20:53, Joseph Spagnol wrote:
>> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>>
>> # mount -t btrfs /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>>
>> # btrfs-find-root /dev/sda4
>> Couldn't read tree root
>> Superblock thinks the generation is 432440
>> Superblock thinks the level is 1
>> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> .......
>>
>> # btrfsck -b -p /dev/sda4
>> Opening filesystem to check...
>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0
>
> Some range is completely wiped out.
> And that wiped out range is in extent tree.
>
>
> There are several two theories for it:
>
> - Some discard related bug
> It can be the firmware of disk, or btrfs itself.
> Some range got wiped out even we're still needing it.
>
> - Some missing writes
> The write should reach disk but didn't.
> This means the barrier is not working.
> In that case, disk firmware may be the problem.
>
>
>> ERROR: failed to read block groups: Input/output error
>> ERROR: cannot open file system
>>
>> Here are some more details;
>> # uname -a
>> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
>> # btrfs --version
>> btrfs-progs v5.16.2
>> # btrfs fi show
>> Label: 'OLDDATA' uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
>> Total devices 1 FS bytes used 34.20GiB
>> devid 1 size 50.00GiB used 37.52GiB path /dev/sda3
>> Label: 'OPENSUSE' uuid: c3632d30-a117-43ef-8993-88f1933f6676
>> Total devices 1 FS bytes used 24.60GiB
>> devid 1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
>> Label: 'DATA' uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
>> Total devices 1 FS bytes used 118.67GiB
>> devid 1 size 200.00GiB used 122.02GiB path /dev/sda4
>> # btrfs fi df /dev/sda4
>> ERROR: not a directory: /dev/sda4
>> # btrfs fi df /data/
>> ERROR: not a btrfs filesystem: /data/
>> ## dmesg.log ##
>> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
>> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
>> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
>> [65500.890771] BTRFS info (device sda4): disabling disk space caching
>> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
>> [65500.890775] BTRFS info (device sda4): has skinny extents
>> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>> [65500.893593] BTRFS warning (device sda4): failed to read tree root
>> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>> [65500.893856] BTRFS warning (device sda4): failed to read tree root
>> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
>> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
>> [65500.963167] BTRFS error (device sda4): open_ctree failed
>>
>> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed
>
> There are already at least two tree blocks got wiped.
>
> I won't be surprised if there are more.
>
> For now, only data salvage can be even attempted.
>
> Using newer enough kernel (like from openSUSE tumbleweed), then mount
> with -o rescue=all,ro to see if it can be mounted.
>
> That's more or less the same as btrfs-restore, but more convenient to
> copy things out.
>
> Thanks,
> Qu
>>
>> Is there any way I could rebuild the tree?
>>
>> Thanks in advance
>> Joseph
>>
>>

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Joseph Spagnol]

Hello again,

> # uname -r
> 5.16.11-1-default
> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.

> Dmesg please.
Here is the dmesg

[21560.215563] BTRFS info (device sda3): flagging fs with big metadata feature
[21560.215570] BTRFS info (device sda3): disk space caching is enabled
[21560.215572] BTRFS info (device sda3): has skinny extents
[21560.218654] BTRFS info (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3181, gen 0
[21560.229756] BTRFS info (device sda3): enabling ssd optimizations
[87063.535960] BTRFS info (device nvme0n1p4): qgroup scan completed (inconsistency flag cleared)
[161387.456900] BTRFS info (device sda4): flagging fs with big metadata feature
[161387.456905] BTRFS info (device sda4): disk space caching is enabled
[161387.456906] BTRFS info (device sda4): has skinny extents
[161387.458569] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
[161387.458584] BTRFS warning (device sda4): couldn't read tree root
[161387.458847] BTRFS error (device sda4): open_ctree failed
[161620.891324] BTRFS info (device sda4): flagging fs with big metadata feature
[161620.891336] BTRFS info (device sda4): enabling all of the rescue options
[161620.891338] BTRFS info (device sda4): ignoring data csums
[161620.891340] BTRFS info (device sda4): ignoring bad roots
[161620.891342] BTRFS info (device sda4): disabling log replay at mount time
[161620.891345] BTRFS info (device sda4): disk space caching is enabled
[161620.891347] BTRFS info (device sda4): has skinny extents
[161620.893575] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
[161620.893599] BTRFS warning (device sda4): couldn't read tree root
[161620.894212] BTRFS error (device sda4): open_ctree failed

> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.

> Without dmesg of that incident, pretty hard to say.
I'm not sure I am able to retrieve this, but I will try.

> Did you changed the partition size without using btrfs device resize?
On the following boot I had a disk check and it asked for a resize and I executed it but it was not on the failed partition which at that moment I had no clue it was failed.

Thanks
Joseph

Sent: Wednesday, March 23, 2022 at 12:39 AM
From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
To: "Joseph Spagnol" <joseph.spagnol@programmer.net>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

On 2022/3/23 01:17, Joseph Spagnol wrote:
> Hello, thanks for the quick response.
> unfortunately the ro mount from a more recent kernel does not work as well
>
> # uname -r
> 5.16.11-1-default
> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.

Dmesg please.

But I guess there are more errors on the critical trees, thus it failed.

>
> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.

Without dmesg of that incident, pretty hard to say.

>
> Is there a possibility to check an fix the partition size?
> I believe it could be an issue with the actual size of the partition/partitions.

Not sure what you mean here.

Did you changed the partition size without using btrfs device resize?

Thanks,
Qu

>
> Sent: Tuesday, March 22, 2022 at 2:05 PM
> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>, linux-btrfs@vger.kernel.org
> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>
> On 2022/3/22 20:53, Joseph Spagnol wrote:
>> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>>
>> # mount -t btrfs /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>>
>> # btrfs-find-root /dev/sda4
>> Couldn't read tree root
>> Superblock thinks the generation is 432440
>> Superblock thinks the level is 1
>> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>> .......
>>
>> # btrfsck -b -p /dev/sda4
>> Opening filesystem to check...
>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0
>
> Some range is completely wiped out.
> And that wiped out range is in extent tree.
>
>
> There are several two theories for it:
>
> - Some discard related bug
> It can be the firmware of disk, or btrfs itself.
> Some range got wiped out even we're still needing it.
>
> - Some missing writes
> The write should reach disk but didn't.
> This means the barrier is not working.
> In that case, disk firmware may be the problem.
>
>
>> ERROR: failed to read block groups: Input/output error
>> ERROR: cannot open file system
>>
>> Here are some more details;
>> # uname -a
>> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
>> # btrfs --version
>> btrfs-progs v5.16.2
>> # btrfs fi show
>> Label: 'OLDDATA' uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
>> Total devices 1 FS bytes used 34.20GiB
>> devid 1 size 50.00GiB used 37.52GiB path /dev/sda3
>> Label: 'OPENSUSE' uuid: c3632d30-a117-43ef-8993-88f1933f6676
>> Total devices 1 FS bytes used 24.60GiB
>> devid 1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
>> Label: 'DATA' uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
>> Total devices 1 FS bytes used 118.67GiB
>> devid 1 size 200.00GiB used 122.02GiB path /dev/sda4
>> # btrfs fi df /dev/sda4
>> ERROR: not a directory: /dev/sda4
>> # btrfs fi df /data/
>> ERROR: not a btrfs filesystem: /data/
>> ## dmesg.log ##
>> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
>> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
>> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
>> [65500.890771] BTRFS info (device sda4): disabling disk space caching
>> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
>> [65500.890775] BTRFS info (device sda4): has skinny extents
>> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>> [65500.893593] BTRFS warning (device sda4): failed to read tree root
>> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>> [65500.893856] BTRFS warning (device sda4): failed to read tree root
>> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
>> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
>> [65500.963167] BTRFS error (device sda4): open_ctree failed
>>
>> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed
>
> There are already at least two tree blocks got wiped.
>
> I won't be surprised if there are more.
>
> For now, only data salvage can be even attempted.
>
> Using newer enough kernel (like from openSUSE tumbleweed), then mount
> with -o rescue=all,ro to see if it can be mounted.
>
> That's more or less the same as btrfs-restore, but more convenient to
> copy things out.
>
> Thanks,
> Qu
>>
>> Is there any way I could rebuild the tree?
>>
>> Thanks in advance
>> Joseph
>>
>>

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Qu Wenruo]

On 2022/3/24 22:07, Joseph Spagnol wrote:
> Hello again,
>
>> # uname -r
>> 5.16.11-1-default
>> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
>> Dmesg please.
> Here is the dmesg
>
> [21560.215563] BTRFS info (device sda3): flagging fs with big metadata feature
> [21560.215570] BTRFS info (device sda3): disk space caching is enabled
> [21560.215572] BTRFS info (device sda3): has skinny extents
> [21560.218654] BTRFS info (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3181, gen 0
> [21560.229756] BTRFS info (device sda3): enabling ssd optimizations
> [87063.535960] BTRFS info (device nvme0n1p4): qgroup scan completed (inconsistency flag cleared)
> [161387.456900] BTRFS info (device sda4): flagging fs with big metadata feature
> [161387.456905] BTRFS info (device sda4): disk space caching is enabled
> [161387.456906] BTRFS info (device sda4): has skinny extents
> [161387.458569] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [161387.458584] BTRFS warning (device sda4): couldn't read tree root
> [161387.458847] BTRFS error (device sda4): open_ctree failed
> [161620.891324] BTRFS info (device sda4): flagging fs with big metadata feature
> [161620.891336] BTRFS info (device sda4): enabling all of the rescue options
> [161620.891338] BTRFS info (device sda4): ignoring data csums
> [161620.891340] BTRFS info (device sda4): ignoring bad roots
> [161620.891342] BTRFS info (device sda4): disabling log replay at mount time
> [161620.891345] BTRFS info (device sda4): disk space caching is enabled
> [161620.891347] BTRFS info (device sda4): has skinny extents
> [161620.893575] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [161620.893599] BTRFS warning (device sda4): couldn't read tree root

The critical root tree has part of its tree blocks wiped out, just like
other tree blocks.

Thus rescue=all can not help much.

> [161620.894212] BTRFS error (device sda4): open_ctree failed
>
>> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.
>
>> Without dmesg of that incident, pretty hard to say.
> I'm not sure I am able to retrieve this, but I will try.
>
>> Did you changed the partition size without using btrfs device resize?
> On the following boot I had a disk check and it asked for a resize and I executed it but it was not on the failed partition which at that moment I had no clue it was failed.

So far it looks like a lot of tree blocks are wiped out.

Any more detailed history on this?

Thanks,
Qu

>
> Thanks
> Joseph
>
> Sent: Wednesday, March 23, 2022 at 12:39 AM
> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>
> Cc: linux-btrfs@vger.kernel.org
> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>
> On 2022/3/23 01:17, Joseph Spagnol wrote:
>> Hello, thanks for the quick response.
>> unfortunately the ro mount from a more recent kernel does not work as well
>>
>> # uname -r
>> 5.16.11-1-default
>> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
> Dmesg please.
>
> But I guess there are more errors on the critical trees, thus it failed.
>
>>
>> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.
>
> Without dmesg of that incident, pretty hard to say.
>
>>
>> Is there a possibility to check an fix the partition size?
>> I believe it could be an issue with the actual size of the partition/partitions.
>
> Not sure what you mean here.
>
> Did you changed the partition size without using btrfs device resize?
>
> Thanks,
> Qu
>
>>
>> Sent: Tuesday, March 22, 2022 at 2:05 PM
>> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>, linux-btrfs@vger.kernel.org
>> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>>
>> On 2022/3/22 20:53, Joseph Spagnol wrote:
>>> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>>>
>>> # mount -t btrfs /dev/sda4 /mnt/
>>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>>>
>>> # btrfs-find-root /dev/sda4
>>> Couldn't read tree root
>>> Superblock thinks the generation is 432440
>>> Superblock thinks the level is 1
>>> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> .......
>>>
>>> # btrfsck -b -p /dev/sda4
>>> Opening filesystem to check...
>>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>>> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0
>>
>> Some range is completely wiped out.
>> And that wiped out range is in extent tree.
>>
>>
>> There are several two theories for it:
>>
>> - Some discard related bug
>> It can be the firmware of disk, or btrfs itself.
>> Some range got wiped out even we're still needing it.
>>
>> - Some missing writes
>> The write should reach disk but didn't.
>> This means the barrier is not working.
>> In that case, disk firmware may be the problem.
>>
>>
>>> ERROR: failed to read block groups: Input/output error
>>> ERROR: cannot open file system
>>>
>>> Here are some more details;
>>> # uname -a
>>> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
>>> # btrfs --version
>>> btrfs-progs v5.16.2
>>> # btrfs fi show
>>> Label: 'OLDDATA' uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
>>> Total devices 1 FS bytes used 34.20GiB
>>> devid 1 size 50.00GiB used 37.52GiB path /dev/sda3
>>> Label: 'OPENSUSE' uuid: c3632d30-a117-43ef-8993-88f1933f6676
>>> Total devices 1 FS bytes used 24.60GiB
>>> devid 1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
>>> Label: 'DATA' uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
>>> Total devices 1 FS bytes used 118.67GiB
>>> devid 1 size 200.00GiB used 122.02GiB path /dev/sda4
>>> # btrfs fi df /dev/sda4
>>> ERROR: not a directory: /dev/sda4
>>> # btrfs fi df /data/
>>> ERROR: not a btrfs filesystem: /data/
>>> ## dmesg.log ##
>>> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
>>> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
>>> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
>>> [65500.890771] BTRFS info (device sda4): disabling disk space caching
>>> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
>>> [65500.890775] BTRFS info (device sda4): has skinny extents
>>> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>>> [65500.893593] BTRFS warning (device sda4): failed to read tree root
>>> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>>> [65500.893856] BTRFS warning (device sda4): failed to read tree root
>>> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
>>> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
>>> [65500.963167] BTRFS error (device sda4): open_ctree failed
>>>
>>> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed
>>
>> There are already at least two tree blocks got wiped.
>>
>> I won't be surprised if there are more.
>>
>> For now, only data salvage can be even attempted.
>>
>> Using newer enough kernel (like from openSUSE tumbleweed), then mount
>> with -o rescue=all,ro to see if it can be mounted.
>>
>> That's more or less the same as btrfs-restore, but more convenient to
>> copy things out.
>>
>> Thanks,
>> Qu
>>>
>>> Is there any way I could rebuild the tree?
>>>
>>> Thanks in advance
>>> Joseph
>>>
>>>

Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

[Joseph Spagnol]

[-- Attachment #1: Type: text/plain, Size: 10376 bytes --]

Hello,

>>> Did you changed the partition size without using btrfs device resize?
>> On the following boot I had a disk check and it asked for a resize and I executed it but it was not on the failed partition which at that moment I had no clue it was failed.
>
> So far it looks like a lot of tree blocks are wiped out.
>
> Any more detailed history on this?

Am not sure the attached can help, but that is all I could get which could be related to all this issue.
 
Thanks
Joseph
 

Sent: Friday, March 25, 2022 at 1:37 AM
From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
To: "Joseph Spagnol" <joseph.spagnol@programmer.net>
Cc: linux-btrfs@vger.kernel.org
Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;

On 2022/3/24 22:07, Joseph Spagnol wrote:
> Hello again,
>
>> # uname -r
>> 5.16.11-1-default
>> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
>> Dmesg please.
> Here is the dmesg
>
> [21560.215563] BTRFS info (device sda3): flagging fs with big metadata feature
> [21560.215570] BTRFS info (device sda3): disk space caching is enabled
> [21560.215572] BTRFS info (device sda3): has skinny extents
> [21560.218654] BTRFS info (device sda3): bdev /dev/sda3 errs: wr 0, rd 0, flush 0, corrupt 3181, gen 0
> [21560.229756] BTRFS info (device sda3): enabling ssd optimizations
> [87063.535960] BTRFS info (device nvme0n1p4): qgroup scan completed (inconsistency flag cleared)
> [161387.456900] BTRFS info (device sda4): flagging fs with big metadata feature
> [161387.456905] BTRFS info (device sda4): disk space caching is enabled
> [161387.456906] BTRFS info (device sda4): has skinny extents
> [161387.458569] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [161387.458584] BTRFS warning (device sda4): couldn't read tree root
> [161387.458847] BTRFS error (device sda4): open_ctree failed
> [161620.891324] BTRFS info (device sda4): flagging fs with big metadata feature
> [161620.891336] BTRFS info (device sda4): enabling all of the rescue options
> [161620.891338] BTRFS info (device sda4): ignoring data csums
> [161620.891340] BTRFS info (device sda4): ignoring bad roots
> [161620.891342] BTRFS info (device sda4): disabling log replay at mount time
> [161620.891345] BTRFS info (device sda4): disk space caching is enabled
> [161620.891347] BTRFS info (device sda4): has skinny extents
> [161620.893575] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
> [161620.893599] BTRFS warning (device sda4): couldn't read tree root

The critical root tree has part of its tree blocks wiped out, just like
other tree blocks.

Thus rescue=all can not help much.

> [161620.894212] BTRFS error (device sda4): open_ctree failed
>
>> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.
>
>> Without dmesg of that incident, pretty hard to say.
> I'm not sure I am able to retrieve this, but I will try.
>
>> Did you changed the partition size without using btrfs device resize?
> On the following boot I had a disk check and it asked for a resize and I executed it but it was not on the failed partition which at that moment I had no clue it was failed.

So far it looks like a lot of tree blocks are wiped out.

Any more detailed history on this?

Thanks,
Qu

>
> Thanks
> Joseph
>
> Sent: Wednesday, March 23, 2022 at 12:39 AM
> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>
> Cc: linux-btrfs@vger.kernel.org
> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>
> On 2022/3/23 01:17, Joseph Spagnol wrote:
>> Hello, thanks for the quick response.
>> unfortunately the ro mount from a more recent kernel does not work as well
>>
>> # uname -r
>> 5.16.11-1-default
>> # mount -t btrfs -o rescue=all,ro /dev/sda4 /mnt/
>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>
> Dmesg please.
>
> But I guess there are more errors on the critical trees, thus it failed.
>
>>
>> Am not sure this can help but this btrfs partition become like this after a sudden system freeze.
>
> Without dmesg of that incident, pretty hard to say.
>
>>
>> Is there a possibility to check an fix the partition size?
>> I believe it could be an issue with the actual size of the partition/partitions.
>
> Not sure what you mean here.
>
> Did you changed the partition size without using btrfs device resize?
>
> Thanks,
> Qu
>
>>
>> Sent: Tuesday, March 22, 2022 at 2:05 PM
>> From: "Qu Wenruo" <quwenruo.btrfs@gmx.com>
>> To: "Joseph Spagnol" <joseph.spagnol@programmer.net>, linux-btrfs@vger.kernel.org
>> Subject: Re: failed to read block groups: Input/output error; bad tree block - bytenr mismatch;
>>
>> On 2022/3/22 20:53, Joseph Spagnol wrote:
>>> Hello, recently one of my btrfs partitions has become unavailable and am not able to mount it.
>>>
>>> # mount -t btrfs /dev/sda4 /mnt/
>>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sda4, missing codepage or helper program, or other error.
>>>
>>> # btrfs-find-root /dev/sda4
>>> Couldn't read tree root
>>> Superblock thinks the generation is 432440
>>> Superblock thinks the level is 1
>>> Well block 23235313664(gen: 432440 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23231447040(gen: 432439 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23229202432(gen: 432438 level: 0) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23192911872(gen: 432431 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23177084928(gen: 432430 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23149035520(gen: 432429 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23124443136(gen: 432427 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23113547776(gen: 432426 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23080730624(gen: 432425 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23048241152(gen: 432424 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> Well block 23013031936(gen: 432422 level: 1) seems good, but generation/level doesn't match, want gen: 432440 level: 1
>>> .......
>>>
>>> # btrfsck -b -p /dev/sda4
>>> Opening filesystem to check...
>>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>>> checksum verify failed on 23234035712 wanted 0x00000000 found 0x0810faf8
>>> bad tree block 23234035712, bytenr mismatch, want=23234035712, have=0
>>
>> Some range is completely wiped out.
>> And that wiped out range is in extent tree.
>>
>>
>> There are several two theories for it:
>>
>> - Some discard related bug
>> It can be the firmware of disk, or btrfs itself.
>> Some range got wiped out even we're still needing it.
>>
>> - Some missing writes
>> The write should reach disk but didn't.
>> This means the barrier is not working.
>> In that case, disk firmware may be the problem.
>>
>>
>>> ERROR: failed to read block groups: Input/output error
>>> ERROR: cannot open file system
>>>
>>> Here are some more details;
>>> # uname -a
>>> Linux msi-b17-manjaro 5.4.184-1-MANJARO #1 SMP PREEMPT Fri Mar 11 13:59:07 UTC 2022 x86_64 GNU/Linux
>>> # btrfs --version
>>> btrfs-progs v5.16.2
>>> # btrfs fi show
>>> Label: 'OLDDATA' uuid: 9bc104b4-c889-477f-aae1-4d865cdc0372
>>> Total devices 1 FS bytes used 34.20GiB
>>> devid 1 size 50.00GiB used 37.52GiB path /dev/sda3
>>> Label: 'OPENSUSE' uuid: c3632d30-a117-43ef-8993-88f1933f6676
>>> Total devices 1 FS bytes used 24.60GiB
>>> devid 1 size 150.00GiB used 31.05GiB path /dev/nvme0n1p4
>>> Label: 'DATA' uuid: 4ce61b29-8c8d-4c04-b715-96f0dda809a4
>>> Total devices 1 FS bytes used 118.67GiB
>>> devid 1 size 200.00GiB used 122.02GiB path /dev/sda4
>>> # btrfs fi df /dev/sda4
>>> ERROR: not a directory: /dev/sda4
>>> # btrfs fi df /data/
>>> ERROR: not a btrfs filesystem: /data/
>>> ## dmesg.log ##
>>> [65500.890756] BTRFS info (device sda4): flagging fs with big metadata feature
>>> [65500.890766] BTRFS warning (device sda4): 'recovery' is deprecated, use 'usebackuproot' instead
>>> [65500.890768] BTRFS info (device sda4): trying to use backup root at mount time
>>> [65500.890771] BTRFS info (device sda4): disabling disk space caching
>>> [65500.890773] BTRFS info (device sda4): force clearing of disk cache
>>> [65500.890775] BTRFS info (device sda4): has skinny extents
>>> [65500.893556] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>>> [65500.893593] BTRFS warning (device sda4): failed to read tree root
>>> [65500.893852] BTRFS error (device sda4): bad tree block start, want 23235280896 have 0
>>> [65500.893856] BTRFS warning (device sda4): failed to read tree root
>>> [65500.908097] BTRFS error (device sda4): bad tree block start, want 23234035712 have 0
>>> [65500.908111] BTRFS error (device sda4): failed to read block groups: -5
>>> [65500.963167] BTRFS error (device sda4): open_ctree failed
>>>
>>> P.S. I must say that I get the same results when I try to mount the partition from another linux system OpenSuse tumbleweed
>>
>> There are already at least two tree blocks got wiped.
>>
>> I won't be surprised if there are more.
>>
>> For now, only data salvage can be even attempted.
>>
>> Using newer enough kernel (like from openSUSE tumbleweed), then mount
>> with -o rescue=all,ro to see if it can be mounted.
>>
>> That's more or less the same as btrfs-restore, but more convenient to
>> copy things out.
>>
>> Thanks,
>> Qu
>>>
>>> Is there any way I could rebuild the tree?
>>>
>>> Thanks in advance
>>> Joseph
>>>
>>>

[-- Attachment #2: possible-required-info.xz --]
[-- Type: application/x-xz, Size: 36988 bytes --]