Help Dealing with BTRFS errors on a root partition in NVMe M2 PCIe

Help Dealing with BTRFS errors on a root partition in NVMe M2 PCIe

[Fernando Peral]

Hi.

I'm having an error on the root partition of a opensuse leap 15.3 system.

I have been asking for help in 
https://forums.opensuse.org/showthread.php/557380-Dealing-with-BTRFS-errors-on-a-root-partition-in-NVMe-M2-PCIe

The problem seems to have been caused by a faulty ram module wich has 
been already replaced, but the error of the fs is still there.

It has been suggested that it has been a bitflip and to ask here if  a 
btrfs check and repair should be done.



#btrfs
check --readonly --force /dev/nvme0n1p1
[1/7] checking root items
[2/7] checking extents
data backref 227831808 root 263 owner 7983 offset 0 num_refs 0 not found 
in extent tree
incorrect local backref count on 227831808 root 263 owner 7983 offset 0 
found 1 wanted 0 back 0x5559e0ab7020
incorrect local backref count on 227831808 root 263 owner 
140737488363311offset 0 found 0 wanted 1 back 0x5559dde718d0
backref disk bytenr does not match extent record, bytenr=227831808, ref 
bytenr=0
backpointer mismatch on [227831808 4096]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7]checking free space cache
[4/7]checking fs roots
[5/7]checking only csums items (without verifying data)
[6/7]checking root refs
[7/7]checking quota groups
Qgroup are marked as inconsistent.
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p1
UUID: 5b000355-3a1a-49f5-8005-f10668008aa7
Rescan hasn't been initialized, a difference in qgroup accounting is 
expected
found 51878920192 bytes used, error(s) found
total csum bytes: 48135312
total tree bytes: 991313920
total fs tree bytes: 885358592
total extent tree bytes: 48414720
btree space waste bytes: 151592274
file data blocks allocated: 239972728832
referenced 85539778560




pruebas:~# uname -a
Linux pruebas 5.3.18-59.13-default #1 SMP Tue Jul 6
07:33:56 UTC 2021 (23ab94f) x86_64 x86_64 x86_64 GNU/Linux


pruebas:~#btrfs --version
btrfs-progs v4.19.1  ç


pruebas:~# btrfs fi show
Label: none  uuid: 5b000355-3a1a-49f5-8005-f10668008aa7
        Totaldevices 1 FS bytes used 48.42GiB
        devid   1 size 931.51GiB used 51.05GiB path /dev/nvme0n1p1


pruebas:~#btrfs fi df /
Data, single: total=49.01GiB, used=47.48GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=2.01GiB, used=962.69MiB
GlobalReserve, single: total=101.06MiB, used=0.00B

Re: Help Dealing with BTRFS errors on a root partition in NVMe M2 PCIe

[Qu Wenruo]

On 2021/7/24 下午5:47, Qu Wenruo wrote:
>
>
> On 2021/7/24 下午5:25, Fernando Peral wrote:
>> Hi.
>>
>> I'm having an error on the root partition of a opensuse leap 15.3 system.
>>
>> I have been asking for help in the opensuse forums
>>
>> The problem seems to have been caused by a faulty ram module wich has
>> been already replaced, but the error of the fs is still there.
>>
>> It has been suggested that it has been a bitflip and to ask here if a
>> btrfs check and repair should be done. >
>>
>>
>> #btrfs
>> check --readonly --force /dev/nvme0n1p1
>> [1/7] checking root items
>> [2/7] checking extents
>> data backref 227831808 root 263 owner 7983 offset 0 num_refs 0 not found
>> in extent tree
>> incorrect local backref count on 227831808 root 263 owner 7983 offset 0
>> found 1 wanted 0 back 0x5559e0ab7020
>> incorrect local backref count on 227831808 root 263 owner
>> 140737488363311offset 0 found 0 wanted 1 back 0x5559dde718d0
>
> The owner number 140737488363311 (0x800000001f2f) really looks like a
> bitflip in the high bit.
>
> Thus the faulty memory theory looks pretty solid.
>
> So far just two bad EXTENT_ITEM, I think btrfs check --repair is able to
> fix it.
>
> But before calling "btrfs check --repair", please backup all your
> important data, as there is still a low chance that "btrfs check
> --repair" may make things worse.
>
> Thanks,
> Qu
>
>> backref disk bytenr does not match extent record, bytenr=227831808, ref
>> bytenr=0
>> backpointer mismatch on [227831808 4096]
>> ERROR: errors found in extent allocation tree or chunk allocation
>> [3/7]checking free space cache
>> [4/7]checking fs roots
>> [5/7]checking only csums items (without verifying data)
>> [6/7]checking root refs
>> [7/7]checking quota groups
>> Qgroup are marked as inconsistent.
>> Opening filesystem to check...
>> Checking filesystem on /dev/nvme0n1p1
>> UUID: 5b000355-3a1a-49f5-8005-f10668008aa7
>> Rescan hasn't been initialized, a difference in qgroup accounting is
>> expected
>> found 51878920192 bytes used, error(s) found
>> total csum bytes: 48135312
>> total tree bytes: 991313920
>> total fs tree bytes: 885358592
>> total extent tree bytes: 48414720
>> btree space waste bytes: 151592274
>> file data blocks allocated: 239972728832
>> referenced 85539778560
>>
>>
>> pruebas:~# uname -a
>> Linux pruebas 5.3.18-59.13-default #1 SMP Tue Jul 6
>> 07:33:56 UTC 2021 (23ab94f) x86_64 x86_64 x86_64 GNU/Linux
>>
>>
>> pruebas:~#btrfs --version
>> btrfs-progs v4.19.1  ç

My bad, your btrfs-progs looks a little old.

It would be safer to use newer btrfs.

Since it's your root fs, you may want to use a liveUSB with newer
btrfs-progs to fix the fs.

Thanks,
Qu
>>
>> pruebas:~# btrfs fi show
>> Label: none  uuid: 5b000355-3a1a-49f5-8005-f10668008aa7
>> Totaldevices 1 FS bytes used 48.42GiB
>> devid  1 size 931.51GiB used 51.05GiB path /dev/nvme0n1p1
>>
>>
>> pruebas:~#btrfs fi df /
>> Data, single: total=49.01GiB, used=47.48GiB
>> System, single: total=32.00MiB, used=16.00KiB
>> Metadata, single: total=2.01GiB, used=962.69MiB
>> GlobalReserve, single: total=101.06MiB, used=0.00B
>>
>>

Re: Help Dealing with BTRFS errors on a root partition in NVMe M2 PCIe

[Qu Wenruo]

On 2021/7/24 下午5:25, Fernando Peral wrote:
> Hi.
>
> I'm having an error on the root partition of a opensuse leap 15.3 system.
>
> I have been asking for help in the opensuse forums
>
> The problem seems to have been caused by a faulty ram module wich has
> been already replaced, but the error of the fs is still there.
>
> It has been suggested that it has been a bitflip and to ask here if a
> btrfs check and repair should be done. >
>
>
> #btrfs
> check --readonly --force /dev/nvme0n1p1
> [1/7] checking root items
> [2/7] checking extents
> data backref 227831808 root 263 owner 7983 offset 0 num_refs 0 not found
> in extent tree
> incorrect local backref count on 227831808 root 263 owner 7983 offset 0
> found 1 wanted 0 back 0x5559e0ab7020
> incorrect local backref count on 227831808 root 263 owner
> 140737488363311offset 0 found 0 wanted 1 back 0x5559dde718d0

The owner number 140737488363311 (0x800000001f2f) really looks like a
bitflip in the high bit.

Thus the faulty memory theory looks pretty solid.

So far just two bad EXTENT_ITEM, I think btrfs check --repair is able to
fix it.

But before calling "btrfs check --repair", please backup all your
important data, as there is still a low chance that "btrfs check
--repair" may make things worse.

Thanks,
Qu

> backref disk bytenr does not match extent record, bytenr=227831808, ref
> bytenr=0
> backpointer mismatch on [227831808 4096]
> ERROR: errors found in extent allocation tree or chunk allocation
> [3/7]checking free space cache
> [4/7]checking fs roots
> [5/7]checking only csums items (without verifying data)
> [6/7]checking root refs
> [7/7]checking quota groups
> Qgroup are marked as inconsistent.
> Opening filesystem to check...
> Checking filesystem on /dev/nvme0n1p1
> UUID: 5b000355-3a1a-49f5-8005-f10668008aa7
> Rescan hasn't been initialized, a difference in qgroup accounting is
> expected
> found 51878920192 bytes used, error(s) found
> total csum bytes: 48135312
> total tree bytes: 991313920
> total fs tree bytes: 885358592
> total extent tree bytes: 48414720
> btree space waste bytes: 151592274
> file data blocks allocated: 239972728832
> referenced 85539778560
>
>
> pruebas:~# uname -a
> Linux pruebas 5.3.18-59.13-default #1 SMP Tue Jul 6
> 07:33:56 UTC 2021 (23ab94f) x86_64 x86_64 x86_64 GNU/Linux
>
>
> pruebas:~#btrfs --version
> btrfs-progs v4.19.1  ç
>
> pruebas:~# btrfs fi show
> Label: none  uuid: 5b000355-3a1a-49f5-8005-f10668008aa7
> Totaldevices 1 FS bytes used 48.42GiB
> devid  1 size 931.51GiB used 51.05GiB path /dev/nvme0n1p1
>
>
> pruebas:~#btrfs fi df /
> Data, single: total=49.01GiB, used=47.48GiB
> System, single: total=32.00MiB, used=16.00KiB
> Metadata, single: total=2.01GiB, used=962.69MiB
> GlobalReserve, single: total=101.06MiB, used=0.00B
>
>

Help Dealing with BTRFS errors on a root partition in NVMe M2 PCIe

[Fernando Peral]

Hi.

I'm having an error on the root partition of a opensuse leap 15.3 system.

I have been asking for help in the opensuse forums

The problem seems to have been caused by a faulty ram module wich has 
been already replaced, but the error of the fs is still there.

It has been suggested that it has been a bitflip and to ask here if a 
btrfs check and repair should be done.



#btrfs
check --readonly --force /dev/nvme0n1p1
[1/7] checking root items
[2/7] checking extents
data backref 227831808 root 263 owner 7983 offset 0 num_refs 0 not found
in extent tree
incorrect local backref count on 227831808 root 263 owner 7983 offset 0
found 1 wanted 0 back 0x5559e0ab7020
incorrect local backref count on 227831808 root 263 owner
140737488363311offset 0 found 0 wanted 1 back 0x5559dde718d0
backref disk bytenr does not match extent record, bytenr=227831808, ref
bytenr=0
backpointer mismatch on [227831808 4096]
ERROR: errors found in extent allocation tree or chunk allocation
[3/7]checking free space cache
[4/7]checking fs roots
[5/7]checking only csums items (without verifying data)
[6/7]checking root refs
[7/7]checking quota groups
Qgroup are marked as inconsistent.
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p1
UUID: 5b000355-3a1a-49f5-8005-f10668008aa7
Rescan hasn't been initialized, a difference in qgroup accounting is
expected
found 51878920192 bytes used, error(s) found
total csum bytes: 48135312
total tree bytes: 991313920
total fs tree bytes: 885358592
total extent tree bytes: 48414720
btree space waste bytes: 151592274
file data blocks allocated: 239972728832
referenced 85539778560


pruebas:~# uname -a
Linux pruebas 5.3.18-59.13-default #1 SMP Tue Jul 6
07:33:56 UTC 2021 (23ab94f) x86_64 x86_64 x86_64 GNU/Linux


pruebas:~#btrfs --version
btrfs-progs v4.19.1  ç

pruebas:~# btrfs fi show
Label: none  uuid: 5b000355-3a1a-49f5-8005-f10668008aa7
Totaldevices 1 FS bytes used 48.42GiB
devid  1 size 931.51GiB used 51.05GiB path /dev/nvme0n1p1


pruebas:~#btrfs fi df /
Data, single: total=49.01GiB, used=47.48GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=2.01GiB, used=962.69MiB
GlobalReserve, single: total=101.06MiB, used=0.00B


-- 
Fernando Peral Pérez
I.E.S. María Moliner.
Departamento de Electrónica
Tfno: 921427011
Fax: 921444366
Telegram: @fperal
Linkedin: www.linkedin.com/in/fppmm