Re: off-by-one uncompressed invalid ram_bytes corruptions

[Steve Leung <sjleung@shaw.ca>]

[-- Attachment #1: Type: text/plain, Size: 3923 bytes --]

On 05/17/2018 11:49 PM, Qu Wenruo wrote:
> On 2018年05月18日 13:23, Steve Leung wrote:
>> Hi list,
>>
>> I've got 3-device raid1 btrfs filesystem that's throwing up some
>> "corrupt leaf" errors in dmesg.  This is a uniquified list I've
>> observed lately:

Evidently I forgot that I added a fourth device to this system, from the 
info below, but I don't think it matters.  :)

>>    BTRFS critical (device sda1): corrupt leaf: root=1 block=4970196795392
>> slot=307 ino=206231 file_offset=0, invalid ram_bytes for uncompressed
>> inline extent, have 3468 expect 3469
> 
> Would you please use "btrfs-debug-tree -b 4970196795392 /dev/sda1" to
> dump the leaf?

Attached btrfs-debug-tree dumps for all of the blocks that I saw 
messages for.

> It's caught by tree-checker code which is ensuring all tree blocks are
> correct before btrfs can take use of them.
> 
> That inline extent size check is tested, so I'm wondering if this
> indicates any real corruption.
> That btrfs-debug-tree output will definitely help.
> 
> BTW, if I didn't miss anything, there should not be any inlined extent
> in root tree.
> 
>>    BTRFS critical (device sda1): corrupt leaf: root=1 block=4970552426496
>> slot=91 ino=209736 file_offset=0, invalid ram_bytes for uncompressed
>> inline extent, have 3496 expect 3497
> 
> Same dump will definitely help.
> 
>>    BTRFS critical (device sda1): corrupt leaf: root=1 block=4970712399872
>> slot=221 ino=205230 file_offset=0, invalid ram_bytes for uncompressed
>> inline extent, have 1790 expect 1791
>>    BTRFS critical (device sda1): corrupt leaf: root=1 block=4970803920896
>> slot=368 ino=205732 file_offset=0, invalid ram_bytes for uncompressed
>> inline extent, have 2475 expect 2476
>>    BTRFS critical (device sda1): corrupt leaf: root=1 block=4970987945984
>> slot=236 ino=208896 file_offset=0, invalid ram_bytes for uncompressed
>> inline extent, have 490 expect 491
>>
>> All of them seem to be 1 short of the expected value.
>>
>> Some files do seem to be inaccessible on the filesystem, and btrfs
>> inspect-internal on any of those inode numbers fails with:
>>
>>   ERROR: ino paths ioctl: Input/output error
>>
>> and another message for that inode appears.
>>
>> 'btrfs check' (output attached) seems to notice these corruptions (among
>> a few others, some of which seem to be related to a problematic attempt
>> to build Android I posted about some months ago).
>>
>> Other information:
>>
>> Arch Linux x86-64, kernel 4.16.6, btrfs-progs 4.16.  The filesystem has
>> about 25 snapshots at the moment, only a handful of compressed files,
>> and nothing fancy like qgroups enabled.
>>
>> btrfs fi show:
>>
>>   Label: none  uuid: 9d4db9e3-b9c3-4f6d-8cb4-60ff55e96d82
>>           Total devices 4 FS bytes used 2.48TiB
>>           devid    1 size 1.36TiB used 1.13TiB path /dev/sdd1
>>           devid    2 size 464.73GiB used 230.00GiB path /dev/sdc1
>>           devid    3 size 1.36TiB used 1.13TiB path /dev/sdb1
>>           devid    4 size 3.49TiB used 2.49TiB path /dev/sda1
>>
>> btrfs fi df:
>>
>>   Data, RAID1: total=2.49TiB, used=2.48TiB
>>   System, RAID1: total=32.00MiB, used=416.00KiB
>>   Metadata, RAID1: total=7.00GiB, used=5.29GiB
>>   GlobalReserve, single: total=512.00MiB, used=0.00B
>>
>> dmesg output attached as well.
>>
>> Thanks in advance for any assistance!  I have backups of all the
>> important stuff here but it would be nice to fix the corruptions in place.
> 
> And btrfs check doesn't report the same problem as the default original
> mode doesn't have such check.
> 
> Please also post the result of "btrfs check --mode=lowmem /dev/sda1"

Also, attached.  It seems to notice the same off-by-one problems, though 
there also seem to be a couple of examples of being off by more than one.

Thanks for looking at this!  I'll get my backups ready, just in case.

Steve

[-- Attachment #2: btrfs-check-lowmem.txt.gz --]
[-- Type: application/gzip, Size: 5468 bytes --]

[-- Attachment #3: btrfs-debug-tree-4970196795392.txt.gz --]
[-- Type: application/gzip, Size: 9068 bytes --]

[-- Attachment #4: btrfs-debug-tree-4970552426496.txt.gz --]
[-- Type: application/gzip, Size: 8321 bytes --]

[-- Attachment #5: btrfs-debug-tree-4970712399872.txt.gz --]
[-- Type: application/gzip, Size: 7581 bytes --]

[-- Attachment #6: btrfs-debug-tree-4970803920896.txt.gz --]
[-- Type: application/gzip, Size: 8601 bytes --]

[-- Attachment #7: btrfs-debug-tree-4970987945984.txt.gz --]
[-- Type: application/gzip, Size: 7559 bytes --]