Re: Next steps in recovery?

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2021/9/6 下午10:42, Robert Wyrick wrote:
> 42+ hours of memtest86+, no errors detected.  4 passes complete.
> Is that good enough?

That's strange, such obvious bitflip should be easily detected.

Is the fs only mounted on that computer?

Anyway, you can continue try to repair with *latest* btrfs-progs.

Thanks,
Qu
>
> On Sun, Sep 5, 2021 at 4:03 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>
>> On 2021/9/6 上午12:00, Robert Wyrick wrote:
>>> Running memtest86+ now....  20 hours in.  No errors yet.
>>> Thanks for the analysis.  I'll let this run for another day or so.
>>
>> Just to mention, since 5.11 btrfs kernel module has the ability to
>> detect most high bitflip before writing tree blocks to disks.
>>
>> Thus even with less reliable RAM, it's still more reliable than nothing.
>>
>> But still, with the existing errors, the RAM test is still an essential
>> one before doing anything.
>>
>> Thanks,
>> Qu
>>>
>>>
>>> On Fri, Sep 3, 2021 at 12:53 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2021/9/3 下午2:48, Qu Wenruo wrote:
>>>>>
>>>>>
>>>>> On 2021/9/3 上午10:43, Robert Wyrick wrote:
>>>>>> I cannot mount my btrfs filesystem.
>>>>>> $ uname -a
>>>>>> Linux bigbox 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11
>>>>>> 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
>>>>>> $ btrfs version
>>>>>> btrfs-progs v5.4.1
>>>>>
>>>>> The tool is a little too old, thus if you're going to repair, you'd
>>>>> better to update the progs.
>>>>>>
>>>>>> I'm seeing the following from check:
>>>>>> $ btrfs check -p /dev/sda
>>>>>> Opening filesystem to check...
>>>>>> Checking filesystem on /dev/sda
>>>>>> UUID: 75f1f45c-552e-4ae2-a56f-46e44b6647cf
>>>>>> [1/7] checking root items                      (0:00:59 elapsed,
>>>>>> 2649102 items checked)
>>>>>> ERROR: invalid generation for extent 38179182174208, have
>>>>>> 140737491486755 expect (0, 4057084]
>>>>>
>>>>> This is a repairable problem.
>>>>>
>>>>> We have test case for exactly the same case in tests/fsck-test/044 for it.
>>>>
>>>> Oh, this invalid extent generation is already a more direct indication
>>>> of memory bitflip.
>>>>
>>>> 140737491486755 = 0x8000002fc823
>>>>
>>>> Without the high 0x8 bit, the remaining part is completely valid
>>>> generation, 0x2fc823, which is inside the expectation.
>>>>
>>>> So, a memtest is a must before doing any repair.
>>>> You won't want another bitflip to ruin your perfectly repairable fs.
>>>>
>>>> Thanks,
>>>> Qu
>>>>>
>>>>>
>>>>>> [2/7] checking extents                         (0:02:17 elapsed,
>>>>>> 1116143 items checked)
>>>>>> ERROR: errors found in extent allocation tree or chunk allocation
>>>>>> cache and super generation don't match, space cache will be invalidated
>>>>>> [3/7] checking free space cache                (0:00:00 elapsed)
>>>>>> [4/7] chunresolved ref dir 8348950 index 3 namelen 7 name posters
>>>>>> filetype 2 errors 2, no dir index
>>>>>
>>>>> No dir index can also be repaired.
>>>>>
>>>>> The dir index will be added back.
>>>>>
>>>>>> unresolved ref dir 8348950 index 3 namelen 7 name poSters filetype 2
>>>>>> errors 5, no dir item, no inode ref
>>>>>
>>>>> No dir item nor inode ref can also be repaired, but with dir item and
>>>>> inode ref removed.
>>>>>
>>>>> But the problem here looks very strange.
>>>>>
>>>>> It's the same dir and the same index, but different name.
>>>>> posters vs poSters.
>>>>>
>>>>> 'S' is 0x53 and 's' is 0x73, I'm wondering if your system had a bad
>>>>> memory which caused a bitflip and the problem.
>>>>>
>>>>> Thus I prefer to do a full memtest before running btrfs check --repair.
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>>
>>>>>> [4/7] checking fs roots                        (0:00:42 elapsed,
>>>>>> 108894 items checked)
>>>>>> ERROR: errors found in fs roots
>>>>>> found 15729059057664 bytes used, error(s) found
>>>>>> total csum bytes: 15313288548
>>>>>> total tree bytes: 18286739456
>>>>>> total fs tree bytes: 1791819776
>>>>>> total extent tree bytes: 229130240
>>>>>> btree space waste bytes: 1018844959
>>>>>> file data blocks allocated: 51587230502912
>>>>>>     referenced 15627926712320
>>>>>>
>>>>>> I've tried everything I've found on the internet, but haven't
>>>>>> attempted to repair based on the warnings...
>>>>>>
>>>>>> What more info do you need to help me diagnose/fix this?
>>>>>>
>>>>>> Thanks!
>>>>>> -Rob
>>>>>>
>