On 2019/8/25 下午2:41, Patrick Dijkgraaf wrote:
> Hi Qu,
> 
> At the end of my first initial post, I mentioned that I finally was
> able to mount the volume using:
> 
> mount -o usebackuproot,ro /dev/sdh2 /mnt/data
> 
> The chunk tree and super blocks dumps were taken after that.
> 
> Now I noticed that I was able to mount the volume without special
> options (same kernel version). YAY! ☺
> Could it be that the "usebackuproot,ro" mount options already fixed the
> issue?

Nope. It should be something wrong with the kernel code verifying the
total devices space.
As long as all your later mount is using ro, and no log tree replay
happened, the fs should not be modified.
It may be a race, but anyway, your on-disk data is completely valid, so
as long as your kernel is doing something correct, it should mount the fs.

Thanks,
Qu

> 
> Cheers,
> Patrick
> 
> 
> On Sat, 2019-08-24 at 21:24 +0800, Qu Wenruo wrote:
>> On 2019/8/24 下午8:05, Patrick Dijkgraaf wrote:
>>> Thanks for the quick reply!
>>> See responses inline.
>>>
>>> On Sat, 2019-08-24 at 19:01 +0800, Qu Wenruo wrote:
>>>> On 2019/8/24 下午2:48, Patrick Dijkgraaf wrote:
>>>>> Hi all,
>>>>>
>>>>> My server hung this morning, and I had to hard-reset is. I did
>>>>> not
>>>>> apply any updates. After the reboot, my FS won't mount:
>>>>>
>>>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2):
>>>>> super_total_bytes
>>>>> 92017957797888 mismatch with fs_devices total_rw_bytes
>>>>> 184035915595776
>>>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2): failed to
>>>>> read
>>>>> chunk tree: -22
>>>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2):
>>>>> open_ctree
>>>>> failed
>>>>>
>>>>> However, running btrfs rescue shows:
>>>>> root@cornelis ~]# btrfs rescue fix-device-size /dev/sdh2
>>>>> No device size related problem found
>>>>
>>>> That's strange.
>>>>
>>>> Would you please dump the chunk tree and super blocks?
>>>> # btrfs ins dump-super -fFa /dev/sdh2
>>>
>>> See: 
>>> https://pastebin.com/f5Wn15sx
>>>
>>
>> Did a quick calculation, from your fi show result, it's 83.72TiB,
>> thus
>> the super total_bytes is correct.
>>
>> It's the kernel doing incorrect calculation for its
>> fs_devices->total_rw_bytes.
>>
>> This matches the output of dump-super. No wonder why btrfs-progs
>> refuse
>> to fix.
>>>> # btrfs ins dump-tree -t chunk /dev/sdh2
>>>
>>> This output is too large for pastebin. The output is
>>> viewable/downloadable here: 
>>> https://kwek.duckstad.net/tree.txt
>>>
>>
>> This also proves your chunk tree is CORRECT!
>> The sum of all devices is 92017957797888, which matches with super
>> block.
>> [...]
>>>> The simplest way to fix it is to just update the
>>>
>>> Nice teaser! 😉 What should I update?
>>
>> Sorry, I meant to say just update the "superblock", but it turns out,
>> it's something wrong with your kernel. Probably some old bug we fixed
>> before.
>>
>> Would you try to use latest ARCH kernel from an Archiso to try to
>> mount
>> it RO (just to be safe)?
>>
>> I checked latest v5.3-rc kernels, haven't found an obvious problem
>> with
>> the fs_devices->total_rw_bytes update routines.
>>
>> So it may be an old bug which is already fixed.
>>
>> Thanks,
>> Qu
>>
>>>> Thanks,
>>>> Qu
>>>>> Other info:
>>>>> [root@cornelis ~]# uname -r
>>>>> 4.18.16-arch1-1-ARCH
>>>>>
>>>>> I was able to mount is using:
>>>>> [root@cornelis ~]# mount -o usebackuproot,ro /dev/sdh2
>>>>> /mnt/data
>>>>>
>>>>> Now updating my backup, but I REALLY hope to get this fixed on
>>>>> the
>>>>> production server!
>>>>>
>>>
>>>
> 
>