Re: Need help: super_total_bytes mismatch with fs_devices total_rw_bytes

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2504 bytes --]



On 2019/8/24 下午8:05, Patrick Dijkgraaf wrote:
> Thanks for the quick reply!
> See responses inline.
> 
> On Sat, 2019-08-24 at 19:01 +0800, Qu Wenruo wrote:
>> On 2019/8/24 下午2:48, Patrick Dijkgraaf wrote:
>>> Hi all,
>>>
>>> My server hung this morning, and I had to hard-reset is. I did not
>>> apply any updates. After the reboot, my FS won't mount:
>>>
>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2):
>>> super_total_bytes
>>> 92017957797888 mismatch with fs_devices total_rw_bytes
>>> 184035915595776
>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2): failed to
>>> read
>>> chunk tree: -22
>>> [Sat Aug 24 08:16:31 2019] BTRFS error (device sde2): open_ctree
>>> failed
>>>
>>> However, running btrfs rescue shows:
>>> root@cornelis ~]# btrfs rescue fix-device-size /dev/sdh2
>>> No device size related problem found
>>
>> That's strange.
>>
>> Would you please dump the chunk tree and super blocks?
>> # btrfs ins dump-super -fFa /dev/sdh2
> 
> See: https://pastebin.com/f5Wn15sx

Did a quick calculation, from your fi show result, it's 83.72TiB, thus
the super total_bytes is correct.

It's the kernel doing incorrect calculation for its
fs_devices->total_rw_bytes.

This matches the output of dump-super. No wonder why btrfs-progs refuse
to fix.
> 
>> # btrfs ins dump-tree -t chunk /dev/sdh2
> 
> This output is too large for pastebin. The output is
> viewable/downloadable here: https://kwek.duckstad.net/tree.txt

This also proves your chunk tree is CORRECT!
The sum of all devices is 92017957797888, which matches with super block.
[...]
> 
>> The simplest way to fix it is to just update the
> 
> Nice teaser! 😉 What should I update?

Sorry, I meant to say just update the "superblock", but it turns out,
it's something wrong with your kernel. Probably some old bug we fixed
before.

Would you try to use latest ARCH kernel from an Archiso to try to mount
it RO (just to be safe)?

I checked latest v5.3-rc kernels, haven't found an obvious problem with
the fs_devices->total_rw_bytes update routines.

So it may be an old bug which is already fixed.

Thanks,
Qu

> 
>> Thanks,
>> Qu
>>> Other info:
>>> [root@cornelis ~]# uname -r
>>> 4.18.16-arch1-1-ARCH
>>>
>>> I was able to mount is using:
>>> [root@cornelis ~]# mount -o usebackuproot,ro /dev/sdh2 /mnt/data
>>>
>>> Now updating my backup, but I REALLY hope to get this fixed on the
>>> production server!
>>>
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]