Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2513 bytes --]



On 2018/9/20 上午4:11, Hans van Kranenburg wrote:
> On 09/19/2018 10:04 PM, Martin Steigerwald wrote:
>> Hans van Kranenburg - 19.09.18, 19:58:
>>> However, as soon as we remount the filesystem with space_cache=v2 -
>>>
>>>> writes drop to just around 3-10 MB/s to each disk. If we remount to
>>>> space_cache - lots of writes, system unresponsive. Again remount to
>>>> space_cache=v2 - low writes, system responsive.
>>>>
>>>> That's a huuge, 10x overhead! Is it expected? Especially that
>>>> space_cache=v1 is still the default mount option?
>>>
>>> Yes, that does not surprise me.
>>>
>>> https://events.static.linuxfound.org/sites/events/files/slides/vault20
>>> 16_0.pdf
>>>
>>> Free space cache v1 is the default because of issues with btrfs-progs,
>>> not because it's unwise to use the kernel code. I can totally
>>> recommend using it. The linked presentation above gives some good
>>> background information.
>>
>> What issues in btrfs-progs are that?
> 
> Missing code to make offline changes to a filesystem that has a free
> space tree. So when using btrfstune / repair / whatever you first need
> to remove the whole free space tree with a command, and then add it back
> on the next mount.
> 
> For me personally that's not a problem (I don't have to make offline
> changes), but I understand that having that situation out of the box for
> every new user would be a bit awkward.
> 
>> I am wondering whether to switch to freespace tree v2. Would it provide 
>> benefit for a regular / and /home filesystems as dual SSD BTRFS RAID-1 
>> on a laptop?
> 
> As shown in the linked presentation, it provides benefit on a largeish
> filesystem and if your writes are touching a lot of different block
> groups (since v1 writes out the full space cache for all of them on
> every transaction commit).

In fact that's the problem.

From free space cache inode flags, it's
NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC.

But the fact is, if it's modified, the whole file just get CoWed.

If we could change it to follow the inode flags, we could reduce
overhead even smaller than v2 one.
(v1 needs at least (1 + n) * sectorsize(4K) one for the header which
contains the csum, while v2 needs metadata CoW which is at least
nodesize (default to 16K)).

Thanks,
Qu

> I'd say, it provides benefit as soon as you
> encounter filesystem delays because of it, and as soon as you see using
> it eases the pain a lot. So, yes, that's your case.
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]