Re: [PATCH v2] btrfs: speedup mount time with readahead chunk tree

[Robbie Ko <robbieko@synology.com>]

David Sterba 於 2020/7/8 上午3:25 寫道:
> On Tue, Jul 07, 2020 at 11:59:44AM +0800, robbieko wrote:
>> From: Robbie Ko <robbieko@synology.com>
>>
>> When mounting, we always need to read the whole chunk tree,
>> when there are too many chunk items, most of the time is
>> spent on btrfs_read_chunk_tree, because we only read one
>> leaf at a time.
>>
>> It is unreasonable to limit the readahead mechanism to a
>> range of 64k, so we have removed that limit.
>>
>> In addition we added reada_maximum_size to customize the
>> size of the pre-reader, The default is 64k to maintain the
>> original behavior.
>>
>> So we fix this by used readahead mechanism, and set readahead
>> max size to ULLONG_MAX which reads all the leaves after the
>> key in the node when reading a level 1 node.
> The readahead of chunk tree is a special case as we know we will need
> the whole tree, in all other cases the search readahead needs is
> supposed to read only one leaf.

If, in most cases, readahead requires that only one leaf be read, then
reada_ maximum_size should be nodesize instead of 64k, or use
reada_maximum_ nr (default:1) seems better.

>
> For that reason I don't want to touch the current path readahead logic
> at all and do the chunk tree readahead in one go instead of the
> per-search.

I don't know why we don't make the change to readahead, because the current
readahead is limited to the logical address in 64k is very unreasonable,
and there is a good chance that the logical address of the next leaf 
node will
not appear in 64k, so the existing readahead is almost useless.

>
> Also I don't like to see size increase of btrfs_path just to use the
> custom once.

This variable is the parameter that controls the speed of the readahead,
and I think it should be adjustable, not the hard code in the readahead 
function.
In the future, more scenarios will be available.
For example, BGTREE. will be improved significantly faster,
My own tests have improved the speed by almost 500%.
Reference: https://lwn.net/Articles/801990 /

>
> The idea of the whole tree readahead is to do something like:
>
> - find first item
> - start readahead on all leaves from its level 1 node parent
>    (readahead_tree_block)
> - when the level 1 parent changes during iterating items, start the
>    readahead again
>
> This skips readahead of all nodes above level 1, if you find a nicer way
> to readahead the whole tree I won't object, but for the first
> implementation the level 1 seems ok to me.
>
>> I have a test environment as follows:
>>
>> 200TB btrfs volume: used 192TB
>>
>> Data, single: total=192.00TiB, used=192.00TiB
>> System, DUP: total=40.00MiB, used=19.91MiB
> Can you please check what's the chunk tree height? 'btrfs inspect
> tree-stats' prints that but it takes long as needs to go through the
> whole metadata, so extracting it from 'btrfs inspect dump-tree -c chunk'
> would be faster. Thanks.
Chunk tree height 3, level (0-2)