Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

[David Lang <david@lang.hm>]

On Mon, 11 May 2015, Daniel Phillips wrote:

> On 05/11/2015 03:12 PM, Pavel Machek wrote:
>>>> It is a fact of life that when you change one aspect of an intimately interconnected system,
>>>> something else will change as well. You have naive/nonexistent free space management now; when you
>>>> design something workable there it is going to impact everything else you've already done. It's an
>>>> easy bet that the impact will be negative, the only question is to what degree.
>>>
>>> You might lose that bet. For example, suppose we do strictly linear allocation
>>> each delta, and just leave nice big gaps between the deltas for future
>>> expansion. Clearly, we run at similar or identical speed to the current naive
>>> strategy until we must start filling in the gaps, and at that point our layout
>>> is not any worse than XFS, which started bad and stayed that way.
>>
>> Umm, are you sure. If "some areas of disk are faster than others" is
>> still true on todays harddrives, the gaps will decrease the
>> performance (as you'll "use up" the fast areas more quickly).
>
> That's why I hedged my claim with "similar or identical". The
> difference in media speed seems to be a relatively small effect
> compared to extra seeks. It seems that XFS puts big spaces between
> new directories, and suffers a lot of extra seeks because of it.
> I propose to batch new directories together initially, then change
> the allocation goal to a new, relatively empty area if a big batch
> of files lands on a directory in a crowded region. The "big" gaps
> would be on the order of delta size, so not really very big.

This is an interesting idea, but what happens if the files don't arrive as a big 
batch, but rather trickle in over time (think a logserver that if putting files 
into a bunch of directories at a fairly modest rate per directory)

And when you then decide that you have to move the directory/file info, doesn't 
that create a potentially large amount of unexpected IO that could end up 
interfering with what the user is trying to do?

David Lang