Re: Btrfs autodefrag wrote 5TB in one day to a 0.5TB SSD without a measurable benefit

[Jan Ziak <0xe2.0x9a.0x9b@gmail.com>]

On Sat, Mar 12, 2022 at 12:39 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> On 2022/3/12 07:28, Jan Ziak wrote:
> > On Sat, Mar 12, 2022 at 12:04 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >> As stated before, autodefrag is not really that useful for database.
> >
> > Do you realize that you are claiming that btrfs autodefrag should not
> > - by design - be effective in the case of high-fragmentation files?
>
> Unfortunately, that's exactly what I mean.
>
> We all know random writes would cause fragments, but autodefrag is not
> like regular defrag ioctl, as it only scan newer extents.
>
> For example:
>
> Our autodefrag is required to defrag writes newer than gen 100, and our
> inode has the following layout:
>
> |---Ext A---|--- Ext B---|---Ext C---|---Ext D---|---Ext E---|
>      Gen 50       Gen 101     Gen 49      Gen 30      Gen 30
>
> Then autodefrag will only try to defrag extent B and extent C.
>
> Extent B meets the generation requirement, and is mergable with the next
> extent C.
>
> But all the remaining extents A, D, E will not be defragged as their
> generations don't meet the requirement.
>
> While for regular defrag ioctl, we don't have such generation
> requirement, and is able to defrag all extents from A to E.
> (But cause way more IO).
>
> Furthermore, autodefrag works by marking the target range dirty, and
> wait for writeback (and hopefully get more writes near it, so it can get
> even larger)
>
> But if the application, like the database, is calling fsync()
> frequently, such re-dirtied range is going to writeback almost
> immediately, without any further chance to get merged larger.

So, basically, what you are saying is that you are refusing to work
together towards fixing/improving the auto-defragmentation algorithm.

Based on your decision in this matter, I am now forced either to find
a replacement filesystem with features similar to btrfs or to
implement a filesystem (where auto-defragmentation works correctly)
myself.

Since I failed to persuade you that there are serious errors/mistakes
in the current btrfs-autodefrag implementation, this is my last email
in this whole forum thread.

Sincerely
Jan