Re: Still seeing high autodefrag IO with kernel 5.16.5

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2022/2/6 15:51, Rylee Randall wrote:
> I am experiencing the same issue on 5.16.7, near the end of a large
> steam game download btrfs-cleaner sits at the top of iotop, and shut
> downs take about ten minutes because of various btrfs hangs.
>
> I compiled 5.15.21 with the mentioned patch and tried to recreate the
> issue and so far have been unable to. I seem to get far faster an dmore
> consistent write speeds from steam, and rather than btrfs-cleaner being
> the main source of io usage it is steam. btrfs-cleaner is far down the
> list along with various other btrfs- tasks.

Thanks for the report, this indeed looks like the bug in v5.15 that it
doesn't defrag a lot of extents is not the root cause.

Mind to re-check with the following branch?

https://github.com/adam900710/linux/tree/autodefrag_fixes

It has one extra patch to emulate the older behavior of not using
btrfs_get_em(), which can cause quite some problem for autodefrag.

Thanks,
Qu

>
> On 4/2/22 12:54, Qu Wenruo wrote:
>>
>>
>> On 2022/2/4 09:17, Qu Wenruo wrote:
>>>
>>>
>>> On 2022/2/4 04:05, Benjamin Xiao wrote:
>>>> Hello all,
>>>>
>>>> Even after the defrag patches that landed in 5.16.5, I am still seeing
>>>> lots of cpu usage and disk writes to my SSD when autodefrag is enabled.
>>>> I kinda expected slightly more IO during writes compared to 5.15, but
>>>> what I am actually seeing is massive amounts of btrfs-cleaner i/o even
>>>> when no programs are actively writing to the disk.
>>>>
>>>> I can reproduce it quite reliably on my 2TB Btrfs Steam library
>>>> partition. In my case, I was downloading Strange Brigade, which is a
>>>> roughly 25GB download and 33.65GB on disk. Somewhere during the
>>>> download, iostat will start reporting disk writes around 300 MB/s, even
>>>> though Steam itself reports disk usage of 40-45MB/s. After the download
>>>> finishes and nothing else is being written to disk, I still see around
>>>> 90-150MB/s worth of disk writes. Checking in iotop, I can see btrfs
>>>> cleaner and other btrfs processes writing a lot of data.
>>>>
>>>> I left it running for a while to see if it was just some maintenance
>>>> tasks that Btrfs needed to do, but it just kept going. I tried to
>>>> reboot, but it actually prevented me from properly rebooting. After
>>>> systemd timed out, my system finally shutdown.
>>>>
>>>> Here are my mount options:
>>>> rw,relatime,compress-force=zstd:2,ssd,autodefrag,space_cache=v2,subvolid=5,subvol=/
>>>>
>>>>
>>>
>>> Compression, I guess that's the reason.
>>>
>>>  From the very beginning, btrfs defrag doesn't handle compressed extent
>>> well.
>>>
>>> Even if a compressed extent is already at its maximum capacity, btrfs
>>> will still try to defrag it.
>>>
>>> I believe the behavior is masked by other problems in older kernels thus
>>> not that obvious.
>>>
>>> But after rework of defrag in v5.16, this behavior is more exposed.
>>
>> And if possible, please try this diff on v5.15.x, and see if v5.15 is
>> really doing less IO than v5.16.x.
>>
>> The diff will solve a problem in the old code, where autodefrag is
>> almost not working.
>>
>> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
>> index cc61813213d8..f6f2468d4883 100644
>> --- a/fs/btrfs/ioctl.c
>> +++ b/fs/btrfs/ioctl.c
>> @@ -1524,13 +1524,8 @@ int btrfs_defrag_file(struct inode *inode, struct
>> file *file,
>>                         continue;
>>                 }
>>
>> -               if (!newer_than) {
>> -                       cluster = (PAGE_ALIGN(defrag_end) >>
>> -                                  PAGE_SHIFT) - i;
>> -                       cluster = min(cluster, max_cluster);
>> -               } else {
>> -                       cluster = max_cluster;
>> -               }
>> +               cluster = (PAGE_ALIGN(defrag_end) >> PAGE_SHIFT) - i;
>> +               cluster = min(cluster, max_cluster);
>>
>>                 if (i + cluster > ra_index) {
>>                         ra_index = max(i, ra_index);
>>
>>>
>>> There are patches to address the compression related problem, but not
>>> yet merged:
>>>
>>> https://patchwork.kernel.org/project/linux-btrfs/list/?series=609387
>>>
>>> Mind to test them to see if that's the case?
>>>
>>> Thanks,
>>> Qu
>>>>
>>>>
>>>> I've disabled autodefrag again for now to save my SSD, but just wanted
>>>> to say that there is still an issue. Have the defrag issues been fully
>>>> fixed or are there more patches incoming despite what Reddit and
>>>> Phoronix say? XD
>>>>
>>>> Thanks!
>>>> Ben
>>>>
>>>>