Re: Feature requests: online backup - defrag - change RAID level

[webmaster@zedlx.com]

This post is a reply to Remi Gauvin's post, but the email got lost so  
I can't reply to him directly.

Remi Gauvin wrote on 2019-09-09 17:24 :
>
> On 2019-09-09 11:29 a.m., Graham Cobb wrote:
>
>>  and does anyone really care about
>> defrag any more?).
>>
>
>
> Err, yes, yes absolutely.
>
> I don't have any issues with the current btrfs defrag implementions, but
> it's *vital* for btrfs. (which works just as the OP requested, as far as
> I can tell, recursively for a subvolume)
>
> Just booting Windows on a BTRFS virtual image, for example, will create
> almost 20,000 file fragments.  Even on SSD's, you get into problems
> trying to work with files that are over 200,000 fragments.
>
> Another huge problem is rsync --inplace.  which is perfect backup
> solution to take advantage of BTRFS snapshots, but fragments larges
> files into tiny pieces (and subsequently creates files that are very
> slow to read.).. for some reason, autodefrag doesn't catch that one either.
>
> But the wiki could do a beter job of trying to explain that the snapshot
> duplication of defrag only affects the fragmented portions.  As I
> understand, it's really only a problem when using defrag to change
> compression.


Ok, a few things.

First, my defrag suggestion doesn't EVER unshare extents. The defrag  
should never unshare, not even a single extent. Why? Because it  
violates the expectation that defrag would not decrease free space.

Defrag may break up extents. Defrag may fuse extents. But it shouln't  
ever unshare extents.

Therefore, I doubt that the current defrag does "just as the OP  
requested". Nonsense. The current implementation does the unsharing  
all the time.

Second, I never used btrfs defrag in my life, despite mananging at  
least 10 btrfs filesystems. I can't. Because, all my btrfs volumes  
have lot of subvolumes, so I'm afraid that defrag will unshare much  
more than I can tolerate. In my subvolumes, over 90% of data is  
shared. If all subvolumes were to be unshared, the disk usage would  
likely increase tenfold, and that I cannot afford.

I agree that btrfs defrag is vital. But currently, it's unusable for  
many use cases.

Also, I don't quite understand what the poster means by "the snapshot  
duplication of defrag only affects the fragmented portions". Possibly  
it means approximately: if a file wasn't modified in the current  
(latest) subvolume, it doesn't need to be unshared. But, that would  
still unshare all the log files, for example, even all files that have  
been appended, etc... that's quite bad. Even if just one byte was  
appended to a log file, then defrag will unshare the entire file (I  
suppose).