Re: Feature requests: online backup - defrag - change RAID level

[webmaster@zedlx.com]

Quoting Remi Gauvin <remi@georgianit.com>:

> On 2019-09-11 7:21 p.m., webmaster@zedlx.com wrote:
>
>> For example, lets examine the typical home user. If he is using btrfs,
>> it means he probably wants snapshots of his data. And, after a few
>> snapshots, his data is fragmented, and the current defrag can't help
>> because it does a terrible job in this particualr case.
>>
>
> I shouldn't be replying to your provocative posts, but this is just
> nonsense.

I really hope that I'm not such a bad person as that sentence is suggesting.
About the provocative posts, I don't know of any other way to get my  
thoughts across.
If I offended people, I appologize, but I cannot change the way I communicate.

Certainly, I like the btrfs filesystem and the features it offers, and  
I'll continue using it, and no matter what you think of me I want to  
say thanks to you guys who are making it all work.

>  Not to say that Defragmentation can't be better, smarter,, it happens
> to work very well for typical use.

My thought is that the only reason why it appears that it works is  
that a typical home user rarely needs defragmentation. He runs the  
"btrfs fi defrag", virtually nothing happens (a few log files get  
defragged, if they were shared than they are unshared), prints out  
"Done", the user is happy. Placebo effect.

> This sounds like you're implying that snapshots fragment data... can you
> explain that?  as far as I know, snapshotting has nothing to do with
> fragmentation of data.  All data is COW, and all files that are subject
> to random read write will be fragmented, with or without snapshots.

Close, but essentially: yes. I'm implying that snapshots induce future  
fragmentation. The mere act of snapshoting won't create fragments  
immediately, but if there are any future writes to previously  
snapshoted files, those writes are likely to cause fragmentation. I  
think that this is not hard to figure out, but if you wish, I can  
elaborate further.

The real question is: does it really matter? Looking at the typical  
home user, most of his files rarely change, they are rarely written  
to. More likely, most new writes will go to new files. So, maybe the  
"home user" is not the best study-case for defragmentation. He has to  
be at least some kind of power-user, or content-creator to experience  
any significant fragmentation.

> And running defrag on your system regularly works just fine.  There's a
> little overhead of space if you are taking regular snapshots, (say
> hourly snapshots with snapper.)  If you have more control/liberty when
> you take your snapshots, ideally, you would defrag before taking the
> snaptshop/reflink copy.  Again, this only matters to files that are
> subject to fragmentation in the first place.

Btrfs defrag works just fine until you get some serious fragmentation.  
At that point, if you happen to have some snapshots, you better delete  
them before running defrag. Because, if you do run defrag on  
snapshoted and heavily fragmented filesystem, you are going to run out  
of disk space really fast.

> I suspect if you actually tried using the btrfs defrag, you would find
> you are making a mountain of a molehill.. There are lots of far more
> important problems to solve.

About importance, well, maybe you are right there, maybe not. Somehow  
I guess that after so many years in development and a stable feature  
set, most remaining problems are bugs and trivialities. So you are  
fixing them, one by one, many of those are urgent. I see you are  
working on deduplication, that's a hell of a work which actually won't  
end up well if it is not supplemented by a good defrag.

Didn't someone say, earlier in this discussion, that the defrag is  
important for btrfs. I would guess that it is. On many OSes defrag is  
run automatically. All older filesystems have a pretty good defrag.

What I would say is that btrfs can have a much better defrag than it  
has now. If defrag is deemed important, thay why are improvements to  
defrag unimportant?