Re: Anyone tried out btrbk yet?

[Duncan <1i5t5.duncan@cox.net>]

Donald Pearson posted on Mon, 20 Jul 2015 00:15:26 -0500 as excerpted:

> I'm starting to think there's something wrong with creating and removing
> snapshots that leaves btrfs-cleaner either locked up or nearly so.  If
> the btrfs-cleaner process was hard-disk limited I should be seeing some
> HDD I/O to coincide but I don't.
> 
> So far btrfs-cleaner is has been using lots of CPU for 1900+ hours and
> my disk I/O is basically idle.  My hourly snaps via cronjob stalled 11
> hours ago.
> 
> Otherwise attempts to read/write to the filesystem appear to be
> perfectly normal.

Hourly snaps.  How many snapshots/subvolumes on the filesystem?  I assume 
the snap removal you mention is scheduled thinning?

A general rule of thumb is under 2000 snapshots per filesystem total, 
under 1000 if at all reasonable.  At 250 snapshots per subvolume, 2000 
snapshots is eight subvolumes worth, and 250-ish snapshots per subvolume 
is well within reason with reasonable thinning.

If you have lots of subvolumes, 3000 snapshots per filesystem isn't /too/ 
bad, but filesystem maintenance including snapshot deletion simply does 
/not/ scale well, and you'll likely run into scalability issues if you 
let it reach 10K snapshots on the filesystem.

If you're at 10K snapshots on the filesystem, it's quite likely the usual 
scalability issue's you're seeing, almost certain at 100K+ snapshots.  
OTOH, if you're under 1K or even 2K snapshots, it's very likely something 
abnormal.  2K-10K, should be usable in most cases, but there are likely 
corner-cases where it's bad.

Also, FWIW, the btrfs quota subsystem increases snapshot management 
complexity dramatically, so if you're using that, aim for the low ends of 
the above recommendation if at all possible, and/or consider either 
turning off the quota stuff or using a filesystem other than btrfs, as in 
addition to the scaling issues, the quota management code has been a 
source of repeated bugs and isn't a feature I'd recommend relying on 
until it has at least several kernel cycles worth of trouble-free history 
behind it.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman