Re: BTRFS Raid 5 Space missing - ideas ?

[Adam Borowski <kilobyte@angband.pl>]

On Sat, Apr 20, 2019 at 12:46:16PM +0200, Juergen Sauer wrote:
> I wish a happy Easer Days before :)

Same to you!

> During my tests with BTRFS as Raid5 setup, I found a courious little
> "problem".

>         Total devices 3 FS bytes used 9.98TiB
>         devid    1 size 9.09TiB used 4.99TiB path /dev/sdb1
>         devid    2 size 5.46TiB used 4.99TiB path /dev/sdc1
>         devid    3 size 5.46TiB used 4.99TiB path /dev/sde1

> All patitioins sdb1 sdc1 sde1 are the same size: 9.0 TiB. But BTRFS ist
> not using the bigger space on sdc1, sde1, there is only 5.46 TiB used,
> even there are 9.0 Tib Avaible, so 4.0 TiB are unused.

It's working as expected: while btrfs does RAID per block group rather than
per whole block device, there's no way to place a raid5 block group in a way
that doesn't require at least 3 devices.  This means with a 3-disk setup the
space utilized will be only as big as the smallest one.

This is also the case for raid1 on 2-disk, and for raid10 on 4-disk.

Btrfs can use uneven disks only when it has some freedom how to place the
data around.

There's a tool that lets you visualize space utilization:
    http://carfax.org.uk/btrfs-usage/
or a command-line implementation:
    btrfs-space-calculator (package python[3]-btrfs)


By the way, you can greatly improve performance and safety by switching
metadata profile to raid1: "btrfs bal start -mraid1".  RAID5 is very slow
for random writes, which is nearly all metadata write access; RAID1 doesn't
suffer from this problem -- and metadata tends to be only around 1-2% of
space so having it take a bit more doesn't hurt.

It would also solve your utilization problem, except that metadata uses so
little space.  Having mixed block groups means the space not taken by RAID5
can be recovered by taking twice as much from sdb1 than from each of sdc1
and sde1:

sdb1 *********************
sdc1 * * * * * * * * * * *
sde1  * * * * * * * * * *
(each RAID1 block group is either sdb1,sdc1 or sdb1:sde1)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄⠀⠀⠀⠀