Re: Recommended why to use btrfs for production?

[Duncan <1i5t5.duncan@cox.net>]

Austin S. Hemmelgarn posted on Fri, 03 Jun 2016 10:21:12 -0400 as
excerpted:

> As far as BTRFS raid10 mode in general, there are a few things that are 
> important to remember about it:
> 1. It stores exactly two copies of everything, any extra disks just add 
> to the stripe length on each copy.

I'll add one more, potentially very important, related to this one:

Btrfs raid mode (any of them) works in relation to individual chunks, 
*NOT* individual devices.

What that means for btrfs raid10 in combination with the above exactly 
two copies rule, is that it works rather differently than a standard 
raid10, which can tolerate loss of two devices as long as they're from 
the same mirror set, as the other mirror set will then still be whole.  
Because with btrfs raid10 the mirror sets are dynamic per-chunk, loss of 
a second device close to assures loss of data, because the very likely 
true assumption is that both mirror sets will be affected for some 
chunks, but not others.

By using a layered approach, btrfs raid1 on top (for its error correction 
from the other copy feature) of a pair of mdraid0s, you force one of the 
btrfs raid1 copies to each of the mdraid0s, thus making allocation more 
deterministic than btrfs raid10, and can thus again tolerate loss of two 
devices, as long as they're from the same underlying mdraid0.

(Traditionally, raid1 on top of raid0 is called raid01, and is 
discouraged compared to raid10, raid0 on top of raid1, because device 
failure and replacement with the latter triggers a much more localized 
rebuild than the former, across the pair of devices in the raid1 when 
it's closest to the physical devices, across the whole array, one raid0 
to the other, when the raid1 is on top.  However, btrfs raid1's data 
integrity and error repair from the good mirror feature is generally 
considered to be useful enough to be worth the rebuild-inefficiency of 
the raid01 design.)

So in regard to failure tolerance, btrfs raid10 is far closer to 
traditional raid5, loss of a single device is tolerated, loss of a second 
before a repair is complete generally means data loss -- there's not the 
chance of it being on the same mirror set to save you that traditional 
raid10 has.

Similarly, btrfs raid10 doesn't have the cleanly separate pair of mirrors 
on raid0 arrays that traditional raid10 does, thus doesn't have the fault 
tolerance of losing say the connection or power to one entire device 
bank, as long as it's all one mirror set, that traditional raid10 has.

And again, doing the layered thing with btrfs raid1 on top and mdraid0 
(or whatever else) underneath gets that back for you, if you set it up 
that way, of course.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman