Re: Linux raid-like idea

[antlists <antlists@youngman.org.uk>]

On 11/09/2020 21:14, Brian Allen Vanderburg II wrote:
> That's right, I get the various combinations confused.  So does raid61
> allow for losing 4 disks in any order and still recovering? or would
> some order of disks make it where just 3 disks lost and be bad?
> Iinteresting non-the-less and I'll have to look into it.  Obviously it's
> not intended to as a replacement for backing up important data, but, for
> me any way, just away to minimize loss of any trivial bulk data/files.

Yup. Raid 6 has two parity disks, and that's mirrored to give four 
parity disks. So as an *absolute* *minimum*, raid-61 could lose four 
disks with no data loss.

Throw in the guarantee that, with a mirror, you can lose an entire 
mirror with no data-loss, that means - with luck and a following wind - 
you could lose half your disks, PLUS the two parities in the remaining 
disks, and still recover your data. So with a raid-6+1, if I had twelve 
disks, I could lose EIGHT disks and still have a *chance* of recovering 
my array. I'm not quite sure what difference raid-61 would make.

(That says to me, if I have a raid-61, I need as a minimum a complete 
set of data disks. That also says to me, if I've splatted an 8+2 raid-61 
across 11 disks, I only need 7 for a full recovery despite needing a 
minimum of 8, so something isn't quite right here... I suspect the 7 
would be enough but I did say my mind goes Whooaaa!!!!)
> 
> It would be nice if the raid modules had support for methods that could
> support a total of more disks in any order lost without loosing data.
> Snapraid source states that it uses some Cauchy Matrix algorithm which
> in theory could loose up to 6 disks if using 6 parity disks, in any
> order, and still be able to restore the data.  I'm not familiar with the
> math behind it so can't speak to the accuracy of that claim.

That's easy, it's just whether it's worth it. Look at the maths behind 
raid-6. The "one parity disk" methods, 4 or 5, just use XOR. But that 
only works once, a second XOR parity disk adds no new redundancy and is 
worthless. I'm guessing raid-6 uses that Cauchy method you talk about - 
certainly it can generate as many parity disks as you like ... so that 
claim is good, even if raid-6 doesn't use that particular technique.

If someone wants to, mod'ing raid-6 to use 3 parity disks shouldn't be 
that hard ...


But going back to your original idea, I've been thinking about it. And 
it struck me - you NEED to regenerate parity EVERY TIME you write data 
to disk! Otherwise, writing one file on one disk instantly trashes your 
ability to recover all the other files in the same position on the other 
disks. WHOOPS! But if you think it's a good idea, by all means try and 
do it.

The other thing I'd suggest here, is try and make it more like raid-5 
than raid-4. You have X disks, let's say 5. So one disk each is numbered 
0, 1, 2, 3, 4. As part of formatting the disk ready for raid, you create 
a file containing every block where LBA mod 5 equals disk number. So as 
you recalculate your parities, that's where they go.

Cheers,
Wol