[linux-lvm] (Why) Does lvmcreate --raid6 need 5 drives?

[linux-lvm] (Why) Does lvmcreate --raid6 need 5 drives?

[tudor]

This question was originally put on ServerFault here: 
https://serverfault.com/questions/931150/why-does-lvmraid6-need-5-drives

I'm investigating using LVMRAID instead of MD+LVM. In my case, I'm 
looking at a RAID6. As expected, the docs say you need N+1 devices for 
RAID5, and a minimum of 3 drives. The docs say for RAID 6 you need N+2.

However, it then says that the minimum drives required is 5, which means 
that N is different.

In an MD RAID6, only 4 drives are required; 2 for striping, 2 for 
parity. So if I built RAID6+LVM I'd only need 4 drives whereas, 
according to the docs, I'd need 5 drives for LVMRAID6.  It was my 
understanding that LVMRAID is really just LVM on MD, so I don't 
understand the difference.

Thomas on ServerFault pointed out that a minimum of 3 stripes is 
hardcoded into the source code[1].

But the commit message[2] doesn't seem to indicate a reasoning behind 
requiring a higher number of striped for RAID6 than RAID5.

Why does LVMRAID need 3 stripes for RAID6 where MD only needs 2?

Cheers,
Tudor.

[1] 
https://sourceware.org/git/?p=lvm2.git;a=blob;f=tools/lvcreate.c;h=3c22794b6f1c3cbd155df65421de21dac0bfa9fc;hb=HEAD#l528
[2] 
https://sourceware.org/git/?p=lvm2.git;a=commit;f=tools/lvcreate.c;h=ea0f604e704dc8418d962e20529fd1bcddf06bb7

Re: [linux-lvm] (Why) Does lvmcreate --raid6 need 5 drives?

[Heinz Mauelshagen]

On 09/17/2018 01:36 AM, tudor@tudorholton.com wrote:
> This question was originally put on ServerFault here: 
> https://serverfault.com/questions/931150/why-does-lvmraid6-need-5-drives
>
> I'm investigating using LVMRAID instead of MD+LVM. In my case, I'm 
> looking at a RAID6. As expected, the docs say you need N+1 devices for 
> RAID5, and a minimum of 3 drives. The docs say for RAID 6 you need N+2.
>
> However, it then says that the minimum drives required is 5, which 
> means that N is different.
>
> In an MD RAID6, only 4 drives are required; 2 for striping, 2 for 
> parity. So if I built RAID6+LVM I'd only need 4 drives whereas, 
> according to the docs, I'd need 5 drives for LVMRAID6.  It was my 
> understanding that LVMRAID is really just LVM on MD, so I don't 
> understand the difference.

That's correct.


>
> Thomas on ServerFault pointed out that a minimum of 3 stripes is 
> hardcoded into the source code[1].
>
> But the commit message[2] doesn't seem to indicate a reasoning behind 
> requiring a higher number of striped for RAID6 than RAID5.
>
> Why does LVMRAID need 3 stripes for RAID6 where MD only needs 2?

We do have a constraint in lvm2 to require the raid6 minimum for N to be 3.

Configuring a raid6 LV (an array by MD terms) with 2 data stripes is 
suboptimal for performance,
because data striping is minimal in this case.  In addition, the 
metadata overhead is maximal
for parity, P- and Q-syndromes being half of the brutto size of the 
raid6 LV.

Regards,
Heinz

>
> Cheers,
> Tudor.
>
> [1] 
> https://sourceware.org/git/?p=lvm2.git;a=blob;f=tools/lvcreate.c;h=3c22794b6f1c3cbd155df65421de21dac0bfa9fc;hb=HEAD#l528
> [2] 
> https://sourceware.org/git/?p=lvm2.git;a=commit;f=tools/lvcreate.c;h=ea0f604e704dc8418d962e20529fd1bcddf06bb7
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] (Why) Does lvmcreate --raid6 need 5 drives?

[tudor]

--------snip------
> We do have a constraint in lvm2 to require the raid6 minimum for N to 
> be 3.
> 
> Configuring a raid6 LV (an array by MD terms) with 2 data stripes is
> suboptimal for performance,
> because data striping is minimal in this case.  In addition, the
> metadata overhead is maximal
> for parity, P- and Q-syndromes being half of the brutto size of the 
> raid6 LV.
> 
My apologies.  I googled "brutto size" but came up with nothing.  I do 
have a basic understanding of P and Q syndromes so I think I have a 
vague understanding of your meaning.

I understand it's suboptimal, but not non-existent.  In particular why 
is this a restriction and not just a warning?

Performance aside, I could also argue that there's a use case for a 
minimal (and not degraded) set before expansion.

Cheers,
Tudor.

Re: [linux-lvm] (Why) Does lvmcreate --raid6 need 5 drives?

[Heinz Mauelshagen]

On 09/18/2018 01:41 AM, tudor@tudorholton.com wrote:
> --------snip------
>> We do have a constraint in lvm2 to require the raid6 minimum for N to 
>> be 3.
>>
>> Configuring a raid6 LV (an array by MD terms) with 2 data stripes is
>> suboptimal for performance,
>> because data striping is minimal in this case.  In addition, the
>> metadata overhead is maximal
>> for parity, P- and Q-syndromes being half of the brutto size of the 
>> raid6 LV.
>>
> My apologies.  I googled "brutto size" but came up with nothing. I do 
> have a basic understanding of P and Q syndromes so I think I have a 
> vague understanding of your meaning.

I have to apologize, I meant gross size

>
> I understand it's suboptimal, but not non-existent.  In particular why 
> is this a restriction and not just a warning?

The lvm2 code assumes data stripes to be more than parity/syndrome stripes.

>
> Performance aside, I could also argue that there's a use case for a 
> minimal (and not degraded) set before expansion.

Sure you could,  when expansion is planned one can create the raid6 set 
with the intended total stripes though.

That aside, this is a constraint for such use cases and we may 
eventually get rid of it.

Regards,
Heinz

>
>
> Cheers,
> Tudor.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/