RE: Two-disk RAID5?

RE: Two-disk RAID5?

[Jansen, Frank]

Tuomas,

It doesn't quite work that way.  IN RAID5 implementations the parity is
distributed across disks in a pattern such as this (3 disk case shown):

  Disk 1   Disk 2   Disk 3
  ------   ------   ------
   Data     Data    Parity
   Data    Parity    Data
  Parity    Data     Data

If you lose a disk (or start in degraded mode) you will get the
following pattern (disk 3 shown missing):

  Disk 1   Disk 2
  ------   ------
   Data     Data
   Data    Parity
  Parity    Data

If data needs to be read that was on disk 3, then the data and parity
from disk 1 and 2 are read and the data is reconstructed from that
information.  A similar strategy is used for writing.  As a result there
is a performance hit for reading/writing data that resides on disk 3.

Note that the loss of 1 more disk renders the RAID 5 set inoperable, as
it will no longer be possible to reconstruct data that was on either of
the missing disks.  Most raid controllers will give up and assume that
you rather treat this as a dead device than take into account the
possibility that you might want some of the data from the remaining
disks.  Thus the real risk of running in degraded mode is that all data
can be lost.

It is not possible to flip a bit to change a set of disks from RAID 1 to
RAID 5, as the physical layout is different.

Cheers,

Frank

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Tuomas Leikola
Sent: Wednesday, April 26, 2006 2:46 PM
To: Jon Lewis
Cc: John Rowe; linux-raid@vger.kernel.org
Subject: Re: Two-disk RAID5?

> No.  When one of the 2 drives in your RAID5 dies, and all you have for
> some blocks is parity info, how will the missing data be
reconstructed?
>
> You could [I suspect] create a 2 disk RAID5 in degraded mode (3rd
member
> missing), but it'll obviously lack redundancy until you add a 3rd
disk,
> which won't add anything to your RAID5 storage capacity.

IMO if you have a 2-disk raid5, the parity for each block is the same
as the data. There is performance drop as I suspect md isn't smart
enough to read data from both disks, but that's all.

When one disk fails, the (lone) parity block is quite enough to
reconstruct. With XOR parity, you can always assume any amount of
additional disks full of zero, it doesn't really change the algorithm.

(maybe mdadm could/can change a raid-1 into raid5 by just changing the
superblocks, for the purpose of expanding into more disks..)

- tuomas
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Two-disk RAID5?

[Jon Lewis]

On Wed, 26 Apr 2006, Jansen, Frank wrote:

> It is not possible to flip a bit to change a set of disks from RAID 1 to
> RAID 5, as the physical layout is different.

As Tuomas pointed out though, a 2 disk RAID5 is kind of a special case 
where all you have is data and parity which is actually also just data. 
Seems kind of like a RAID1 with extra overhead.  I don't think I've ever 
heard of a RAID5 implementation willing to handle <3 drives though.

I suspect I should have just kept out of this, and waited for someone like 
Neil to answer authoratatively.

So...Neil, what's the right answer to Tuomas's 2 disk RAID5 question? :)

----------------------------------------------------------------------
  Jon Lewis                   |  I route
  Senior Network Engineer     |  therefore you are
  Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

RE: Two-disk RAID5?

[Neil Brown]

On Wednesday April 26, jlewis@lewis.org wrote:
> 
> I suspect I should have just kept out of this, and waited for someone like 
> Neil to answer authoratatively.
> 
> So...Neil, what's the right answer to Tuomas's 2 disk RAID5 question? :)
> 

.. and a deep resounding voice from on-high spoke and in it's infinite
wisdom it said....
 
   "yeh, whatever"


The data layout on a 2disk raid5 and a 2 disk raid1 is identical (if
you ignore chunksize issues (raid1 doesn't need one) and the
superblock (which isn't part of the data)).  Each drive contains
identical data(*).

Write throughput to a the r5 would be a bit slower because data is
always copied in memory first, then written.
Read through put would be largely the same if the r5 chunk size was
fairly large, but much poorer for r5 if the chunksize was small.

Converting a raid1 to a raid5 while offline would be quite straight
forward except for the chunksize issue.  If the r1 wasn't a multiple
of the chunksize you chose for r5, then you would lose the last
fraction of a chunk.  So if you are planning to do this, set the size
of your r1 to something that is nice and round (e.g. a multiple of
128k).

Converting a raid1 to a raid5 while online is something I have been
thinking about, but it is not likely to happen any time soon.

I think that answers all the issues.

NeilBrown

(*) The term 'mirror' for raid1 has always bothered me because a
mirror presents a reflected image, while raid1 copies the data without
any transformation.

With a 2drive raid5, one drive gets the original data, and the other
drive gets the data after it has been 'reflected' through an XOR
operation, so maybe a 2drive raid5 is really a 'mirrored' pair....
Except that the data is still the same as XOR with 0 produces no
change.
So, if we made a tiny change to raid5 and got the xor operation to
start with 0xff in every byte, then the XOR would "reflect" each byte
in a reasonable meaningful way, and we might actually get a "mirrored
pair"!!!  

But I don't think that would provide any real value :-)

Re: Two-disk RAID5?

[Al Boldi]

Neil Brown wrote:
> (*) The term 'mirror' for raid1 has always bothered me because a
> mirror presents a reflected image, while raid1 copies the data without
> any transformation.
>
> With a 2drive raid5, one drive gets the original data, and the other
> drive gets the data after it has been 'reflected' through an XOR
> operation, so maybe a 2drive raid5 is really a 'mirrored' pair....
> Except that the data is still the same as XOR with 0 produces no
> change.
> So, if we made a tiny change to raid5 and got the xor operation to
> start with 0xff in every byte, then the XOR would "reflect" each byte
> in a reasonable meaningful way, and we might actually get a "mirrored
> pair"!!!
>
> But I don't think that would provide any real value :-)

Why not?

Consider disks w/ 100mb/s thruput (theoretical):
2disk raid0 stripes data to yield 200mb/s read/write thruput.
2disk raid5 stripes data to yield 100mb/s write, 200mb/s read thruput always.
2disk raid1 mirrors data to yield 100mb/s write, 100/200mb/s single/multiple 
read thruput only.

Would you think that this is enough of a real value?

Thanks!

--
Al

Re: Two-disk RAID5?

[Erik Mouw]

On Wed, Apr 26, 2006 at 03:22:38PM -0400, Jon Lewis wrote:
> On Wed, 26 Apr 2006, Jansen, Frank wrote:
> 
> >It is not possible to flip a bit to change a set of disks from RAID 1 to
> >RAID 5, as the physical layout is different.
> 
> As Tuomas pointed out though, a 2 disk RAID5 is kind of a special case 
> where all you have is data and parity which is actually also just data. 

No, the other way around: RAID1 is a special case of RAID5.

The parity of RAID5 with n disks is contructed like[1]:

  parity = disk1 XOR disk2 XOR ... XOR disk n-1

With n = 2, this reduces to:

  parity = disk1 XOR nothing = disk1

Which is just mirroring, which we usually call "RAID1".

> Seems kind of like a RAID1 with extra overhead.  I don't think I've ever 
> heard of a RAID5 implementation willing to handle <3 drives though.

Our own RAID recovery tools can handle that just fine.


Erik

[1] Yes, there's also an algorithm to select which disk is used for
parity for what block, but that doesn't change the way *how* parity is
calculated.

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: Two-disk RAID5?

[Bill Davidsen]

Erik Mouw wrote:

>On Wed, Apr 26, 2006 at 03:22:38PM -0400, Jon Lewis wrote:
>  
>
>>On Wed, 26 Apr 2006, Jansen, Frank wrote:
>>
>>    
>>
>>>It is not possible to flip a bit to change a set of disks from RAID 1 to
>>>RAID 5, as the physical layout is different.
>>>      
>>>
>>As Tuomas pointed out though, a 2 disk RAID5 is kind of a special case 
>>where all you have is data and parity which is actually also just data. 
>>    
>>
>
>No, the other way around: RAID1 is a special case of RAID5.
>
No it isn't. If you have N drives in RAID1 you have N independent copies 
of the data and no parity, there's just no corresponding thing in RAID5, 
which has one copy of the data, plus parity. There is no special case, 
it just doesn't work that way. Set N>2 and report back.

Sorry, I couldn't find a diplomatic way to say you're completely wrong.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Two-disk RAID5?

[John Rowe]

> Sorry, I couldn't find a diplomatic way to say you're completely wrong.

We don't necessarily expect a diplomatic way, but a clear and
intelligent one would be helpful. 

In two-disk RAID5 which is it?

  1) The 'parity bit' is the same as the datum.

  2) The parity bit is the complement of the datum.

  3) It doesn't work at a bit-wise level.

Many of us feel that RAID5 looks like:

  parity = data[0];
  for (i=1; i < ndisks; ++i)
      parity ^= data[i];

which implies (1). It could easily be (2) but merely saying "it's not
data, it's parity" doesn't clarify matters a great deal. 

But I'm pleased my question has stirred up such controversy!

John

Re: Two-disk RAID5?

[Neil Brown]

On Friday May 5, rowe@excc.ex.ac.uk wrote:
> 
> > Sorry, I couldn't find a diplomatic way to say you're completely wrong.
> 
> We don't necessarily expect a diplomatic way, but a clear and
> intelligent one would be helpful. 
> 
> In two-disk RAID5 which is it?
> 
>   1) The 'parity bit' is the same as the datum.

Yes.

> 
>   2) The parity bit is the complement of the datum.

No.

> 
>   3) It doesn't work at a bit-wise level.

No.

> 
> Many of us feel that RAID5 looks like:
> 
>   parity = data[0];
>   for (i=1; i < ndisks; ++i)
>       parity ^= data[i];

Actually in linux/md/raid5 it is more like

    parity = 0
    for (i=0; i < ndisks; ++i)
	parity ^= data[i];

which has exactly the same result.
(well, it should really be ndatadisks, but I think we both knew that
was what you meant).

> 
> which implies (1). It could easily be (2) but merely saying "it's not
> data, it's parity" doesn't clarify matters a great deal. 
> 
> But I'm pleased my question has stirred up such controversy!

A bit of controversy is always a nice way to pass those long winter
nights.... only it isn't winter anywhere at the moment :-)

NeilBrown

Re: Two-disk RAID5?

[Erik Mouw]

On Thu, May 04, 2006 at 09:18:57PM -0400, Bill Davidsen wrote:
> Erik Mouw wrote:
> >No, the other way around: RAID1 is a special case of RAID5.
> >
> No it isn't. If you have N drives in RAID1 you have N independent copies 
> of the data and no parity, there's just no corresponding thing in RAID5, 
> which has one copy of the data, plus parity. There is no special case, 
> it just doesn't work that way. Set N>2 and report back.

Just write out the formulas and it becomes obvious.

> Sorry, I couldn't find a diplomatic way to say you're completely wrong.

I guess we have to agree to disagree.


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

Re: Two-disk RAID5?

[Bill Davidsen]

John Rowe wrote:

>I'm about to create a RAID1 file system and a strange thought occurs to
>me: if I create a two-disk RAID5 array then I can grow it later by the
>simple expedient of adding a third disk and hence doubling its size.
>
>Is there any real down-side to this, such as performance? Alternatively
>is it likely that mdadm will soon be able to convert a RAID1 pair to
>RAID5 any time soon? (Just how different are they anyway? Isn't the
>RAID4/5 checksum just an OR?)
>
I think it works, I just set up a little test case with two 20MB files 
and loopback mount. The mdadm seems to work, the mke2fs seems to work, 
the f/s is there. Please verify, this system is a bit (okay a bunch) hacked.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Two-disk RAID5?

[Tuomas Leikola]

> No.  When one of the 2 drives in your RAID5 dies, and all you have for
> some blocks is parity info, how will the missing data be reconstructed?
>
> You could [I suspect] create a 2 disk RAID5 in degraded mode (3rd member
> missing), but it'll obviously lack redundancy until you add a 3rd disk,
> which won't add anything to your RAID5 storage capacity.

IMO if you have a 2-disk raid5, the parity for each block is the same
as the data. There is performance drop as I suspect md isn't smart
enough to read data from both disks, but that's all.

When one disk fails, the (lone) parity block is quite enough to
reconstruct. With XOR parity, you can always assume any amount of
additional disks full of zero, it doesn't really change the algorithm.

(maybe mdadm could/can change a raid-1 into raid5 by just changing the
superblocks, for the purpose of expanding into more disks..)

- tuomas

Re: Two-disk RAID5?

[Jon Lewis]

On Wed, 26 Apr 2006, John Rowe wrote:

> I'm about to create a RAID1 file system and a strange thought occurs to
> me: if I create a two-disk RAID5 array then I can grow it later by the
> simple expedient of adding a third disk and hence doubling its size.

No.  When one of the 2 drives in your RAID5 dies, and all you have for 
some blocks is parity info, how will the missing data be reconstructed?

You could [I suspect] create a 2 disk RAID5 in degraded mode (3rd member 
missing), but it'll obviously lack redundancy until you add a 3rd disk, 
which won't add anything to your RAID5 storage capacity.

----------------------------------------------------------------------
  Jon Lewis                   |  I route
  Senior Network Engineer     |  therefore you are
  Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Two-disk RAID5?

[John Rowe]

I'm about to create a RAID1 file system and a strange thought occurs to
me: if I create a two-disk RAID5 array then I can grow it later by the
simple expedient of adding a third disk and hence doubling its size.

Is there any real down-side to this, such as performance? Alternatively
is it likely that mdadm will soon be able to convert a RAID1 pair to
RAID5 any time soon? (Just how different are they anyway? Isn't the
RAID4/5 checksum just an OR?)

Thanks

John