debian software raid1

debian software raid1

[b2]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

Hi , list 
i have two identical 500GB harddrives - sda & sdb. sda has already
installed debian with root and other partitions and data on it and sdb
had no partiotions and data on it (it is brand new). my question is :
can i setup software raid 1 on two harddisks without loosing any data
from sda ? just to create raid array and somehow copy partitions and
data to it ? 
sorry for my lame question , just don't know how (if possible) to do it.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: debian software raid1

[Mathias Burén]

On 19 April 2011 13:12, b2 <b2@playtime.bg> wrote:
> Hi , list
> i have two identical 500GB harddrives - sda & sdb. sda has already
> installed debian with root and other partitions and data on it and sdb
> had no partiotions and data on it (it is brand new). my question is :
> can i setup software raid 1 on two harddisks without loosing any data
> from sda ? just to create raid array and somehow copy partitions and
> data to it ?
> sorry for my lame question , just don't know how (if possible) to do it.
>

http://www.debian-administration.org/articles/238

Plenty of articles on the net on how to do that.

Regards,
M

Re: debian software raid1

[Iordan Iordanov]

Hey guys,

On 04/19/11 08:25, Mathias Burén wrote:
> http://www.debian-administration.org/articles/238
>
> Plenty of articles on the net on how to do that.

There is a lot of information, but also, I suspect there is a lot of 
disagreement. One thing that I could not find a definitive answer on was 
the question of whether or not to mirror swap.

There are articles that propose mirroring partitions independently 
(rather than the entire disk), and not mirroring swap, but adding two 
swap partitions with equal priority. On the other hand, there are people 
who point out that in the event where one of the disks in the mirror 
dies, the machine may cease to function, because a part of its "memory" 
will have disappeared. However, making swap part of the mirror opens a 
whole new can of worms. For instance, could there be a deadlock 
possibility (for example after a suspend/resume cycle) where mdadm is 
waiting for something which is swapped out onto swap which is mirrored?

It would be nice to have a discussion among people who have experience 
with all of this.

Cheers,
Iordan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: debian software raid1

[Gordon Henderson]

On Tue, 19 Apr 2011, Iordan Iordanov wrote:

> There is a lot of information, but also, I suspect there is a lot of 
> disagreement. One thing that I could not find a definitive answer on was the 
> question of whether or not to mirror swap.
>
> There are articles that propose mirroring partitions independently (rather 
> than the entire disk), and not mirroring swap, but adding two swap partitions 
> with equal priority. On the other hand, there are people who point out that 
> in the event where one of the disks in the mirror dies, the machine may cease 
> to function, because a part of its "memory" will have disappeared. However, 
> making swap part of the mirror opens a whole new can of worms. For instance, 
> could there be a deadlock possibility (for example after a suspend/resume 
> cycle) where mdadm is waiting for something which is swapped out onto swap 
> which is mirrored?
>
> It would be nice to have a discussion among people who have experience with 
> all of this.

I've not been active here for a long time - mostly because as far as I'm 
concerned Linux-Raid "just works" and it's as stable as I need it to be, 
however this subject caught my eye, so I'll add in my 2 penneth 
worth...

I've been putting swap on mirrored, and raided drives for as long as I've 
been using the MD stuff, and that's now well over 10 years now.

I do recall that there were such deadlock issues in the early days, but 
I'm also sure these were resolved very quickly indeed. MD has certianly 
saved my bacon when a disk has failled - and that includes swap 
partitions.

An example - swap on a 5-drive RAID-6 system:

   cat /proc/swaps
   Filename                                Type            Size    Used    Priority
   /dev/md2                                partition       2987704 1720    -1

and:

   md2 : active raid6 sde2[4] sdd2[3] sdc2[2] sdb2[1] sda2[0]
       2987712 blocks level 6, 64k chunk, algorithm 2 [5/5] [UUUUU]

another: (RAID1 - mirrored)

   cat /proc/swaps
   Filename                                Type            Size    Used    Priority
   /dev/md2                                partition       2007992 0       -1

and:

   md2 : active raid1 sdb2[0] sda2[1]
       2008000 blocks [2/2] [UU]

however just because I do it, doesn't mean it's perfect - but I can say 
that I've never had a server panic or otherwise crash due to me having 
swap on MD - at least not that I've been aware of. (but also note that 
I've never used MD on a system that I'd ever suspend or hibernate, but I 
have used MD with drive spin-down software with no ill effects!)

I'm actually of the opinion that to NOT put swap on RAID in an otherwise 
RAIDed system is just plain silly. Do it. You know it makes sense!

Gordon

Re: debian software raid1

[Steven Haigh]

On 20/04/2011 7:10 AM, Gordon Henderson wrote:
> I'm actually of the opinion that to NOT put swap on RAID in an otherwise
> RAIDed system is just plain silly. Do it. You know it makes sense!

I think the preferred method at the moment (and at least for 2010/2011) 
is to have a smallish RAID1 partition for /boot then create a second 
RAID array for everything else and use LVM on top of that.

That allows you to have the host on the same RAID1 and use the 
flexibility to use LVM for all your other requirements.

I personally run a Xen virtualisation server - it has the following:
/dev/md0 - /boot - 512Mb - RAID1
/dev/md1 - VG 'RAID1' - rest of disk - RAID1
/dev/md2 - VG 'RAID5' - 3 x 1Tb whole disks - RAID5

Then the majority of the 7 VMs on that machine live on the RAID5 VG 
(with one special case that is on the RAID1 VG) and it runs very happily.

The swap and root of Dom0 is on the RAID1 VG. This seems to be the best 
way I've found to have the most flexibility and not have to change 
things as time progresses.

--
Steven Haigh

Email: netwiz@crc.id.au
Web: http://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
Fax: (03) 8338 0299

There are no great men, only great challenges that ordinary men are 
forced by circumstances to meet. -- Admiral William Halsey

Re: debian software raid1

[Roberto Spadim]

hum, some doubts....
what´s better (faster / secure)?
swap over one disk (no raid)
swap over one md raid1
swap over one md raid0
swap over one md raid10
swap over one md raid5
swap over one md raid6



2011/4/19 Steven Haigh <netwiz@crc.id.au>:
> On 20/04/2011 7:10 AM, Gordon Henderson wrote:
>>
>> I'm actually of the opinion that to NOT put swap on RAID in an otherwise
>> RAIDed system is just plain silly. Do it. You know it makes sense!
>
> I think the preferred method at the moment (and at least for 2010/2011) is
> to have a smallish RAID1 partition for /boot then create a second RAID array
> for everything else and use LVM on top of that.
>
> That allows you to have the host on the same RAID1 and use the flexibility
> to use LVM for all your other requirements.
>
> I personally run a Xen virtualisation server - it has the following:
> /dev/md0 - /boot - 512Mb - RAID1
> /dev/md1 - VG 'RAID1' - rest of disk - RAID1
> /dev/md2 - VG 'RAID5' - 3 x 1Tb whole disks - RAID5
>
> Then the majority of the 7 VMs on that machine live on the RAID5 VG (with
> one special case that is on the RAID1 VG) and it runs very happily.
>
> The swap and root of Dom0 is on the RAID1 VG. This seems to be the best way
> I've found to have the most flexibility and not have to change things as
> time progresses.
>
> --
> Steven Haigh
>
> Email: netwiz@crc.id.au
> Web: http://www.crc.id.au
> Phone: (03) 9001 6090 - 0412 935 897
> Fax: (03) 8338 0299
>
> There are no great men, only great challenges that ordinary men are forced
> by circumstances to meet. -- Admiral William Halsey
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: debian software raid1

[Mathias Burén]

On 19 April 2011 23:01, Roberto Spadim <roberto@spadim.com.br> wrote:
> hum, some doubts....
> what´s better (faster / secure)?
> swap over one disk (no raid)
> swap over one md raid1
> swap over one md raid0
> swap over one md raid10
> swap over one md raid5
> swap over one md raid6
>
>
>
> 2011/4/19 Steven Haigh <netwiz@crc.id.au>:
>> On 20/04/2011 7:10 AM, Gordon Henderson wrote:
>>>
>>> I'm actually of the opinion that to NOT put swap on RAID in an otherwise
>>> RAIDed system is just plain silly. Do it. You know it makes sense!
>>
>> I think the preferred method at the moment (and at least for 2010/2011) is
>> to have a smallish RAID1 partition for /boot then create a second RAID array
>> for everything else and use LVM on top of that.
>>
>> That allows you to have the host on the same RAID1 and use the flexibility
>> to use LVM for all your other requirements.
>>
>> I personally run a Xen virtualisation server - it has the following:
>> /dev/md0 - /boot - 512Mb - RAID1
>> /dev/md1 - VG 'RAID1' - rest of disk - RAID1
>> /dev/md2 - VG 'RAID5' - 3 x 1Tb whole disks - RAID5
>>
>> Then the majority of the 7 VMs on that machine live on the RAID5 VG (with
>> one special case that is on the RAID1 VG) and it runs very happily.
>>
>> The swap and root of Dom0 is on the RAID1 VG. This seems to be the best way
>> I've found to have the most flexibility and not have to change things as
>> time progresses.
>>
>> --
>> Steven Haigh
>>
>> Email: netwiz@crc.id.au
>> Web: http://www.crc.id.au
>> Phone: (03) 9001 6090 - 0412 935 897
>> Fax: (03) 8338 0299
>>
>> There are no great men, only great challenges that ordinary men are forced
>> by circumstances to meet. -- Admiral William Halsey
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

You can't say "faster/secure", it's mostly either one. Swap over raid0
or some raid10 implementation would probably be the fastest. Swap on
raid456 would be incredibly slow write wise...
// M
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: debian software raid1

[NeilBrown]

On Tue, 19 Apr 2011 12:03:27 -0400 Iordan Iordanov <iordan@cdf.toronto.edu>
wrote:

> Hey guys,
> 
> On 04/19/11 08:25, Mathias Burén wrote:
> > http://www.debian-administration.org/articles/238
> >
> > Plenty of articles on the net on how to do that.
> 
> There is a lot of information, but also, I suspect there is a lot of 
> disagreement. One thing that I could not find a definitive answer on was 
> the question of whether or not to mirror swap.
> 
> There are articles that propose mirroring partitions independently 
> (rather than the entire disk), and not mirroring swap, but adding two 
> swap partitions with equal priority. On the other hand, there are people 
> who point out that in the event where one of the disks in the mirror 
> dies, the machine may cease to function, because a part of its "memory" 
> will have disappeared. However, making swap part of the mirror opens a 
> whole new can of worms. For instance, could there be a deadlock 
> possibility (for example after a suspend/resume cycle) where mdadm is 
> waiting for something which is swapped out onto swap which is mirrored?

If there was such a deadlock, it would be a serious bug.   I don't believe
such a bug exists (but hey - I keep finding bugs in this code, when I'm not
busy writing new bugs, so I guess it could crash you machine and kill your
cat).

> 
> It would be nice to have a discussion among people who have experience 
> with all of this.

Definitely put swap on RAID1 if you have RAID1 at all.

My personal preference with RAID1 to to have a single RAID1 (probably
--metadata=1.0) across the whole devices, and partition that for root, swap,
home.

Others prefer to partition the devices and run RAID1 across the partitions
independently.  While that is not my preference I known nothing against it.
This is the price of freedom - you have to make choices

If I had a few identical drives, I would partition them each into a
small /boot, a small swap and the rest.  I would then RAID1 all the /boots
together, RAID10 all the swaps, and RAID5 or RAID6 all the rest.

The most important thing though it to create an configuration that you
understand and are comfortable with, because you are the one who will have to
manage it.

NeilBrown


> 
> Cheers,
> Iordan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: debian software raid1

[Joe Landman]

On 04/19/2011 06:51 PM, NeilBrown wrote:
> On Tue, 19 Apr 2011 12:03:27 -0400 Iordan Iordanov<iordan@cdf.toronto.edu>

[...]

> If there was such a deadlock, it would be a serious bug.   I don't believe
> such a bug exists (but hey - I keep finding bugs in this code, when I'm not
> busy writing new bugs, so I guess it could crash you machine and kill your
> cat).

I ran into this a few years ago (early 2.6 series, maybe 2.6.15 or so). 
  Some sort of race condition.  Required a reset to get the machines 
attention.  Haven't tried since.  Since we do lots of MD raid, I 
generally either put the swap on hardware RAID, or stripe it and 
sometimes hope for the best.  The latter doesn't always work.  When a 
drive goes pear shaped, and takes down pages with it, your machine will 
crash, and pretty hard at that.

We'll definitely try the swap atop MD RAID going forward.  I like the 
idea of a RAID10 for it.

>> It would be nice to have a discussion among people who have experience
>> with all of this.
>
> Definitely put swap on RAID1 if you have RAID1 at all.
>
> My personal preference with RAID1 to to have a single RAID1 (probably
> --metadata=1.0) across the whole devices, and partition that for root, swap,
> home.

This gets to my question, since we haven't tried this yet ... can we do 
whole device MD RAIDs, and boot from them?  Is there anything special we 
need to do in their construction (use --auto=mdp or similar)?  We'd 
really prefer to go this route rather than building multiple smaller 
RAIDs.  We can manage the latter, but the former is far easier/cleaner 
to handle rebuilds with.

Regards,

Joe


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman@scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

Re: debian software raid1

[NeilBrown]

On Tue, 19 Apr 2011 20:33:46 -0400 Joe Landman <joe.landman@gmail.com> wrote:

> On 04/19/2011 06:51 PM, NeilBrown wrote:
> > On Tue, 19 Apr 2011 12:03:27 -0400 Iordan Iordanov<iordan@cdf.toronto.edu>
> 
> [...]
> 
> > If there was such a deadlock, it would be a serious bug.   I don't believe
> > such a bug exists (but hey - I keep finding bugs in this code, when I'm not
> > busy writing new bugs, so I guess it could crash you machine and kill your
> > cat).
> 
> I ran into this a few years ago (early 2.6 series, maybe 2.6.15 or so). 
>   Some sort of race condition.  Required a reset to get the machines 
> attention.  

I wonder what this might have been...
Resume from suspend when swap in on md/raid can be a bit tricky and may not
have been well tested at that time.  You need to be able to assemble the
RAID1 for read without writing anything.  That can be done now by setting the
'start_ro' module parameter (echo 1
> /sys/modules/md_mod/parameters/start_ro).

I haven't actually tested it myself though so I cannot speak from experience.

>               Haven't tried since.  Since we do lots of MD raid, I 
> generally either put the swap on hardware RAID, or stripe it and 
> sometimes hope for the best.  The latter doesn't always work.  When a 
> drive goes pear shaped, and takes down pages with it, your machine will 
> crash, and pretty hard at that.
> 
> We'll definitely try the swap atop MD RAID going forward.  I like the 
> idea of a RAID10 for it.
> 
> >> It would be nice to have a discussion among people who have experience
> >> with all of this.
> >
> > Definitely put swap on RAID1 if you have RAID1 at all.
> >
> > My personal preference with RAID1 to to have a single RAID1 (probably
> > --metadata=1.0) across the whole devices, and partition that for root, swap,
> > home.
> 
> This gets to my question, since we haven't tried this yet ... can we do 
> whole device MD RAIDs, and boot from them?  Is there anything special we 
> need to do in their construction (use --auto=mdp or similar)?  We'd 
> really prefer to go this route rather than building multiple smaller 
> RAIDs.  We can manage the latter, but the former is far easier/cleaner 
> to handle rebuilds with.

The little 'server' in my garage runs md/raid1 across two device (using 0.90
metadata - never had any reason to convert).

I seem to remember that I had to do something special to grub to get it to
boot initially, but I also vaguely remember that after some series of updates
(this is Debian) it seemed to get a lot easier.  My "grub.cfg" contains:

menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-686' --class debian --class gnu-linux --class gnu --class os {
        insmod raid
        insmod mdraid
        insmod part_msdos
        insmod ext2
        set root='(md0,1)'
        search --no-floppy --fs-uuid --set 03e9951f-d174-4d88-9b4c-dd7a31c0de61
        echo    'Loading Linux 2.6.32-5-686 ...'
        linux   /vmlinuz-2.6.32-5-686 root=UUID=6e8a3564-5588-476e-bbe5-a68c967c6c8e ro  quiet
        echo    'Loading initial ramdisk ...'
        initrd  /initrd.img-2.6.32-5-686
}

which was all automatically created, and it seems to "just work".

So yes - you can boot for partitioned whole-device md/raid.
You might need to use a "metadata-at-the-end" format list 0.90 or 1.0, but it
is entirely possible that it will work for other metadata formats, I really
don't know.  Other than that, no special options needed when creating the
array.

NeilBrown

Re: debian software raid1

[Asdo]

On 04/20/11 00:51, NeilBrown wrote:
> If there was such a deadlock, it would be a serious bug.   I don't believe
> such a bug exists (but hey - I keep finding bugs in this code, when I'm not
> busy writing new bugs, so I guess it could crash you machine and kill your
> cat).
>

I am aware that raids 0,1,10 do not copy the buffer so I don't think 
there is risk of deadlock...

Raids 456 do copy the buffer... but the destination buffers are 
preallocated, right?

Thank you

Re: debian software raid1

[NeilBrown]

On Wed, 20 Apr 2011 12:28:45 +0200 Asdo <asdo@shiftmail.org> wrote:

> On 04/20/11 00:51, NeilBrown wrote:
> > If there was such a deadlock, it would be a serious bug.   I don't believe
> > such a bug exists (but hey - I keep finding bugs in this code, when I'm not
> > busy writing new bugs, so I guess it could crash you machine and kill your
> > cat).
> >
> 
> I am aware that raids 0,1,10 do not copy the buffer so I don't think 
> there is risk of deadlock...
> 
> Raids 456 do copy the buffer... but the destination buffers are 
> preallocated, right?

Correct - destination buffers - when used - are preallocated.

If this were not the case you would get deadlocks in normal usage without
having to wait for suspend/resume to trigger them.

NeilBrown

Re: debian software raid1

[Iordan Iordanov]

Hi Joe,

On 04/19/11 20:33, Joe Landman wrote:
> This gets to my question, since we haven't tried this yet ... can we do
> whole device MD RAIDs, and boot from them? Is there anything special we
> need to do in their construction (use --auto=mdp or similar)? We'd
> really prefer to go this route rather than building multiple smaller
> RAIDs. We can manage the latter, but the former is far easier/cleaner to
> handle rebuilds with.

A few days ago (just before this thread started, coincidentally), I 
converted a single drive system to RAID1 by adding a second disk, and 
raiding the whole drive. This does have the drawback that any 
replacement drives have to be the same size or bigger than what you had 
to begin with. I am running Ubuntu 10.04.

A few things I had to make sure, and I disclaim that this is not an 
exhaustive list. I created the array and named it /dev/md0. Then, I had 
to do:

1) mdadm --examine --scan > /etc/mdadm/mdadm.conf
2) Rebuild initrd, so that upon boot the array will be assembled 
automatically based on this mdadm.conf. On Debian/Ubuntu, this can be 
done with update-initramfs.

When you partition /dev/md0 with whatever partitioner you prefer, the 
kernel does see the partitions, and makes /dev/md0pN for partition 
number N. Hence you can use these to mount/start swap.

For the record, I have been suspending/resuming that Ubuntu 10.04 with 
swap living on one of the RAID1 partitions with no ill effect, and will 
revisit this thread if I ever do experience issues!

Also, I was thinking of switching from RAID1 to RAID10 with near layout 
and two copies (effectively RAID1) in order to get faster reads - 
anybody have any opinion on this?

Cheers everyone, and thanks for all the discussion!
Iordan

Re: debian software raid1

[Iordan Iordanov]

Hi Neil,

On 04/19/11 21:12, NeilBrown wrote:
> I seem to remember that I had to do something special to grub to get it to
> boot initially, but I also vaguely remember that after some series of updates
> (this is Debian) it seemed to get a lot easier.  My "grub.cfg" contains:
>
> menuentry 'Debian GNU/Linux, with Linux 2.6.32-5-686' --class debian --class gnu-linux --class gnu --class os {
>          insmod raid
>          insmod mdraid
>          insmod part_msdos
>          insmod ext2
>          set root='(md0,1)'
>          search --no-floppy --fs-uuid --set 03e9951f-d174-4d88-9b4c-dd7a31c0de61
>          echo    'Loading Linux 2.6.32-5-686 ...'
>          linux   /vmlinuz-2.6.32-5-686 root=UUID=6e8a3564-5588-476e-bbe5-a68c967c6c8e ro  quiet
>          echo    'Loading initial ramdisk ...'
>          initrd  /initrd.img-2.6.32-5-686
> }
>
> which was all automatically created, and it seems to "just work".

Yes, I am also reminded now that grub and md was a bit strange, mostly 
because I was converting a single drive system to a RAID1 system, and I 
had dd-d /dev/sda onto /dev/sdb. Grub seemed to struggle with the fact 
that the UUID was the same on /dev/sda1, /dev/sdb1, and naturally 
/dev/md0p1, which was initially created on top of /dev/sda. It refused 
to boot a couple of times complaining about the UUID, which was easily 
resolved by editing the grub entry on the fly in the grub prompt, and 
changing root=UUID=6e8a3564-5588-476e-bbe5-a68c967c6c8e to 
root=/dev/md0p1 (note that the UUID is just an example UUID), and 
booting that way. Afterward, doing:

grub-install --modules="raid" /dev/md0

seemed to eventually placate grub. Also, I did not really see any change 
in grub.cfg, but then again, I didn't actually do a "diff" of grub.cfg 
before and after. The point is that it still uses UUID, the UUID is 
still the same among all these devices (naturally, since this is RAID1), 
and it *no longer complains*. It's a bit disconcerting, but I will learn 
to live with it :D.

I hope this helps!
Iordan

Re: debian software raid1

[Luca Berra]

On Wed, Apr 20, 2011 at 10:51:29AM -0400, Iordan Iordanov wrote:
> raiding the whole drive. This does have the drawback that any replacement 
> drives have to be the same size or bigger than what you had to begin with. 

I am missing your point here.
The requirement of having a replacement drive at least as big as the
original one, usually comes from the fact that a smaller drive won't be
able to contain your data, not from some md voodoo.

If someone here still believes that drives from different vendors with
the same nominal size have different real size please read:
http://www.idema.org/wp-content/plugins/download-monitor/download.php?id=1192


-- 
Luca Berra -- bluca@comedia.it

Re: debian software raid1

[Iordan Iordanov]

Hi Luca,

On 04/21/11 02:15, Luca Berra wrote:
> If someone here still believes that drives from different vendors with
> the same nominal size have different real size please read:
> http://www.idema.org/wp-content/plugins/download-monitor/download.php?id=1192

I don't think my point is invalid. If you wanted to mix SSDs and spindle 
HDDs (for, say, write-mostly "backup" of the SSDs), quite likely your 
drives will be of different sizes, as the SSDs frequently come in sizes 
that are power of two's unlike the spindle drives. You could start of 
with just one type of drive, and only then come up with the idea of 
mixing types, etc. Probably not the best example, but one nonetheless.

There are other benefits to using a partition instead of the whole 
device. For example, you could use a GPT label, which allows you to 
label individual partitions. In an earlier post, I sent out a gpt_id 
script and UDEV rules which insert /dev/disk/by-label/ links for you to 
GPT-labeled partitions. We use it at the CS department of U of Toronto 
to keep track of the physical location of disks (within iscsi target hosts).

I think that if you are not interested in the 2nd point here, you could 
simply use the --size option to mdadm to use less than the total 
capacity of the drives, but apparently it doesn't work for all RAID 
levels (RAID 10 is not listed in Debian Lenny's man-page). Somebody 
correct me if I am wrong about this.

Cheers!
Iordan

Re: debian software raid1

[Luca Berra]

On Thu, Apr 21, 2011 at 10:50:28AM -0400, Iordan Iordanov wrote:
> Hi Luca,
>
> On 04/21/11 02:15, Luca Berra wrote:
>> If someone here still believes that drives from different vendors with
>> the same nominal size have different real size please read:
>> http://www.idema.org/wp-content/plugins/download-monitor/download.php?id=1192
>
> I don't think my point is invalid. If you wanted to mix SSDs and spindle 
> HDDs (for, say, write-mostly "backup" of the SSDs), quite likely your 
> drives will be of different sizes, as the SSDs frequently come in sizes 
> that are power of two's unlike the spindle drives. You could start of with 
> just one type of drive, and only then come up with the idea of mixing 
> types, etc. Probably not the best example, but one nonetheless.
no, not the best example :D

> There are other benefits to using a partition instead of the whole device. 
there are both benefits and disadvantages, both lists are long enough
without adding non-issues.

> For example, you could use a GPT label, which allows you to label 
> individual partitions. In an earlier post, I sent out a gpt_id script and 
> UDEV rules which insert /dev/disk/by-label/ links for you to GPT-labeled 
> partitions. We use it at the CS department of U of Toronto to keep track of 
> the physical location of disks (within iscsi target hosts).
So, if i understood correctly, you use partitions on something which is
not a real HDD, hope you got the data alignment part correct.

L.
-- 
Luca Berra -- bluca@comedia.it

Re: debian software raid1

[Iordan Iordanov]

On 04/22/11 01:59, Luca Berra wrote:
> no, not the best example :D

I don't know why I'm even bothering to reply, given your all-knowing 
tone, but here it is anyway. Today, I was converting a system from 
single drive to mirrored with a 2nd 80GB drive. Here is proof that there 
are drives that both claim to be 80GB, but happen to be of different 
capacities:

sd 0:0:0:0: [sda] 156301488 512-byte logical blocks: (80.0 GB/74.5 GiB)
sd 2:0:0:0: [sdb] 156250000 512-byte logical blocks: (80.0 GB/74.5 GiB)

 > So, if i understood correctly, you use partitions on something which 
 > isnot a real HDD, hope you got the data alignment part correct.

You have misunderstood, and yes we are using partitions on real drives, 
and indeed yes, we've got the alignment part correct.

Cheers,
Iordan

Re: debian software raid1

[Iordan Iordanov]

Hello again,

On 04/19/11 08:12, b2 wrote:
> sorry for my lame question , just don't know how (if possible) to do it.

After the 3rd system I had to do this on in the last 3 days, I decided 
to write a guide myself :). I hope it helps!

http://iiordanov.blogspot.com/2011/04/how-to-convert-your-single-drive-linux.html

Cheers,
Iordan

Re: debian software raid1

[Mikael Abrahamsson]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 706 bytes --]

On Fri, 22 Apr 2011, Iordan Iordanov wrote:

> I don't know why I'm even bothering to reply, given your all-knowing 
> tone, but here it is anyway. Today, I was converting a system from 
> single drive to mirrored with a 2nd 80GB drive. Here is proof that there 
> are drives that both claim to be 80GB, but happen to be of different 
> capacities:

"At that time, 3.5” 160GB have been shipping for a while so many
system manufacturers have already defined their own LBA counts 
requirement. Most system
manufacturers began to adopt the LBA1-02 implementation at or after 3.5” 
320GB density point."

The document never claims to be about 80GB drives.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: debian software raid1

[NeilBrown]

On Fri, 22 Apr 2011 15:21:39 -0400 Iordan Iordanov <iordan@cdf.toronto.edu>
wrote:

> Hello again,
> 
> On 04/19/11 08:12, b2 wrote:
> > sorry for my lame question , just don't know how (if possible) to do it.
> 
> After the 3rd system I had to do this on in the last 3 days, I decided 
> to write a guide myself :). I hope it helps!
> 
> http://iiordanov.blogspot.com/2011/04/how-to-convert-your-single-drive-linux.html



> Create the array. I decided to make a RAID10 array instead of a RAID1 array, 
> which gives me faster sequential reads, but amounts to the same thing in terms
> of data replication. RAID10 with two drives and "n2"

This is not correct.  RAID10-n2 on 2 drives is exactly the same layout and
very nearly the same speed as RAID1 on 2 drives.  (I say 'very nearly' only
because the read-balancing code is a little different and might have slightly
different results).

Or have you measured these two and found an actually difference?  That would
certainly be interesting.

RAID10-f2 will give faster sequential reads at the cost of slower writes.

NeilBrown



> 
> Cheers,
> Iordan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: debian software raid1

[Iordan Iordanov]

Hi Neil,

On 04/22/11 18:12, NeilBrown wrote:
> This is not correct.  RAID10-n2 on 2 drives is exactly the same layout and
> very nearly the same speed as RAID1 on 2 drives.  (I say 'very nearly' only
> because the read-balancing code is a little different and might have slightly
> different results).
>
> Or have you measured these two and found an actually difference?  That would
> certainly be interesting.

The difference that I see is probably 100% due to the different read 
balancing algorithm. When I start two dd processes reading from two 
separate partitions on the RAID (just so there are no buffers screwing 
up my results), with RAID1, I see less than one drive worth of 
sequential read speed for the two dd processes combined.

On the other hand, with RAID10 I see the two drives being utilized 
fully, and I get one drive worth of sequential read speeds for each dd 
process, or a total of two drives worth of read speed for the two dd 
processes.

The numbers were something like this:

- Single drive speed: ~130MB/s sequential read.
- Two simultaneous dd sequential reads with RAID1, bs=1024k: ~40MB/s per dd.
- Two simultaneous dd sequential reads with RAID10, bs=1024k: ~130MB/s 
per dd.

That's what I meant by better sequential reads, but perhaps I should try 
to phrase it more precisely.

> RAID10-f2 will give faster sequential reads at the cost of slower writes.

I am not sure what RAID10-f2 on a two disk setup will look like, but I 
like the idea of the drives being identical, and in the worst case, 
being able to pull one drive, zero the superblock, and be left with a 
drive with intact data, which only RAID10-n2 can give, if I am not mistaken.

Just to follow up on our discussion on Grub v2 and booting from a RAID 
device. I discovered that if I allow Grub to use UUID, occasionally, it 
would try to mount a raw device for root instead of the RAID device. 
Apart from the nuisance, this would probably cause mismatch_cnt to be 
non-zero!! (heh heh). At any rate, the guide reflects how I deal with 
that - by turning off the use of UUIDs.

Many thanks for taking a look at the guide and sharing your thoughts! 
Please let me know if you still think I should change that part where I 
say that RAID10 gives me faster sequential reads, and what you would say 
instead.

Iordan

Re: debian software raid1

[Iordan Iordanov]

Hi Mikael,

On 04/22/11 15:28, Mikael Abrahamsson wrote:
> The document never claims to be about 80GB drives.

But the document was proffered as proof that drives from different 
manufacturers have the same capacity, whereas I find this to not be the 
case for smaller drives. I never claimed the document is for 80GB drives.

Cheers,
Iordan

Re: debian software raid1

[Jan Ceuleers]

On 19/04/11 18:03, Iordan Iordanov wrote:
> There are articles that propose mirroring partitions independently
> (rather than the entire disk), and not mirroring swap, but adding two
> swap partitions with equal priority.

Adding two swap partitions with equal priority is swap's way of 
implementing striping rather than mirroring.

So it depends what you're after: if you want as much swap space as 
possible then don't use md for swap and just create several swap 
partitions with equal priority. If on the other hand you're after 
robustness, in particular including the ability for the server to 
survive disk failure, then you will need to add a RAID layer.

You can of course do both. In a previous machine I had a 4-disk RAID5. 
On each of the 4 disks I had also set aside some space, which I 
organised into two 2-partition RAID0 devices, which I then used as swap 
devices with equal priority.

I have not experienced any deadlock issues with this setup.

Jan

Re: debian software raid1

[David Brown]

On 23/04/11 02:05, Iordan Iordanov wrote:
> Hi Neil,
>
> On 04/22/11 18:12, NeilBrown wrote:
>> This is not correct. RAID10-n2 on 2 drives is exactly the same layout and
>> very nearly the same speed as RAID1 on 2 drives. (I say 'very nearly'
>> only
>> because the read-balancing code is a little different and might have
>> slightly
>> different results).
>>
>> Or have you measured these two and found an actually difference? That
>> would
>> certainly be interesting.
>
> The difference that I see is probably 100% due to the different read
> balancing algorithm. When I start two dd processes reading from two
> separate partitions on the RAID (just so there are no buffers screwing
> up my results), with RAID1, I see less than one drive worth of
> sequential read speed for the two dd processes combined.
>
> On the other hand, with RAID10 I see the two drives being utilized
> fully, and I get one drive worth of sequential read speeds for each dd
> process, or a total of two drives worth of read speed for the two dd
> processes.
>
> The numbers were something like this:
>
> - Single drive speed: ~130MB/s sequential read.
> - Two simultaneous dd sequential reads with RAID1, bs=1024k: ~40MB/s per
> dd.
> - Two simultaneous dd sequential reads with RAID10, bs=1024k: ~130MB/s
> per dd.
>
> That's what I meant by better sequential reads, but perhaps I should try
> to phrase it more precisely.
>
>> RAID10-f2 will give faster sequential reads at the cost of slower writes.
>
> I am not sure what RAID10-f2 on a two disk setup will look like, but I
> like the idea of the drives being identical, and in the worst case,
> being able to pull one drive, zero the superblock, and be left with a
> drive with intact data, which only RAID10-n2 can give, if I am not
> mistaken.
>

Look at this to see some pictures of raid10 layouts:
<http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10>

Raid10,far will fair worse than raid10,near when degraded.  But it will 
still work - your data is mirrored, and you can pull a drive without 
losing anything.

> Just to follow up on our discussion on Grub v2 and booting from a RAID
> device. I discovered that if I allow Grub to use UUID, occasionally, it
> would try to mount a raw device for root instead of the RAID device.
> Apart from the nuisance, this would probably cause mismatch_cnt to be
> non-zero!! (heh heh). At any rate, the guide reflects how I deal with
> that - by turning off the use of UUIDs.
>
> Many thanks for taking a look at the guide and sharing your thoughts!
> Please let me know if you still think I should change that part where I
> say that RAID10 gives me faster sequential reads, and what you would say
> instead.
>
> Iordan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

Re: debian software raid1

[Keld Jørn Simonsen]

On Fri, Apr 22, 2011 at 08:05:06PM -0400, Iordan Iordanov wrote:
> Hi Neil,
> 
> On 04/22/11 18:12, NeilBrown wrote:
> >This is not correct.  RAID10-n2 on 2 drives is exactly the same layout and
> >very nearly the same speed as RAID1 on 2 drives.  (I say 'very nearly' only
> >because the read-balancing code is a little different and might have 
> >slightly
> >different results).

Well, I think it has some significantly different results with the
different load balancing algorithms. For example the one reported in
this thread. Also other bemchmarks indicate this.

> >Or have you measured these two and found an actually difference?  That 
> >would
> >certainly be interesting.
> 
> The difference that I see is probably 100% due to the different read 
> balancing algorithm. When I start two dd processes reading from two 
> separate partitions on the RAID (just so there are no buffers screwing 
> up my results), with RAID1, I see less than one drive worth of 
> sequential read speed for the two dd processes combined.
> 
> On the other hand, with RAID10 I see the two drives being utilized 
> fully, and I get one drive worth of sequential read speeds for each dd 
> process, or a total of two drives worth of read speed for the two dd 
> processes.
> 
> The numbers were something like this:
> 
> - Single drive speed: ~130MB/s sequential read.
> - Two simultaneous dd sequential reads with RAID1, bs=1024k: ~40MB/s per dd.
> - Two simultaneous dd sequential reads with RAID10, bs=1024k: ~130MB/s 
> per dd.
> 
> That's what I meant by better sequential reads, but perhaps I should try 
> to phrase it more precisely.
> 
> >RAID10-f2 will give faster sequential reads at the cost of slower writes.

The writes will not be much slower, maybe 3 % slower, and in some cases
faster, according to some benchmarks.

> I am not sure what RAID10-f2 on a two disk setup will look like, but I 
> like the idea of the drives being identical, and in the worst case, 
> being able to pull one drive, zero the superblock, and be left with a 
> drive with intact data, which only RAID10-n2 can give, if I am not mistaken.

Yes, RAID10-far and RAID10-offset will not do that. However both
RAID10-far and RAID10-offset will be able to run in degraded mode with
just one disk, and with all data intact.

raid10-far will perform similarily to raid10-near with 2 dd'sC, also a
near 100 % utilization of both drives. however, with just 1 dd,
raid10-far wil also give almost 100 % utilization on bothe drives, while
raid10-near will give 100 % on one drive and 0 % on the other drive (I
think). Also when you ar doing multiple IO, RAID10-far will tend to give
you speeds for an additional sequential read above the speed of a single
drive - none of the other MD raid1/10 formats would do that. 


> Just to follow up on our discussion on Grub v2 and booting from a RAID 
> device. I discovered that if I allow Grub to use UUID, occasionally, it 
> would try to mount a raw device for root instead of the RAID device. 
> Apart from the nuisance, this would probably cause mismatch_cnt to be 
> non-zero!! (heh heh). At any rate, the guide reflects how I deal with 
> that - by turning off the use of UUIDs.
> 
> Many thanks for taking a look at the guide and sharing your thoughts! 
> Please let me know if you still think I should change that part where I 
> say that RAID10 gives me faster sequential reads, and what you would say 
> instead.

best regards
keld