Software raid, booting and bios

Software raid, booting and bios

[Paul van der Vlis]

Hello,

I use software raid (mdadm). The main problem for me is that when the
drive with the MBR fails, it can become a problem to boot.

When the bios would use another drive to boot when the first drive
failes, this problem would be gone. But I don't know rackservers who do
that. Do you?

Or is there maybe some kind of fake-raid card what uses mdadm to solve
this problem?

Another way would be to use e.g. an USB device to boot to solve this
problem. Any experiences with that?

(hmm, I realize that netboot is an option too).

With regards,
Paul van der Vlis.


-- 
http://www.vandervlis.nl

Re: Software raid, booting and bios

[Simon Mcnair]

Please can you further define what you mean by 'it can become a
problem to boot' ?
Generally this is resolved by having a mbr and boot partition on each
of your mirrored drives so that whichever you use to boot has the
pertinent information to boot the kernel and construct the raid array.
If you have raid 5 with 3 disks you'd have a 3 drive mirror partition
on each disk and a raid 5 set across all three too.

I'm not a guru on this and can't provide much knowledge past the
theory and high level ;-)
Simon

On 20 May 2011, at 07:55, Paul van der Vlis <paul@vandervlis.nl> wrote:

> Hello,
>
> I use software raid (mdadm). The main problem for me is that when the
> drive with the MBR fails, it can become a problem to boot.
>
> When the bios would use another drive to boot when the first drive
> failes, this problem would be gone. But I don't know rackservers who do
> that. Do you?
>
> Or is there maybe some kind of fake-raid card what uses mdadm to solve
> this problem?
>
> Another way would be to use e.g. an USB device to boot to solve this
> problem. Any experiences with that?
>
> (hmm, I realize that netboot is an option too).
>
> With regards,
> Paul van der Vlis.
>
>
> --
> http://www.vandervlis.nl
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 09:03, Simon Mcnair schreef:
> Please can you further define what you mean by 'it can become a
> problem to boot' ?
> Generally this is resolved by having a mbr and boot partition on each
> of your mirrored drives so that whichever you use to boot has the
> pertinent information to boot the kernel and construct the raid array.
> If you have raid 5 with 3 disks you'd have a 3 drive mirror partition
> on each disk and a raid 5 set across all three too.

In the bios from my machines (Supermicro, Dell) I can select only one
drive to boot. Wenn the drive fails, no other disk is tried.

I can go into the bios and change the drive when it fails, or I can
exchange the disks. But I would like it, when the machine would simple
boot even when the first disk is corrupt.

With regards,
Paul van der Vlis.


> I'm not a guru on this and can't provide much knowledge past the
> theory and high level ;-)
> Simon
> 
> On 20 May 2011, at 07:55, Paul van der Vlis <paul@vandervlis.nl> wrote:
> 
>> Hello,
>>
>> I use software raid (mdadm). The main problem for me is that when the
>> drive with the MBR fails, it can become a problem to boot.
>>
>> When the bios would use another drive to boot when the first drive
>> failes, this problem would be gone. But I don't know rackservers who do
>> that. Do you?
>>
>> Or is there maybe some kind of fake-raid card what uses mdadm to solve
>> this problem?
>>
>> Another way would be to use e.g. an USB device to boot to solve this
>> problem. Any experiences with that?
>>
>> (hmm, I realize that netboot is an option too).
>>
>> With regards,
>> Paul van der Vlis.
>>
>>
>> --
>> http://www.vandervlis.nl
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Software raid, booting and bios

[Simon Mcnair]

I have not come across a pc which does not allow you to boot a
secondary drive before... Please can you read  the manual and triple
check this ?  The only possible reason I can think this would happen
Is that you're using an add on board and you would configure this from
a secondary bios. My Dell, Asus, and all other motherboards I've had
over the past 10 years all allow a second device.
Cheers
Simon

On 20 May 2011, at 08:15, Paul van der Vlis <paul@vandervlis.nl> wrote:

> Op 20-05-11 09:03, Simon Mcnair schreef:
>> Please can you further define what you mean by 'it can become a
>> problem to boot' ?
>> Generally this is resolved by having a mbr and boot partition on each
>> of your mirrored drives so that whichever you use to boot has the
>> pertinent information to boot the kernel and construct the raid array.
>> If you have raid 5 with 3 disks you'd have a 3 drive mirror partition
>> on each disk and a raid 5 set across all three too.
>
> In the bios from my machines (Supermicro, Dell) I can select only one
> drive to boot. Wenn the drive fails, no other disk is tried.
>
> I can go into the bios and change the drive when it fails, or I can
> exchange the disks. But I would like it, when the machine would simple
> boot even when the first disk is corrupt.
>
> With regards,
> Paul van der Vlis.
>
>
>> I'm not a guru on this and can't provide much knowledge past the
>> theory and high level ;-)
>> Simon
>>
>> On 20 May 2011, at 07:55, Paul van der Vlis <paul@vandervlis.nl> wrote:
>>
>>> Hello,
>>>
>>> I use software raid (mdadm). The main problem for me is that when the
>>> drive with the MBR fails, it can become a problem to boot.
>>>
>>> When the bios would use another drive to boot when the first drive
>>> failes, this problem would be gone. But I don't know rackservers who do
>>> that. Do you?
>>>
>>> Or is there maybe some kind of fake-raid card what uses mdadm to solve
>>> this problem?
>>>
>>> Another way would be to use e.g. an USB device to boot to solve this
>>> problem. Any experiences with that?
>>>
>>> (hmm, I realize that netboot is an option too).
>>>
>>> With regards,
>>> Paul van der Vlis.
>>>
>>>
>>> --
>>> http://www.vandervlis.nl
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 09:19, Simon Mcnair schreef:
> I have not come across a pc which does not allow you to boot a
> secondary drive before... Please can you read  the manual and triple
> check this ?  The only possible reason I can think this would happen
> Is that you're using an add on board and you would configure this from
> a secondary bios. My Dell, Asus, and all other motherboards I've had
> over the past 10 years all allow a second device.

You can select the "boot device priority" where you can choose about
devices types (DVD, harddisk, USB, network) but you can choose only one
SATA disk. Study it, and you will see I am right. I've asked it to my
rackserver-vendor, they say: "that's always the case".

But I think I have had systems in the past, what could do it. An
interesting question is then: how well is it tested?  What when e.g. a
disk boots, and then gives an I/O error? I am looking for a well-tested
way to solve this, and I am willing to pay for it or choose another
hardware vendor for it.

With regards,
Paul van der Vlis.



> Cheers
> Simon
> 
> On 20 May 2011, at 08:15, Paul van der Vlis <paul@vandervlis.nl> wrote:
> 
>> Op 20-05-11 09:03, Simon Mcnair schreef:
>>> Please can you further define what you mean by 'it can become a
>>> problem to boot' ?
>>> Generally this is resolved by having a mbr and boot partition on each
>>> of your mirrored drives so that whichever you use to boot has the
>>> pertinent information to boot the kernel and construct the raid array.
>>> If you have raid 5 with 3 disks you'd have a 3 drive mirror partition
>>> on each disk and a raid 5 set across all three too.
>>
>> In the bios from my machines (Supermicro, Dell) I can select only one
>> drive to boot. Wenn the drive fails, no other disk is tried.
>>
>> I can go into the bios and change the drive when it fails, or I can
>> exchange the disks. But I would like it, when the machine would simple
>> boot even when the first disk is corrupt.
>>
>> With regards,
>> Paul van der Vlis.
>>
>>
>>> I'm not a guru on this and can't provide much knowledge past the
>>> theory and high level ;-)
>>> Simon
>>>
>>> On 20 May 2011, at 07:55, Paul van der Vlis <paul@vandervlis.nl> wrote:
>>>
>>>> Hello,
>>>>
>>>> I use software raid (mdadm). The main problem for me is that when the
>>>> drive with the MBR fails, it can become a problem to boot.
>>>>
>>>> When the bios would use another drive to boot when the first drive
>>>> failes, this problem would be gone. But I don't know rackservers who do
>>>> that. Do you?
>>>>
>>>> Or is there maybe some kind of fake-raid card what uses mdadm to solve
>>>> this problem?
>>>>
>>>> Another way would be to use e.g. an USB device to boot to solve this
>>>> problem. Any experiences with that?
>>>>
>>>> (hmm, I realize that netboot is an option too).
>>>>
>>>> With regards,
>>>> Paul van der Vlis.
>>>>
>>>>
>>>> --
>>>> http://www.vandervlis.nl
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Software raid, booting and bios

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Fri, 20 May 2011 10:33:00 +0200
Paul van der Vlis <paul@vandervlis.nl> wrote:

> You can select the "boot device priority" where you can choose about
> devices types (DVD, harddisk, USB, network) but you can choose only one
> SATA disk. Study it, and you will see I am right. I've asked it to my
> rackserver-vendor, they say: "that's always the case".

How about just not buying crappy hardware from this lying vendor anymore.
http://ompldr.org/vOHB6Zw/bios4.jpg <- this is present in majority of
motherboard BIOSes since forever.

> But I think I have had systems in the past, what could do it. An
> interesting question is then: how well is it tested?  What when e.g. a
> disk boots, and then gives an I/O error? I am looking for a well-tested
> way to solve this, and I am willing to pay for it or choose another
> hardware vendor for it.

Yes, I think it is conceivable that if a disk fails in a 'bad' way, i.e. by
locking up on reads, or reading the first sector but not the next ones it can
prevent the system from booting even with this priority system. I don't know
if chances of that are high, considering that quite often disks fail by also
ceasing to be detectable in BIOS, in which case your boot-up would proceed
normally.

-- 
With respect,
Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 10:56, Roman Mamedov schreef:
> On Fri, 20 May 2011 10:33:00 +0200
> Paul van der Vlis <paul@vandervlis.nl> wrote:
> 
>> You can select the "boot device priority" where you can choose about
>> devices types (DVD, harddisk, USB, network) but you can choose only one
>> SATA disk. Study it, and you will see I am right. I've asked it to my
>> rackserver-vendor, they say: "that's always the case".
> 
> How about just not buying crappy hardware from this lying vendor anymore.
> http://ompldr.org/vOHB6Zw/bios4.jpg <- this is present in majority of
> motherboard BIOSes since forever.

Interesting. From what brand server is this?

>> But I think I have had systems in the past, what could do it. An
>> interesting question is then: how well is it tested?  What when e.g. a
>> disk boots, and then gives an I/O error? I am looking for a well-tested
>> way to solve this, and I am willing to pay for it or choose another
>> hardware vendor for it.
> 
> Yes, I think it is conceivable that if a disk fails in a 'bad' way, i.e. by
> locking up on reads, or reading the first sector but not the next ones it can
> prevent the system from booting even with this priority system. I don't know
> if chances of that are high, considering that quite often disks fail by also
> ceasing to be detectable in BIOS, in which case your boot-up would proceed
> normally.

The problem is about detected disks with a defect in the MBR.

With regards,
Paul van der Vlis.

Re: Software raid, booting and bios

[Simon McNair]

Paul,
Please can you get some digital camera shots (resize them, low res and 
compress them) of the bios screen (or screen grabs if it's via a LOM).  
I'm especially interested in knowing if in the device listing there are 
two or more hard disks in the first place (as if there is only one disk 
I'd not be surprised at getting only one set of options).

Can you ask you rack space vendor the specific question 'how do I set up 
a secondary boot device ?".

Simon

On 20/05/2011 10:33, Paul van der Vlis wrote:
> Op 20-05-11 10:56, Roman Mamedov schreef:
>> On Fri, 20 May 2011 10:33:00 +0200
>> Paul van der Vlis<paul@vandervlis.nl>  wrote:
>>
>>> You can select the "boot device priority" where you can choose about
>>> devices types (DVD, harddisk, USB, network) but you can choose only one
>>> SATA disk. Study it, and you will see I am right. I've asked it to my
>>> rackserver-vendor, they say: "that's always the case".
>> How about just not buying crappy hardware from this lying vendor anymore.
>> http://ompldr.org/vOHB6Zw/bios4.jpg<- this is present in majority of
>> motherboard BIOSes since forever.
> Interesting. From what brand server is this?
>
>>> But I think I have had systems in the past, what could do it. An
>>> interesting question is then: how well is it tested?  What when e.g. a
>>> disk boots, and then gives an I/O error? I am looking for a well-tested
>>> way to solve this, and I am willing to pay for it or choose another
>>> hardware vendor for it.
>> Yes, I think it is conceivable that if a disk fails in a 'bad' way, i.e. by
>> locking up on reads, or reading the first sector but not the next ones it can
>> prevent the system from booting even with this priority system. I don't know
>> if chances of that are high, considering that quite often disks fail by also
>> ceasing to be detectable in BIOS, in which case your boot-up would proceed
>> normally.
> The problem is about detected disks with a defect in the MBR.
>
> With regards,
> Paul van der Vlis.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 12:00, Simon McNair schreef:
> Paul,
> Please can you get some digital camera shots (resize them, low res and
> compress them) of the bios screen (or screen grabs if it's via a LOM). 
> I'm especially interested in knowing if in the device listing there are
> two or more hard disks in the first place (as if there is only one disk
> I'd not be surprised at getting only one set of options).

It's not easy. I have machines in production and I had to reboot them to
do this. And I want to buy a new machine, but I don't have it yet.

> Can you ask you rack space vendor the specific question 'how do I set up
> a secondary boot device ?".

I can talk with them about it, but I think in most cases the first
device will be detected, so the second device will not be used.

With regards,
Paul van der Vlis.

Re: Software raid, booting and bios

[Phil Turmel]

On 05/20/2011 05:33 AM, Paul van der Vlis wrote:
> 
> The problem is about detected disks with a defect in the MBR.

This is a crucial point.  A BIOS that supports multiple drives in the boot order should skip to the next if the MBR cannot be read.  But the BIOS loses control once the MBR code is executed.  If an error is encountered in later sectors of the bootloader, there's no way to switch to the next drive.  This is also true when the BIOS only supports dissimilar devices in the boot order.

If I had to minimize the chance of this ever biting me, I'd use a CF <==> IDE adapter with a DMA capable CF card, and set it up as my boot device.  And I wouldn't use it for anything but boot.  A quick google turned up this:

http://www.addonics.com/products/flash_memory_reader/adidecf.asp

(Just to show what's out there.)  The embedded boards I use occasionally have the equivalent of this soldered to their motherboards.

The best DMA capable CF cards are usually found in markets that cater to industrial designers or to professional photographers.

HTH,

Phil

Re: Software raid, booting and bios

[Gordon Henderson]

On Fri, 20 May 2011, Phil Turmel wrote:

> http://www.addonics.com/products/flash_memory_reader/adidecf.asp
>
> (Just to show what's out there.)  The embedded boards I use occasionally 
> have the equivalent of this soldered to their motherboards.
>
> The best DMA capable CF cards are usually found in markets that cater to 
> industrial designers or to professional photographers.

FWIW: I've built a few systems which boot off flash, then run from SATA 
drives - but it all depends on the motherboards. These days there are many 
IDE (and now SATA) drives - maybe the forerunner to SSD's... e.g. I use 
these sort of things:

   http://linitx.com/viewcategory.php?catid=129

It's easy to have root on a small flash IDE drive then the rest of the 
system on RAID'd SATA drives. (swap, /usr, /var, /home, etc.)

Is that good? I don't know - but I also build lots of small embedded 
systems (no drives) for other purposes which boot off these type of 
devices and I've not had an issue with one failling in 5+ years...

Motherboards are increasingly not coming with IDE ports though, but things 
move on!

Gordon

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 14:11, Phil Turmel schreef:
> On 05/20/2011 05:33 AM, Paul van der Vlis wrote:
>> 
>> The problem is about detected disks with a defect in the MBR.
> 
> This is a crucial point.  A BIOS that supports multiple drives in the
> boot order should skip to the next if the MBR cannot be read.  But
> the BIOS loses control once the MBR code is executed.  If an error is
> encountered in later sectors of the bootloader, there's no way to
> switch to the next drive.  This is also true when the BIOS only
> supports dissimilar devices in the boot order.

I think you are right.

> If I had to minimize the chance of this ever biting me, I'd use a CF
> <==> IDE adapter with a DMA capable CF card, and set it up as my boot
> device.  And I wouldn't use it for anything but boot.  A quick google
> turned up this:
> 
> http://www.addonics.com/products/flash_memory_reader/adidecf.asp

In the servers I use, there is normally no place for such a cardreader.
But a low profile PCI card with a CF card on it could do it.

> (Just to show what's out there.)  The embedded boards I use
> occasionally have the equivalent of this soldered to their
> motherboards.
> 
> The best DMA capable CF cards are usually found in markets that cater
> to industrial designers or to professional photographers.

Do you think the risk of a problem with a CF card (or something like
that) is much lower then the risk of a problem with a harddisk?

And what about booting from an USB stick?

With regards,
Paul van der Vlis.


-- 
http://www.vandervlis.nl

Re: Software raid, booting and bios

[Phil Turmel]

{do use reply-to-all on kernel.org lists... not everyone is subscribed}

On 05/20/2011 11:53 AM, Paul van der Vlis wrote:
> Op 20-05-11 14:11, Phil Turmel schreef:
>> (Just to show what's out there.)  The embedded boards I use
>> occasionally have the equivalent of this soldered to their
>> motherboards.
>>
>> The best DMA capable CF cards are usually found in markets that cater
>> to industrial designers or to professional photographers.
> 
> Do you think the risk of a problem with a CF card (or something like
> that) is much lower then the risk of a problem with a harddisk?

the big deal is the lack of moving parts:  No spindle bearing, no head positioner gear train.  On top of that, when set up to support your boot tasks only, there's no write activity to wear it out.

> And what about booting from an USB stick?

Just as good, technically, IMHO.  If mounted internally, just as good, period.  Plugged into an external port, I'd be wary of some uninformed soul pulling it out.  CF cards look like they "belong".

I like Ed W's suggestions, as well, with the caveat that their usefulness would make them more likely to be "borrowed".  Even by yourself, in a pinch.

Phil

Re: Software raid, booting and bios

[Roberto Spadim]

i´m using usb 'pendrives' for 3 years without problems, only damn
small linux, or slitaz running in a desktop machine

2011/5/20 Phil Turmel <philip@turmel.org>:
> {do use reply-to-all on kernel.org lists... not everyone is subscribed}
>
> On 05/20/2011 11:53 AM, Paul van der Vlis wrote:
>> Op 20-05-11 14:11, Phil Turmel schreef:
>>> (Just to show what's out there.)  The embedded boards I use
>>> occasionally have the equivalent of this soldered to their
>>> motherboards.
>>>
>>> The best DMA capable CF cards are usually found in markets that cater
>>> to industrial designers or to professional photographers.
>>
>> Do you think the risk of a problem with a CF card (or something like
>> that) is much lower then the risk of a problem with a harddisk?
>
> the big deal is the lack of moving parts:  No spindle bearing, no head positioner gear train.  On top of that, when set up to support your boot tasks only, there's no write activity to wear it out.
>
>> And what about booting from an USB stick?
>
> Just as good, technically, IMHO.  If mounted internally, just as good, period.  Plugged into an external port, I'd be wary of some uninformed soul pulling it out.  CF cards look like they "belong".
>
> I like Ed W's suggestions, as well, with the caveat that their usefulness would make them more likely to be "borrowed".  Even by yourself, in a pinch.
>
> Phil
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[Brad Campbell]

On 21/05/11 03:32, Phil Turmel wrote:

> the big deal is the lack of moving parts:  No spindle bearing, no head positioner gear train.

Sorry, this just tickled me. "gear train" ? Which decade are we talking about? The last drive I saw 
that had any form of mechanical power transfer mechanism for head positioning was a 60MB RLL Seagate 
clunker.

Now, to add some form of use to the thread I've been using commodity CF cards in home-brew CF to ATA 
adaptors in embedded systems for 10 years. Flash is _the_ way to go for high reliability systems 
that don't have lots of write cycles.

My TV box that has no on-board PXE has been booting from a 10MB USB stick using loadlinux since 
2003. Dead reliable after ~69,000 hours power on time. I've not had a hard disk last that long since 
my old 200MB WD IDE drive (which is still running with over 100,000 hours on it).

Brad
-- 
Dolphins are so intelligent that within a few weeks they can
train Americans to stand at the edge of the pool and throw them
fish.

RE: Software raid, booting and bios

[Leslie Rhorer]

> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
> owner@vger.kernel.org] On Behalf Of Brad Campbell
> Sent: Friday, May 20, 2011 4:52 PM
> To: linux-raid@vger.kernel.org
> Subject: Re: Software raid, booting and bios
> 
> On 21/05/11 03:32, Phil Turmel wrote:
> 
> > the big deal is the lack of moving parts:  No spindle bearing, no head
> positioner gear train.
> 
> Sorry, this just tickled me. "gear train" ? Which decade are we talking
> about? The last drive I saw
> that had any form of mechanical power transfer mechanism for head
> positioning was a 60MB RLL Seagate
> clunker.

	Yeah, the number of moving parts in a hard drive is minimal.  OTOH,
it's not zero.

> Now, to add some form of use to the thread I've been using commodity CF
> cards in home-brew CF to ATA
> adaptors in embedded systems for 10 years. Flash is _the_ way to go for
> high reliability systems
> that don't have lots of write cycles.
> 
> My TV box that has no on-board PXE has been booting from a 10MB USB stick
> using loadlinux since
> 2003. Dead reliable after ~69,000 hours power on time. I've not had a hard
> disk last that long since
> my old 200MB WD IDE drive (which is still running with over 100,000 hours
> on it).

	Oh, we have quite a number of embedded controllers with SCSI hard
drives that have been spinning continuously since 1992.

Re: Software raid, booting and bios

[Simon McNair]

I think you might want to investigate a SATA DOM (Disk on module).  My 
Thecus 5200 uses on and it seems a neat solution.

eg
http://www.innodisk.com/flashstorage-list.jsp?items_name=satadom

Simon

On 21/05/2011 09:19, Leslie Rhorer wrote:
>> -----Original Message-----
>> From: linux-raid-owner@vger.kernel.org [mailto:linux-raid-
>> owner@vger.kernel.org] On Behalf Of Brad Campbell
>> Sent: Friday, May 20, 2011 4:52 PM
>> To: linux-raid@vger.kernel.org
>> Subject: Re: Software raid, booting and bios
>>
>> On 21/05/11 03:32, Phil Turmel wrote:
>>
>>> the big deal is the lack of moving parts:  No spindle bearing, no head
>> positioner gear train.
>>
>> Sorry, this just tickled me. "gear train" ? Which decade are we talking
>> about? The last drive I saw
>> that had any form of mechanical power transfer mechanism for head
>> positioning was a 60MB RLL Seagate
>> clunker.
> 	Yeah, the number of moving parts in a hard drive is minimal.  OTOH,
> it's not zero.
>
>> Now, to add some form of use to the thread I've been using commodity CF
>> cards in home-brew CF to ATA
>> adaptors in embedded systems for 10 years. Flash is _the_ way to go for
>> high reliability systems
>> that don't have lots of write cycles.
>>
>> My TV box that has no on-board PXE has been booting from a 10MB USB stick
>> using loadlinux since
>> 2003. Dead reliable after ~69,000 hours power on time. I've not had a hard
>> disk last that long since
>> my old 200MB WD IDE drive (which is still running with over 100,000 hours
>> on it).
> 	Oh, we have quite a number of embedded controllers with SCSI hard
> drives that have been spinning continuously since 1992.
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[CoolCold]

Здравствуйте, Paul.

Вы писали 20 мая 2011 г., 12:33:00:

> Op 20-05-11 09:19, Simon Mcnair schreef:
>> I have not come across a pc which does not allow you to boot a
>> secondary drive before... Please can you read  the manual and triple
>> check this ?  The only possible reason I can think this would happen
>> Is that you're using an add on board and you would configure this from
>> a secondary bios. My Dell, Asus, and all other motherboards I've had
>> over the past 10 years all allow a second device.

> You can select the "boot device priority" where you can choose about
> devices types (DVD, harddisk, USB, network) but you can choose only one
> SATA disk. Study it, and you will see I am right. I've asked it to my
> rackserver-vendor, they say: "that's always the case".
I've  seen supermicro servers with bios allowing to set several drives
as boot disks and without such option (like in your case).
One      of      those      which      could      do      this     was
http://www.supermicro.com/products/motherboard/Xeon1333/5000P/X7DB3.cfm
if i remember things right.


> But I think I have had systems in the past, what could do it. An
> interesting question is then: how well is it tested?  What when e.g. a
> disk boots, and then gives an I/O error? I am looking for a well-tested
> way to solve this, and I am willing to pay for it or choose another
> hardware vendor for it.

> With regards,
> Paul van der Vlis.



>> Cheers
>> Simon
>> 
>> On 20 May 2011, at 08:15, Paul van der Vlis <paul@vandervlis.nl> wrote:
>> 
>>> Op 20-05-11 09:03, Simon Mcnair schreef:
>>>> Please can you further define what you mean by 'it can become a
>>>> problem to boot' ?
>>>> Generally this is resolved by having a mbr and boot partition on each
>>>> of your mirrored drives so that whichever you use to boot has the
>>>> pertinent information to boot the kernel and construct the raid array.
>>>> If you have raid 5 with 3 disks you'd have a 3 drive mirror partition
>>>> on each disk and a raid 5 set across all three too.
>>>
>>> In the bios from my machines (Supermicro, Dell) I can select only one
>>> drive to boot. Wenn the drive fails, no other disk is tried.
>>>
>>> I can go into the bios and change the drive when it fails, or I can
>>> exchange the disks. But I would like it, when the machine would simple
>>> boot even when the first disk is corrupt.
>>>
>>> With regards,
>>> Paul van der Vlis.
>>>
>>>
>>>> I'm not a guru on this and can't provide much knowledge past the
>>>> theory and high level ;-)
>>>> Simon
>>>>
>>>> On 20 May 2011, at 07:55, Paul van der Vlis <paul@vandervlis.nl> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I use software raid (mdadm). The main problem for me is that when the
>>>>> drive with the MBR fails, it can become a problem to boot.
>>>>>
>>>>> When the bios would use another drive to boot when the first drive
>>>>> failes, this problem would be gone. But I don't know rackservers who do
>>>>> that. Do you?
>>>>>
>>>>> Or is there maybe some kind of fake-raid card what uses mdadm to solve
>>>>> this problem?
>>>>>
>>>>> Another way would be to use e.g. an USB device to boot to solve this
>>>>> problem. Any experiences with that?
>>>>>
>>>>> (hmm, I realize that netboot is an option too).
>>>>>
>>>>> With regards,
>>>>> Paul van der Vlis.
>>>>>
>>>>>
>>>>> --
>>>>> http://www.vandervlis.nl
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 


> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Best regards,
[COOLCOLD-RIPN]

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software raid, booting and bios

[Ed W]

On 20/05/2011 09:33, Paul van der Vlis wrote:
> You can select the "boot device priority" where you can choose about
> devices types (DVD, harddisk, USB, network) but you can choose only one
> SATA disk. Study it, and you will see I am right. I've asked it to my
> rackserver-vendor, they say: "that's always the case".

Hi, what I have done with all my supermicro servers is to buy a tiny USB
flash drive (physically small, not capacity small) - I think what I
bought might be one of the tiny PNY devices, not sure though

The Supermicro boards have internal USB headers mounted on the
motherboard, even with a 1U server I have plenty of room to install my
USB on the MB (could stick them out the back of the server and cable tie
them (or superglue them))

Then I put SysrescueCD on my stick and setup GRUB with a bunch of boot
options.

In my case I'm under the possibly misguided apprehension that my boot
will fail over to the spare disks if one fails. However, I can set the
subsequent failover to be my USB stick also.  I think I have them set at
the moment that the USB stick boots the main drives as normal, but has a
boot menu where I can also boot the sysrescueimage if I need to (I use
this (over IPMI) for initial system installation and serious
maintenance, eg failed grub upgrade or similar).

The only other option that I think the big hosting guys use is to have a
netboot setup which boots everything and can also offer rescue images,
etc.  Beyond my skills to setup for my meagre number of servers, but if
you have more than a couple of machines this could be a very good solution?

For my needs the USB stick option is perfect

Sysrescuecd suits me because all my servers are gentoo based - clearly
it will work for other distros also, but you might want to evaluate
other rescue distros before choosing one?

Good luck

Ed W

Re: Software raid, booting and bios

[Paul van der Vlis]

Op 20-05-11 21:13, Ed W schreef:
> On 20/05/2011 09:33, Paul van der Vlis wrote:
>> You can select the "boot device priority" where you can choose about
>> devices types (DVD, harddisk, USB, network) but you can choose only one
>> SATA disk. Study it, and you will see I am right. I've asked it to my
>> rackserver-vendor, they say: "that's always the case".
> 
> Hi, what I have done with all my supermicro servers is to buy a tiny USB
> flash drive (physically small, not capacity small) - I think what I
> bought might be one of the tiny PNY devices, not sure though
> 
> The Supermicro boards have internal USB headers mounted on the
> motherboard, even with a 1U server I have plenty of room to install my
> USB on the MB (could stick them out the back of the server and cable tie
> them (or superglue them))

Nice to hear.

> Then I put SysrescueCD on my stick and setup GRUB with a bunch of boot
> options.
> 
> In my case I'm under the possibly misguided apprehension that my boot
> will fail over to the spare disks if one fails. However, I can set the
> subsequent failover to be my USB stick also.  I think I have them set at
> the moment that the USB stick boots the main drives as normal, but has a
> boot menu where I can also boot the sysrescueimage if I need to (I use
> this (over IPMI) for initial system installation and serious
> maintenance, eg failed grub upgrade or similar).

Interesting.

> The only other option that I think the big hosting guys use is to have a
> netboot setup which boots everything and can also offer rescue images,
> etc.  Beyond my skills to setup for my meagre number of servers, but if
> you have more than a couple of machines this could be a very good solution?

I don't have much machines, but I have my own IPv4 range.
So it is possible.

> For my needs the USB stick option is perfect

I will think about it. A little disadvantage is that it's not so easy to
have access to the USB stick maybe (you have to open the machine).

> Sysrescuecd suits me because all my servers are gentoo based - clearly
> it will work for other distros also, but you might want to evaluate
> other rescue distros before choosing one?

I use Debian, so I think I would choose a Debian-based rescue system.

What still a question is for me:
Is it better to put /boot on such an usb-stick or only the MBR?

Grub2 can boot from lvm/raid, and when I would have /boot on disk I can
have an MBR on disks too (for the case the usb stick would fail).
But difficult for me to understand that things like the initrd comes
from a software raid....

Another way would be make a /boot on a raid1 with 3 devices: the
harddisks and the USB stick.

With regards,
Paul van der Vlis.

-- 
http://www.vandervlis.nl/

Re: Software raid, booting and bios

[Ed W]

Hi


>> The only other option that I think the big hosting guys use is to have a
>> netboot setup which boots everything and can also offer rescue images,
>> etc.  Beyond my skills to setup for my meagre number of servers, but if
>> you have more than a couple of machines this could be a very good solution?
> 
> I don't have much machines, but I have my own IPv4 range.
> So it is possible.

You don't actually need any range to make netboot work (kind of).  Bear
in mind I have never done this... But you arrange for netboot to boot
from some internal server and in fact try and use your firewall to stop
the machines being visible to the outside world during this process
(think about various ways you can netboot to say some 192.168.x.y range
and if required say bounce ssh off some other machine to your server in
rescue mode (I just use the IPMI port so I don't need the machine to
have any working network access at all)

>> For my needs the USB stick option is perfect
> 
> I will think about it. A little disadvantage is that it's not so easy to
> have access to the USB stick maybe (you have to open the machine).

Why would you ever need to access the USB stick?

As several other people already said - NOT being able to access it is a
bonus.. I would recommend superglue/cable-ties if you can't put the
stick inside the machine...

>> Sysrescuecd suits me because all my servers are gentoo based - clearly
>> it will work for other distros also, but you might want to evaluate
>> other rescue distros before choosing one?
> 
> I use Debian, so I think I would choose a Debian-based rescue system.

Sure - but don't get too bogged down in the source distro.  Sysrescuecd
will have a bang up to date set of binaries, likely far newer than those
on your live debian machine.  It's really a question of whether that's a
benefit or liability...

For any work on the machine you simply chroot into the live machine and
then you are using all identical binaries (in fact the live machine) and
so really all you need is a boot environment which will support your
chosen raid/lvm/cpu architecture

> 
> What still a question is for me:
> Is it better to put /boot on such an usb-stick or only the MBR?

Why pick only one?

I use a small raid 1 partition for /boot which mirrors across all 4
disks in my 1U servers.  I try and install boot sectors on all drives so
I could in theory boot off any disk

Then I customise my sysrescuecd bootloader menu to
- boot the USB stick,
- rebuild the grub bootsector
- Boot drive 1
- Boot drive 2
- ...

For icing you could backup the /boot from the disks onto the USB stick,
but I haven't done that since it can be easily rebuilt (it's just the
kernel, grub boot files and grub.conf)


> Grub2 can boot from lvm/raid, and when I would have /boot on disk I can

I don't see any reason to deviate from the "small raid 1 boot partition"
idea.  Needs only grub1 and no special handling?

> have an MBR on disks too (for the case the usb stick would fail).

I would suggest you boot from the disks first and set your USB stick as
the failover boot drive, and make sure it's boot menu picks to boot from
say disk2 as the default option?  This covers you for the case that
someone pinches disk1 and reboots the machine, but your normal process
of maintaining the boot partition on the local disks holds

The USB drive then remains read-only, so no chance to bugger it up
accidently and you can churn out dozens for a tiny budget and re-use
them for quite some years

I can think of few downsides assuming you figure out how to make the
blasted things boot (not always easy) and can fit them inside the server
or similar?

> But difficult for me to understand that things like the initrd comes
> from a software raid....

Just don't make the boot process complicated...

I'm a ludite, I build my kernel with everything needed to boot builtin,
therefore no initrd needed (unless you want root on an md partition
that's not 0.90 format?).  I can't cope with rebuilding initrds and all
that jazz...   Now you can just update bzImage to update, plus you can
boot from any disk without treating it as raid at all (it's read only,
so with a 0.90 raid format you can use disks individually with GRUB -
mount them as a raid1 to modify them and they all stay in sync)

Easy peasy

> Another way would be make a /boot on a raid1 with 3 devices: the
> harddisks and the USB stick.

I just don't see that /boot is that precious?  You probably have the
same info on every single other server you own (assuming you build same
kernels on all).  Raid1 it across all your drives, then all you need is
the "boot sector" to be copied to more than one place and you do that by
copying it to all drives, plus having a bootable USB drive.

Remember a bootable disk (say USB) can then boot the /boot partition
from any other disk.  All you are trying to do is get to the point that
grub (etc) is running and shows you some menu of choices (and picks
sensible default choices) - the rest is easy

I think you are probably not seeing how simple this really is?  Have a
read through the above again if you still think you need custom USB
sticks or some complex raid1 scenario?  It should just work very simply
and even with limited bios options, as long as you can boot from one
disk and one USB stick then you have a failover

Good luck

Ed W