UEFI and mdadm questions.

UEFI and mdadm questions.

[Wilson, Jonathan]

From what I can tell with UEFI I need to set up a UEFI partition with a
FAT format.

On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
Raid 1 40G.

Obviously Grub installs to the mbr, and then installs a bit into
Biosboot which can read raids, hence it can read and boot from /boot.

Further, from what I can tell, into the UEFI partition can go either a
kernel & initramfs with UEFI support, or a "loader" that then loads the
kernel.

What I am unsure about are... 

1)  Can the loader/kernel understand md raid? so the / can be in a bog
standard raid1 v1.2?

2) I'm guessing I would no longer need the /boot as that would be
replaced by what ever was in the UEFI partition?

3) as my "/boot" is currently in a raid 1 my life is simple, should any
changes occur they are replicated to the drives I have set up as /boot
raid 1, and I installed the mbr portion of boot loader manually on each
disk and tested pulling one, and then booting from another.. it
worked :-)
So I would like to keep things, if not simple, at least less likely to
have problems because I forgot to install duplicates of the UEFI on all
the disks... so can I use a .90v raid1 on the UEFI partition, then
format it as fat... so that all copies of what ever is in the UEFI
partition are replicated across the multiple raid1 disks, instead of
having to remember to copy what ever is in there manually to ach disk?

Jon.

Re: UEFI and mdadm questions.

[Anthonys Lists]

On 01/10/2014 17:33, Wilson, Jonathan wrote:

I've just been struggling with exactly this trying to move my gentoo 
system on to raid ...
>  From what I can tell with UEFI I need to set up a UEFI partition with a
> FAT format.

I believe so, yes.
>
> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
> Raid 1 40G.
>
> Obviously Grub installs to the mbr, and then installs a bit into
> Biosboot which can read raids, hence it can read and boot from /boot.

Is this grub1, or grub2? I gather grub1 actually CAN'T read raid, which 
is why you need to use raid version 0.9. Grub then thinks the disk is 
plain ext4 or whatever, and reads it fine. grub2 actually handles raid, 
and therefore is happy with newer raids like 1.2
>
> Further, from what I can tell, into the UEFI partition can go either a
> kernel & initramfs with UEFI support, or a "loader" that then loads the
> kernel.
>
> What I am unsure about are...
>
> 1)  Can the loader/kernel understand md raid? so the / can be in a bog
> standard raid1 v1.2?

Don't think so. Apparently the kernel can NOT put a raid array together 
so you have a catch-22 - the kernel needs the raid in order to start 
user-space, but it needs user-space in order to find the raid ... :-(

So you need an initramfs to solve the conundrum. And of course, grub1 
doesn't work with uefi :-(
>
> 2) I'm guessing I would no longer need the /boot as that would be
> replaced by what ever was in the UEFI partition?

You do need /boot - not least so grub2 can find its config file (that's 
not quite true, but close enough...)

>
> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
> changes occur they are replicated to the drives I have set up as /boot
> raid 1, and I installed the mbr portion of boot loader manually on each
> disk and tested pulling one, and then booting from another.. it
> worked :-)

In the world of grub2/raid1.2 I wish things looked that simple. From my 
"badly burned novice" viewpoint, I don't think it is.

> So I would like to keep things, if not simple, at least less likely to
> have problems because I forgot to install duplicates of the UEFI on all
> the disks... so can I use a .90v raid1 on the UEFI partition, then
> format it as fat... so that all copies of what ever is in the UEFI
> partition are replicated across the multiple raid1 disks, instead of
> having to remember to copy what ever is in there manually to ach disk?

Interesting idea ... but I get the impression that once you've set up 
the uefi partition and got it working, you shouldn't normally ever need 
to go back to it. This is, I think, one of those cases you need to set 
up a crib sheet and not do anything without it, as you'll forget from 
one time to the next (which is why, I think, you want to use raid :-) 
Personally, I *wouldn't* want to use raid, on the assumption that I'm 
going to screw it up and  don't want a mirror trashing disk2 as I trash 
disk1.

Cheers,
Wol

Re: UEFI and mdadm questions.

[Chris Murphy]

On Oct 1, 2014, at 12:33 PM, "Wilson, Jonathan" <piercing_male@hotmail.com> wrote:

> From what I can tell with UEFI I need to set up a UEFI partition with a
> FAT format.

It's a particular kind of FAT, that's defined as EFI FAT, the idea being that if the originating FAT ever changes, EFI FAT won't.

> 
> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
> Raid 1 40G.
> 
> Obviously Grub installs to the mbr, and then installs a bit into
> Biosboot which can read raids, hence it can read and boot from /boot.

BIOSboot applies to GPT disks on BIOS computers, not MBR. On MBR disks, the GRUB stage1 code jumps to stage2 code in the MBR gap which is the region between the MBR and the first partition's starting LBA.

> 
> Further, from what I can tell, into the UEFI partition can go either a
> kernel & initramfs with UEFI support, or a "loader" that then loads the
> kernel.

No. An OSLoader is required, it's an EFI application. Its job is to load a kernel and initramfs. The kernel and initramfs could be on the ESP (EFI System partition) but this is fraught with limitations. The expectation is that the kernel and initramfs are on some other partition of the same disk. Of course if you're using GRUB it doesn't care and will find a kernel/initramfs off another disk also, or even off md raid.

In the case of GRUB, it directly understands pretty much every filesystem used on linux, so it can read kernel+initramfs from ext4, or ext4 on mdraid, or on LVM, or on Btrfs (including multiple devices).

> 
> What I am unsure about are... 
> 
> 1)  Can the loader/kernel understand md raid? so the / can be in a bog
> standard raid1 v1.2?

If the EFI OSLoader is GRUB2, then yes.


> 
> 2) I'm guessing I would no longer need the /boot as that would be
> replaced by what ever was in the UEFI partition?

The ESP isn't a replacement for /boot. But you can have a unified /boot and / on a single file system, and GRUB2 can boot from it, including if that volume is on an md device. GRUB2 even boots off md raid6 (it's sorta crazy and badass at the same time, but in my testing it does work with the BIOS being the limiting factor as to how many drives are recognized at this stage).



> 
> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
> changes occur they are replicated to the drives I have set up as /boot
> raid 1, and I installed the mbr portion of boot loader manually on each
> disk and tested pulling one, and then booting from another.. it
> worked :-)
> So I would like to keep things, if not simple, at least less likely to
> have problems because I forgot to install duplicates of the UEFI on all
> the disks… so can I use a .90v raid1 on the UEFI partition,

That solution isn't that simple because it requires too many things that aren't supported by any distribution. And also it increases the chances of corruption because of course the EFI firmware only sees these partition/volumes as single devices, not as md devices that need to be assembled first into a logical device. Some EFI firmwares do reportedly write to the ESP, so if they write to one and not the other, while also not changing the metadata on either partition, there's no way to know which one is valid now. So that's just a mess. And even if your firmware doesn't write to the ESP ever, what about dual boot? That too would be proscribed because the ESP is shared among multiple OS's. So the idea is just way off the rails of anything standardized or widely supported. Will it work? Yes. But do you want to support something this non-standard?

An alternative is a single static grub.cfg on the EFI System partition that points to the real grub.cfg on an mdraid1 / or /boot. Since the ESP's are identical, and never need updating again, this is much safer. You can also dispense with this ridiculously bad idea most all linux distros have right now of persistently mounting the ESP at /boot/efi (and persistently mount it rw as well! really bad!).

The way to do this is with the grub configfile command. The most straightforward way to do it is let grub-mkconfig create a grub.cfg for you, and then copy past the parts that tell GRUB how to find /. For example it needs to know the mduuid so it knows there's an mdraid volume to assembled, and also the volume uuid so it knows what filesystem volume it should find the file one, and then "configfile /boot/grub/grub.cfg" and it'll load that grub.cfg. And it's that grub.cfg that your system should update - of course you have to figure out how your system knows to update the grub.cfg so that it does it correctly without stepping on your ESP grub.cfg (for one comment out the /boot/efi line in fstab and then unmount the ESP). This is distribution specific.

Either way it's a bit complicated to figure out initially.


Chris Murphy

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UEFI and mdadm questions.

[Phil Turmel]

On 10/03/2014 01:04 AM, Chris Murphy wrote:
> 
> On Oct 1, 2014, at 12:33 PM, "Wilson, Jonathan"
> <piercing_male@hotmail.com> wrote:
> 
>> From what I can tell with UEFI I need to set up a UEFI partition
>> with a FAT format.
> 
> It's a particular kind of FAT, that's defined as EFI FAT, the idea
> being that if the originating FAT ever changes, EFI FAT won't.
> 
>> 
>> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M
>> and / Raid 1 40G.
>> 
>> Obviously Grub installs to the mbr, and then installs a bit into 
>> Biosboot which can read raids, hence it can read and boot from
>> /boot.
> 
> BIOSboot applies to GPT disks on BIOS computers, not MBR. On MBR
> disks, the GRUB stage1 code jumps to stage2 code in the MBR gap which
> is the region between the MBR and the first partition's starting
> LBA.
> 
>> 
>> Further, from what I can tell, into the UEFI partition can go
>> either a kernel & initramfs with UEFI support, or a "loader" that
>> then loads the kernel.
> 
> No. An OSLoader is required, it's an EFI application. Its job is to
> load a kernel and initramfs. The kernel and initramfs could be on the
> ESP (EFI System partition) but this is fraught with limitations. The
> expectation is that the kernel and initramfs are on some other
> partition of the same disk. Of course if you're using GRUB it doesn't
> care and will find a kernel/initramfs off another disk also, or even
> off md raid.

An option to consider is to compile a kernel using the EFI stub option,
a pre-set command line, and an embedded initramfs.  Then the kernel can
boot directly from the EFI FAT partition with *no bootloader*.  The
embedded initramfs can support any raid/lvm/partitioning scheme
what-so-ever.

http://kroah.com/log/blog/2013/09/02/booting-a-self-signed-linux-kernel/

The cryptography is optional if you just want to boot really fast.

If your BIOS can be configured to try multiple boot images, it should be
possible to have true raid fallback without using motherboard or
hardware raid.  (Set up md raid1 with metadata v1.0 of multiple copies
of the EFI FAT partition.)  I've been meaning to try this....

Phil

Re: UEFI and mdadm questions.

[Chris Murphy]

On Oct 5, 2014, at 2:18 PM, Phil Turmel <philip@turmel.org> wrote:

> On 10/03/2014 01:04 AM, Chris Murphy wrote:
>> 
>> No. An OSLoader is required, it's an EFI application. Its job is to
>> load a kernel and initramfs. The kernel and initramfs could be on the
>> ESP (EFI System partition) but this is fraught with limitations. The
>> expectation is that the kernel and initramfs are on some other
>> partition of the same disk. Of course if you're using GRUB it doesn't
>> care and will find a kernel/initramfs off another disk also, or even
>> off md raid.
> 
> An option to consider is to compile a kernel using the EFI stub option,
> a pre-set command line, and an embedded initramfs.  Then the kernel can
> boot directly from the EFI FAT partition with *no bootloader*.  The
> embedded initramfs can support any raid/lvm/partitioning scheme
> what-so-ever.

Sure but then it means the kernel+initramfs is on FAT32. Ick. And there's no advantage if using maintstream storage options like md raid, lvm, btrfs, xfs, ext234, reiserfs, ufs, zfs, hfs+, ntfs and more, because GRUB2 already understands these things so the kernel can be located on them.

For appliances, there could be some advantage since there'd only be one kernel present at a time, no fallback necessary. My Android phone, for example, has 35 partitions.

> 
> If your BIOS can be configured to try multiple boot images, it should be
> possible to have true raid fallback without using motherboard or
> hardware raid.  (Set up md raid1 with metadata v1.0 of multiple copies
> of the EFI FAT partition.)  I've been meaning to try this….

Problems with this: a.) new Windows 8 hardware might require you boot Windows to get to the feature enabling the firmware setup, because on such hardware USB isn't initialized by default.
http://mjg59.dreamwidth.org/24869.html

I don't know why we don't have free software to initiate this, but I haven't come across it so far.

b.) There's no guarantee the firmware won't write to the ESP, thus rendering the individual md raid members out of sync and without their metadata being updated, i.e. in effect, the logical device they become later, is corrupt. Separately they aren't corrupt, merely out of sync, but you don't have an obvious way of knowing which one.

c.) strictly speaking any partition with mdadm metadata should have the linux raid partition type GUID set; not the EFI System partition type GUID. Those GUIDs are mutually exclusive.

This is why I'm still not a fan of using mdadm to raid1 an EFI System partition.


Chris Murphy

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UEFI and mdadm questions.

[Francis Moreau]

Hello,

On 10/01/2014 06:33 PM, Wilson, Jonathan wrote:
> From what I can tell with UEFI I need to set up a UEFI partition with a
> FAT format.
> 
> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
> Raid 1 40G.
> 
> Obviously Grub installs to the mbr, and then installs a bit into
> Biosboot which can read raids, hence it can read and boot from /boot.
> 
> Further, from what I can tell, into the UEFI partition can go either a
> kernel & initramfs with UEFI support, or a "loader" that then loads the
> kernel.
> 
> What I am unsure about are... 
> 
> 1)  Can the loader/kernel understand md raid? so the / can be in a bog
> standard raid1 v1.2?
> 
> 2) I'm guessing I would no longer need the /boot as that would be
> replaced by what ever was in the UEFI partition?
> 
> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
> changes occur they are replicated to the drives I have set up as /boot
> raid 1, and I installed the mbr portion of boot loader manually on each
> disk and tested pulling one, and then booting from another.. it
> worked :-)
> So I would like to keep things, if not simple, at least less likely to
> have problems because I forgot to install duplicates of the UEFI on all
> the disks... so can I use a .90v raid1 on the UEFI partition, then
> format it as fat... so that all copies of what ever is in the UEFI
> partition are replicated across the multiple raid1 disks, instead of
> having to remember to copy what ever is in there manually to ach disk?
> 

There was already a post on that subject you can find here:
http://comments.gmane.org/gmane.linux.raid/45611

It might answer some of your questions.

Bye.

Re: UEFI and mdadm questions.

[Francis Moreau]

Hello Chris,

On 10/03/2014 07:04 AM, Chris Murphy wrote:
> 
> On Oct 1, 2014, at 12:33 PM, "Wilson, Jonathan" <piercing_male@hotmail.com> wrote:
> 
>> From what I can tell with UEFI I need to set up a UEFI partition with a
>> FAT format.
> 
> It's a particular kind of FAT, that's defined as EFI FAT, the idea being that if the originating FAT ever changes, EFI FAT won't.
> 
>>
>> On my current BIOS system I have a Biosboot 1M, /boot Raid1 200M and /
>> Raid 1 40G.
>>
>> Obviously Grub installs to the mbr, and then installs a bit into
>> Biosboot which can read raids, hence it can read and boot from /boot.
> 
> BIOSboot applies to GPT disks on BIOS computers, not MBR. On MBR disks, the GRUB stage1 code jumps to stage2 code in the MBR gap which is the region between the MBR and the first partition's starting LBA.
> 
>>
>> Further, from what I can tell, into the UEFI partition can go either a
>> kernel & initramfs with UEFI support, or a "loader" that then loads the
>> kernel.
> 
> No. An OSLoader is required, it's an EFI application. Its job is to load a kernel and initramfs. The kernel and initramfs could be on the ESP (EFI System partition) but this is fraught with limitations. The expectation is that the kernel and initramfs are on some other partition of the same disk. Of course if you're using GRUB it doesn't care and will find a kernel/initramfs off another disk also, or even off md raid.
> 

I think the kernel can be compiled as an EFI application, therefore
OSLoader is not always required. The advantage of this is that you can
get rid of grub entirely (cool !). But the downside is that you can't
customize/edit the kernel command line at boot time.

> In the case of GRUB, it directly understands pretty much every filesystem used on linux, so it can read kernel+initramfs from ext4, or ext4 on mdraid, or on LVM, or on Btrfs (including multiple devices).
> 
>>
>> What I am unsure about are... 
>>
>> 1)  Can the loader/kernel understand md raid? so the / can be in a bog
>> standard raid1 v1.2?
> 
> If the EFI OSLoader is GRUB2, then yes.
> 
> 
>>
>> 2) I'm guessing I would no longer need the /boot as that would be
>> replaced by what ever was in the UEFI partition?
> 
> The ESP isn't a replacement for /boot. But you can have a unified /boot and / on a single file system, and GRUB2 can boot from it, including if that volume is on an md device. GRUB2 even boots off md raid6 (it's sorta crazy and badass at the same time, but in my testing it does work with the BIOS being the limiting factor as to how many drives are recognized at this stage).
> 
> 
> 
>>
>> 3) as my "/boot" is currently in a raid 1 my life is simple, should any
>> changes occur they are replicated to the drives I have set up as /boot
>> raid 1, and I installed the mbr portion of boot loader manually on each
>> disk and tested pulling one, and then booting from another.. it
>> worked :-)
>> So I would like to keep things, if not simple, at least less likely to
>> have problems because I forgot to install duplicates of the UEFI on all
>> the disks… so can I use a .90v raid1 on the UEFI partition,
> 
> That solution isn't that simple because it requires too many things that aren't supported by any distribution. And also it increases the chances of corruption because of course the EFI firmware only sees these partition/volumes as single devices, not as md devices that need to be assembled first into a logical device. Some EFI firmwares do reportedly write to the ESP, so if they write to one and not the other, while also not changing the metadata on either partition, there's no way to know which one is valid now. So that's just a mess. And even if your firmware doesn't write to the ESP ever, what about dual boot? That too would be proscribed because the ESP is shared among multiple OS's. So the idea is just way off the rails of anything standardized or widely supported. Will it work? Yes. But do you want to support something this non-standard?
> 
> An alternative is a single static grub.cfg on the EFI System partition that points to the real grub.cfg on an mdraid1 / or /boot. Since the ESP's are identical, and never need updating again, this is much safer. You can also dispense with this ridiculously bad idea most all linux distros have right now of persistently mounting the ESP at /boot/efi (and persistently mount it rw as well! really bad!).
> 
> The way to do this is with the grub configfile command. The most straightforward way to do it is let grub-mkconfig create a grub.cfg for you, and then copy past the parts that tell GRUB how to find /. For example it needs to know the mduuid so it knows there's an mdraid volume to assembled, and also the volume uuid so it knows what filesystem volume it should find the file one, and then "configfile /boot/grub/grub.cfg" and it'll load that grub.cfg. And it's that grub.cfg that your system should update - of course you have to figure out how your system knows to update the grub.cfg so that it does it correctly without stepping on your ESP grub.cfg (for one comment out the /boot/efi line in fstab and then unmount the ESP). This is distribution specific.
> 
> Either way it's a bit complicated to figure out initially.
> 

After thinking about it for a while, I'm not sure about the point of
having ESP mirrored or at least I'm not sure it really worths the
trouble: ESP and /boot are very static partitions and should be written
mostly during kernel upgrade. So even if something bad happens in those
partitions, the system should be able to run without any issues.

The annoying part is that if you decide/have to reboot and ESP is
damaged you won't be able to boot your system again. But that said if
you created initially a copy of /boot on each disk, you should be able
to boot again even though those copies are not uptodate.

Thanks.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UEFI and mdadm questions.

[Phil Turmel]

On 10/05/2014 04:22 PM, Chris Murphy wrote:
> 
> On Oct 5, 2014, at 2:18 PM, Phil Turmel <philip@turmel.org> wrote:

[trim /]

>> If your BIOS can be configured to try multiple boot images, it
>> should be possible to have true raid fallback without using
>> motherboard or hardware raid.  (Set up md raid1 with metadata v1.0
>> of multiple copies of the EFI FAT partition.)  I've been meaning to
>> try this….
> 
> Problems with this: a.) new Windows 8 hardware might require you boot
> Windows to get to the feature enabling the firmware setup, because on
> such hardware USB isn't initialized by default.
> http://mjg59.dreamwidth.org/24869.html
> 
> I don't know why we don't have free software to initiate this, but I
> haven't come across it so far.

Good to know, but totally immaterial to the boot sequence I'm
recommending.  Boot linux of of an EFI FAT and let linux initialize the
USB hardware in its own good time.

Matthew Garrett's post is really all about how to get linux into the
Win8 box in the first place.  Once there, manipulate the boot sequence
as you please.

> b.) There's no guarantee the firmware won't write to the ESP, thus
> rendering the individual md raid members out of sync and without
> their metadata being updated, i.e. in effect, the logical device they
> become later, is corrupt. Separately they aren't corrupt, merely out
> of sync, but you don't have an obvious way of knowing which one.

This is a very good point.  In fact, I withdraw my recommendation to
raid these partitions.  Simply have one on every disk the BIOS could
possibly boot from, and place an EFI bootable kernel in each one (with
embedded initramfs).

> c.) strictly speaking any partition with mdadm metadata should have
> the linux raid partition type GUID set; not the EFI System partition
> type GUID. Those GUIDs are mutually exclusive.

The former is not true at all--mdadm does not care *at all* what
partition types are set.  Grub might care, but it's moot if you don't
use Grub.  :-)

> This is why I'm still not a fan of using mdadm to raid1 an EFI System
> partition.

One further point:  the failure decision tree is nicer if you boot
directly into a kernel.

1) Bios locates and attempts to boot from 1st configured kernel image
2a) Corrupted image or other disk error blocks complete load of kernel
image--bios moves to next EFI choice (possibly on a different disk).
2b) Successful EFI kernel load, boot encounters missing/corrupt root
FS--kernel drops to initramfs rescue shell

versus:

1) Bios locates and attempts to boot from 1st configured grub image
2a) Corrupted image or other disk error blocks complete load of
grub--bios moves to next EFI choice (possibly on a different disk).
2b) Successful EFI grub load, grub encounters corrupt config or grub
module--drop to grub shell
2c) Successful EFI grub load, kernel & initramfs load by grub, boot
encounters missing/corrupt root FS--kernel drops to initramfs rescue shell

I haven't had time to set it up yet, but the clear reduction in points
of failure is compelling.  Faster boot is just icing on the cake.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UEFI and mdadm questions.

[Chris Murphy]

On Oct 6, 2014, at 11:58 AM, Phil Turmel <philip@turmel.org> wrote:

> On 10/05/2014 04:22 PM, Chris Murphy wrote:
>> 
>> On Oct 5, 2014, at 2:18 PM, Phil Turmel <philip@turmel.org> wrote:
> 
> [trim /]
> 
>>> If your BIOS can be configured to try multiple boot images, it
>>> should be possible to have true raid fallback without using
>>> motherboard or hardware raid.  (Set up md raid1 with metadata v1.0
>>> of multiple copies of the EFI FAT partition.)  I've been meaning to
>>> try this….
>> 
>> Problems with this: a.) new Windows 8 hardware might require you boot
>> Windows to get to the feature enabling the firmware setup, because on
>> such hardware USB isn't initialized by default.
>> http://mjg59.dreamwidth.org/24869.html
>> 
>> I don't know why we don't have free software to initiate this, but I
>> haven't come across it so far.
> 
> Good to know, but totally immaterial to the boot sequence I'm
> recommending.  Boot linux of of an EFI FAT and let linux initialize the
> USB hardware in its own good time.

Sure but it means that you can't use the firmware's boot manager to choose anything else. Any other kernel (or copy) only happens in the course of the firmware's fallback mechanisms.

> 
> Matthew Garrett's post is really all about how to get linux into the
> Win8 box in the first place.  Once there, manipulate the boot sequence
> as you please.
> 
>> b.) There's no guarantee the firmware won't write to the ESP, thus
>> rendering the individual md raid members out of sync and without
>> their metadata being updated, i.e. in effect, the logical device they
>> become later, is corrupt. Separately they aren't corrupt, merely out
>> of sync, but you don't have an obvious way of knowing which one.
> 
> This is a very good point.  In fact, I withdraw my recommendation to
> raid these partitions.  Simply have one on every disk the BIOS could
> possibly boot from, and place an EFI bootable kernel in each one (with
> embedded initramfs).

Well, to me that seems more esoteric than just having kernel+initramfs on a conventional md raid1 /boot volume, and having a static (never upgraded or modified) GRUB2 or syslinux point to the modifiable configuration file (basically load a 2nd config file) in the usual location also on /boot. So now kernel upgrades are normal, and the user gets to regress to older kernels on demand should it be necessary and they also get resilient boot.

> 
>> c.) strictly speaking any partition with mdadm metadata should have
>> the linux raid partition type GUID set; not the EFI System partition
>> type GUID. Those GUIDs are mutually exclusive.
> 
> The former is not true at all--mdadm does not care *at all* what
> partition types are set.  Grub might care, but it's moot if you don't
> use Grub.  :-)

The partition type isn't for mdadm, it's for other things that might otherwise modify the partition if it's misidentified as an EFI System partition. And it would be a misidentification to set the partition type GUID to EFI System partition, because first this example partition is an md member, and only after assembly is it an EFI system partition. So I still consider them mutually exclusive.

> 
>> This is why I'm still not a fan of using mdadm to raid1 an EFI System
>> partition.
> 
> One further point:  the failure decision tree is nicer if you boot
> directly into a kernel.
> 
> 1) Bios locates and attempts to boot from 1st configured kernel image
> 2a) Corrupted image or other disk error blocks complete load of kernel
> image--bios moves to next EFI choice (possibly on a different disk).
> 2b) Successful EFI kernel load, boot encounters missing/corrupt root
> FS--kernel drops to initramfs rescue shell
> 
> versus:
> 
> 1) Bios locates and attempts to boot from 1st configured grub image
> 2a) Corrupted image or other disk error blocks complete load of
> grub--bios moves to next EFI choice (possibly on a different disk).
> 2b) Successful EFI grub load, grub encounters corrupt config or grub
> module--drop to grub shell
> 2c) Successful EFI grub load, kernel & initramfs load by grub, boot
> encounters missing/corrupt root FS--kernel drops to initramfs rescue shell
> 
> I haven't had time to set it up yet, but the clear reduction in points
> of failure is compelling.  Faster boot is just icing on the cake.

But we have a lot of experience with the latter, as that's how it's always been on BIOS systems. In a way it's more complicated because BIOS itself didn't really have a meaningful (or complicated) fallback mechanism, that was just up to the boot manager which we (FOSS) control. But with UEFI we don't control this, so exactly how the firmware behaves in failure cases actually needs to be tested on a firmware by firmware basis.

The other thing is this arrangement isn't supported by any distro currently. So setting it up and maintaining is pretty cumbersome.


Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html