How does md(adm) work with fake-raid ?

How does md(adm) work with fake-raid ?

[Francis Moreau]

Hello,

Sorry if the question is stupid but I'm a rookie in md things, but I'd
like to understand the big picture here.

I've been told to use mdadm whenever possible even if my raid is
handled by the bios (fake raid) which use the ddf metadata format.
(unfortunately it seems that I can't desactive this fake raid in
favour of linux soft raid). It's RAID1 BTW.

So my question is rather simple: in my understanding the bios is doing
the mirroring, but when setting up the md device, linux (kernel or
userspace, I don't really know) also handles the mirroring for RAID1
personnality. Is Linux clever enough to see that the mirroring is done
by the bios in my case ?

Could anybody teach me the big picture ?

Thanks a lot.
--
Francis

Re: How does md(adm) work with fake-raid ?

[Martin Wilck]

On 07/18/2013 10:37 PM, Francis Moreau wrote:
> Hello,
> 
> Sorry if the question is stupid but I'm a rookie in md things, but I'd
> like to understand the big picture here.
> 
> I've been told to use mdadm whenever possible even if my raid is
> handled by the bios (fake raid) which use the ddf metadata format.
> (unfortunately it seems that I can't desactive this fake raid in
> favour of linux soft raid). It's RAID1 BTW.
> 
> So my question is rather simple: in my understanding the bios is doing
> the mirroring, but when setting up the md device, linux (kernel or
> userspace, I don't really know) also handles the mirroring for RAID1
> personnality. Is Linux clever enough to see that the mirroring is done
> by the bios in my case ?
> 
> Could anybody teach me the big picture ?

Fake RAID uses a part of every disk to record information about the RAID
arrays. This is called meta data, and your BIOS uses it for setting up
the drives.

Under Linux, first you need a low level SATA or SAS driver that detects
your physical drives, e.g. the ahci driver.

md can then detect the DDF meta data on your disk just like the BIOS,
assemble the array(s), mirror the data, and do other RAID operations.

Distributions can set this up automatically. Currently most distros
don't do this for DDF (they do it only for fake RAID using the Intel
Matrix (IMSM) format). For DDF, for historical reasons, most
distributions will setup a mapping using dmraid (device mapper based
mirroring). That will also basically work, but it isn't a
fully-functional RAID implementation such as MD. The magic to set up
either MD or dmraid automatically as disks are detected is hidden in the
distro's udev rules, and possibly in the distro's installer logic.

The DDF functionality in mdadm is not quite as complete as for other
meta data formats, in particular IMSM. I am currently working on
improving it, because I want to use just a similar fake RAID under Linux
as you, and fully benefit from MD's feature set.

Before doing real work with DDF on your system, make sure that the BIOS
still detects your drives correctly after MD/DDF has written its meta
data (this happens when an array is shut down, for example).
There is a small but non-zero risk that the BIOS may reject the
configuration as written by MD. The same may happen e.g. for a Windows
fake RAID driver.

If you experience any problems, be sure to report them here.

DDF metadata is always at the end of the disks. Thus you can save your
configuration by backing up the last sectors. The size of the meta data
is at least 32MB but may be larger depending on the BIOS implementation.

Hope this helps.
Martin

> 
> Thanks a lot.
> --
> Francis
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How does md(adm) work with fake-raid ?

[Dmitrijs Ledkovs]

On 18 July 2013 23:03, Martin Wilck <mwilck@arcor.de> wrote:
> On 07/18/2013 10:37 PM, Francis Moreau wrote:
>> Hello,
>>
>> Sorry if the question is stupid but I'm a rookie in md things, but I'd
>> like to understand the big picture here.
>>
>> I've been told to use mdadm whenever possible even if my raid is
>> handled by the bios (fake raid) which use the ddf metadata format.
>> (unfortunately it seems that I can't desactive this fake raid in
>> favour of linux soft raid). It's RAID1 BTW.
>>
>> So my question is rather simple: in my understanding the bios is doing
>> the mirroring, but when setting up the md device, linux (kernel or
>> userspace, I don't really know) also handles the mirroring for RAID1
>> personnality. Is Linux clever enough to see that the mirroring is done
>> by the bios in my case ?
>>
>> Could anybody teach me the big picture ?
>
> Fake RAID uses a part of every disk to record information about the RAID
> arrays. This is called meta data, and your BIOS uses it for setting up
> the drives.
>
> Under Linux, first you need a low level SATA or SAS driver that detects
> your physical drives, e.g. the ahci driver.
>
> md can then detect the DDF meta data on your disk just like the BIOS,
> assemble the array(s), mirror the data, and do other RAID operations.
>
> Distributions can set this up automatically. Currently most distros
> don't do this for DDF (they do it only for fake RAID using the Intel
> Matrix (IMSM) format). For DDF, for historical reasons, most
> distributions will setup a mapping using dmraid (device mapper based
> mirroring). That will also basically work, but it isn't a
> fully-functional RAID implementation such as MD. The magic to set up
> either MD or dmraid automatically as disks are detected is hidden in the
> distro's udev rules, and possibly in the distro's installer logic.
>

There patches posted to debian bug tracker to enable using mdadm in
the installer to assemble/setup IMSM/DDF raid arrays, and thus using
mdadm.
I haven't integrated those, but am planning to work on merging them soon.

At the moment dmraid is used by default for both IMSM/DDF on Debian/Ubuntu.

My experience with these fakeraid arrays is very limitted, and I'd
want to enquire of proper migration strategies from dmraid to mdadm.
Whilst looking at the udev rules, at the moment, i have disable
IMSM/DDF from assembly in mdadm udev rules, because dmraid has a nice
property of "activating anything it finds".
I suppose having both mdadm & dmraid racing to activate those drives
wouldn't be nice.

How would one migrate from dmraid to mdadm? I was pondering about
drastic measures: patch out ISMS support out of dmraid, make dmraid
package depend on mdadm and make mdadm activate ISMS drives by
default.
But that sounds harsh, as I wouldn't want to cripple dmraid package
for those who still prefer to use it.

Are there distributions which switched to mdadm by default for ISMS? Suse?!

Regards,

Dmitrijs.

Re: How does md(adm) work with fake-raid ?

[Jiang, Dave]

On Thu, 2013-07-18 at 23:35 +0000, Dmitrijs Ledkovs wrote:
> On 18 July 2013 23:03, Martin Wilck <mwilck@arcor.de> wrote:
> > On 07/18/2013 10:37 PM, Francis Moreau wrote:
> >> Hello,
> >>
> >> Sorry if the question is stupid but I'm a rookie in md things, but I'd
> >> like to understand the big picture here.
> >>
> >> I've been told to use mdadm whenever possible even if my raid is
> >> handled by the bios (fake raid) which use the ddf metadata format.
> >> (unfortunately it seems that I can't desactive this fake raid in
> >> favour of linux soft raid). It's RAID1 BTW.
> >>
> >> So my question is rather simple: in my understanding the bios is doing
> >> the mirroring, but when setting up the md device, linux (kernel or
> >> userspace, I don't really know) also handles the mirroring for RAID1
> >> personnality. Is Linux clever enough to see that the mirroring is done
> >> by the bios in my case ?
> >>
> >> Could anybody teach me the big picture ?
> >
> > Fake RAID uses a part of every disk to record information about the RAID
> > arrays. This is called meta data, and your BIOS uses it for setting up
> > the drives.
> >
> > Under Linux, first you need a low level SATA or SAS driver that detects
> > your physical drives, e.g. the ahci driver.
> >
> > md can then detect the DDF meta data on your disk just like the BIOS,
> > assemble the array(s), mirror the data, and do other RAID operations.
> >
> > Distributions can set this up automatically. Currently most distros
> > don't do this for DDF (they do it only for fake RAID using the Intel
> > Matrix (IMSM) format). For DDF, for historical reasons, most
> > distributions will setup a mapping using dmraid (device mapper based
> > mirroring). That will also basically work, but it isn't a
> > fully-functional RAID implementation such as MD. The magic to set up
> > either MD or dmraid automatically as disks are detected is hidden in the
> > distro's udev rules, and possibly in the distro's installer logic.
> >
> 
> There patches posted to debian bug tracker to enable using mdadm in
> the installer to assemble/setup IMSM/DDF raid arrays, and thus using
> mdadm.
> I haven't integrated those, but am planning to work on merging them soon.
> 
> At the moment dmraid is used by default for both IMSM/DDF on Debian/Ubuntu.
> 
> My experience with these fakeraid arrays is very limitted, and I'd
> want to enquire of proper migration strategies from dmraid to mdadm.
> Whilst looking at the udev rules, at the moment, i have disable
> IMSM/DDF from assembly in mdadm udev rules, because dmraid has a nice
> property of "activating anything it finds".
> I suppose having both mdadm & dmraid racing to activate those drives
> wouldn't be nice.
> 
> How would one migrate from dmraid to mdadm? I was pondering about
> drastic measures: patch out ISMS support out of dmraid, make dmraid
> package depend on mdadm and make mdadm activate ISMS drives by
> default.
> But that sounds harsh, as I wouldn't want to cripple dmraid package
> for those who still prefer to use it.
> 
> Are there distributions which switched to mdadm by default for ISMS? Suse?!

AFAIK for IMSM, Fedora, SLES11sp2, and RHEL 6.x all are using MDRAID and
have been for quite some times. 

Re: How does md(adm) work with fake-raid ?

[Martin Wilck]

On 07/19/2013 01:35 AM, Dmitrijs Ledkovs wrote:

> My experience with these fakeraid arrays is very limitted, and I'd
> want to enquire of proper migration strategies from dmraid to mdadm.
> Whilst looking at the udev rules, at the moment, i have disable
> IMSM/DDF from assembly in mdadm udev rules, because dmraid has a nice
> property of "activating anything it finds".
> I suppose having both mdadm & dmraid racing to activate those drives
> wouldn't be nice.

It can be done in udev. Have a look at the udev rules in CentOS6, for
example; they do it for IMSM and it should be possible for DDF as well.
You can use kernel command line parameters ("nodmraid") to define what
RAID solution you prefer.

Martin

Re: How does md(adm) work with fake-raid ?

[Francis Moreau]

Hi,

On Thu, Jul 18, 2013 at 11:08 PM, Stefan /*St0fF*/ Hübner
<st0ff.npl@googlemail.com> wrote:
> Am 18.07.2013 22:37, schrieb Francis Moreau:
>> Hello,
>>
>> Sorry if the question is stupid but I'm a rookie in md things, but I'd
>> like to understand the big picture here.
>
> Probably a bit of good googling would have given your answers right away.
>

Not really. You think such because you might have missed the point of
my question which is about how does coexists with bios raid.

>>
>> I've been told to use mdadm whenever possible even if my raid is
>> handled by the bios (fake raid) which use the ddf metadata format.
>> (unfortunately it seems that I can't desactive this fake raid in
>> favour of linux soft raid). It's RAID1 BTW.
>
> There are two types of fakeraid that I know of.  Intel and MegaRAID/LSI
> emulation.  Those are NOT handled by the BIOS, the boot process is
> handled by the special op-rom integrated into the BIOS.

Well what's the difference ? Of course a bios is not a piece of
hardware with a dedicated cpu. So when I said it's handled by the
bios, I obviously meant that there's a software stored in the firmware
which is executed by my the processor and this soft can be configured
through the bios.

>  After any
> kernel is booted, all RAID processing shall be done by the respective
> driver.  And none use DDF afaik.  Intel uses its own metadata format
> (referred to as IMSM), LSI also uses its own format (those fakeraid
> drives can be attached to f.e. a MegaRAID 9266 controller and your RAID
> is then recognized - those 9266 f.e. can create DDF arrays).

You probably have missed a lot more.

Bye,
--
Francis
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How does md(adm) work with fake-raid ?

[Francis Moreau]

Hello,

On Fri, Jul 19, 2013 at 12:02 AM, Martin Wilck <mwilck@arcor.de> wrote:
>
> Fake RAID uses a part of every disk to record information about the RAID
> arrays. This is called meta data, and your BIOS uses it for setting up
> the drives.
>
> Under Linux, first you need a low level SATA or SAS driver that detects
> your physical drives, e.g. the ahci driver.
>
> md can then detect the DDF meta data on your disk just like the BIOS,
> assemble the array(s), mirror the data, and do other RAID operations.
>

Yes, but my question is about how md cooperate with the BIOS in this case ?

> Distributions can set this up automatically. Currently most distros
> don't do this for DDF (they do it only for fake RAID using the Intel
> Matrix (IMSM) format). For DDF, for historical reasons, most
> distributions will setup a mapping using dmraid (device mapper based
> mirroring). That will also basically work, but it isn't a
> fully-functional RAID implementation such as MD. The magic to set up
> either MD or dmraid automatically as disks are detected is hidden in the
> distro's udev rules, and possibly in the distro's installer logic.

That's true, and I'm in the process of "fixing" the default setup
(dmraid, done during my distro installation) with MD.

>
> The DDF functionality in mdadm is not quite as complete as for other
> meta data formats, in particular IMSM. I am currently working on
> improving it, because I want to use just a similar fake RAID under Linux
> as you, and fully benefit from MD's feature set.

That's an interesting information thanks.

>
> Before doing real work with DDF on your system, make sure that the BIOS
> still detects your drives correctly after MD/DDF has written its meta
> data (this happens when an array is shut down, for example).
> There is a small but non-zero risk that the BIOS may reject the
> configuration as written by MD. The same may happen e.g. for a Windows
> fake RAID driver.

That's a good idea.

Are there any procedures described out there that I could follow to
test MD DFF ?

>
> If you experience any problems, be sure to report them here.
>

I will.

Thank you.
--
Francis

Re: How does md(adm) work with fake-raid ?

[Martin Wilck]

> Yes, but my question is about how md cooperate with the BIOS in this case ?

Not sure what you mean. When Linux runs, with or without MD, no BIOS
code is ever executed.

The only way of communication between BIOS and MD is the meta data. When
you create an array in your BIOS setup tool, The BIOS will write the
meta data. MD will read it, and when it's done, it will re-write it
(usually unchanged, unless a disk failure occurs, but there are some
fields such as sequential numbers and time stamps that change every time
the meta data is written). The BIOS will read this back the next time
your system boots.

> Are there any procedures described out there that I could follow to
> test MD DFF ?

Not really. Play around, test various RAID levels etc, whatever your
fake RAID supports. If you have a Linux or Windows driver for your fake
raid, check if it correctly detects your setup after MD had control.

Note that RAID arrays *created* by MD (with mdadm -C) will most probably
not be accepted by the BIOS (it will consider them "foreign
configurations" and not import them).

Martin

Re: How does md(adm) work with fake-raid ?

[Francis Moreau]

On Fri, Jul 19, 2013 at 8:52 PM, Martin Wilck <mwilck@arcor.de> wrote:
>> Yes, but my question is about how md cooperate with the BIOS in this case ?
>
> Not sure what you mean. When Linux runs, with or without MD, no BIOS
> code is ever executed.

Ok I think I'm missing something more fundamental here.

For exemple when writing to the disk, each write goes into the 2 disks
in case of RAID1. When Linux runs, is it either DM or MD (and
necessarily one of them) that does this work ?

For windows case, does it relies on a driver that is provided by the
RAID manufacturer ?

>
> The only way of communication between BIOS and MD is the meta data. When
> you create an array in your BIOS setup tool, The BIOS will write the
> meta data. MD will read it, and when it's done, it will re-write it
> (usually unchanged, unless a disk failure occurs, but there are some
> fields such as sequential numbers and time stamps that change every time
> the meta data is written). The BIOS will read this back the next time
> your system boots.
>
>> Are there any procedures described out there that I could follow to
>> test MD DFF ?
>
> Not really. Play around, test various RAID levels etc, whatever your
> fake RAID supports. If you have a Linux or Windows driver for your fake
> raid, check if it correctly detects your setup after MD had control.
>
> Note that RAID arrays *created* by MD (with mdadm -C) will most probably
> not be accepted by the BIOS (it will consider them "foreign
> configurations" and not import them).

Thanks for those information.
--
Francis

Re: How does md(adm) work with fake-raid ?

[Martin Wilck]

On 07/21/2013 03:41 PM, Francis Moreau wrote:
> On Fri, Jul 19, 2013 at 8:52 PM, Martin Wilck <mwilck@arcor.de> wrote:
>>> Yes, but my question is about how md cooperate with the BIOS in this case ?
>>
>> Not sure what you mean. When Linux runs, with or without MD, no BIOS
>> code is ever executed.
> 
> Ok I think I'm missing something more fundamental here.
> 
> For exemple when writing to the disk, each write goes into the 2 disks
> in case of RAID1. When Linux runs, is it either DM or MD (and
> necessarily one of them) that does this work ?

Yes.

> For windows case, does it relies on a driver that is provided by the
> RAID manufacturer ?

Exactly.

Regards
Martin