naming system of raid devices

naming system of raid devices

[c.buhtz]

In the last weeks I played around with mdadm - on VMs, Odroid-Devices,
PCs - all with Debian 10.

I observed a (for me) curious behavior about the naming of RAID
devices.

I did "--create /dev/md0" all the time.

But sometimes it results in /dev/md127.
Sometimes it is /dev/md/0.
Sometimes it switches between upgrade of the kernel image (/dev/md127
become /dev/md0 and booting fails).

Googleing this topic brings up a lot of other users discussing about
that problem.

My current solution is to ignore the /dev/*-name and mount the
device/partition by its UUID.

Another often reported "solution" is to edit the mdadm.conf. But this
is not a good solution for me. mdadm looks in the superblock and knows
(nearly) everything. Each conf-file I need to edit keeps the
possibility for errors/mistakes/faults (because I am not a sysop/admin
but a simple home-server-wannabe-admin).

My Question is how this names come up? How does mdadm, the kernel or
what ever component is responsible here, decide about the "name" of a
raid device?
And which factors influence the re-nameing of such devices (e.g.
between boot or kernel-updates)?

Re: naming system of raid devices

[Reindl Harald]

Am 01.01.21 um 16:28 schrieb c.buhtz@posteo.jp:
> In the last weeks I played around with mdadm - on VMs, Odroid-Devices,
> PCs - all with Debian 10.
> 
> I observed a (for me) curious behavior about the naming of RAID
> devices.
> 
> I did "--create /dev/md0" all the time.
> 
> But sometimes it results in /dev/md127.
> Sometimes it is /dev/md/0.
> Sometimes it switches between upgrade of the kernel image (/dev/md127
> become /dev/md0 and booting fails).
> 
> Googleing this topic brings up a lot of other users discussing about
> that problem.
> 
> My current solution is to ignore the /dev/*-name and mount the
> device/partition by its UUID.

you should do that for about 20 years anyways, no matter RAID or not

> Another often reported "solution" is to edit the mdadm.conf. But this
> is not a good solution for me. mdadm looks in the superblock and knows
> (nearly) everything. Each conf-file I need to edit keeps the
> possibility for errors/mistakes/faults (because I am not a sysop/admin
> but a simple home-server-wannabe-admin).

but how often is anything changed there?

my mdadm.conf is from 2011 and it's not even the same machine with my 
disks which are in the meantime replaced by SSD

you need to make sure that the current config is part if the initrd 
(dracut -f as example) which happens anyways when the kernel is updated 
and a new initrd is generated for the new kernel

---------------------

[root@srv-rhsoft:~]$ lsinitrd | grep mdadm
-rw-r--r--   1 root     root          315 Aug 16  2014 etc/mdadm.conf

---------------------

[root@srv-rhsoft:~]$ cat /etc/mdadm.conf
MAILADDR root
HOMEHOST localhost.localdomain
AUTO +imsm +1.x -all

ARRAY /dev/md0 level=raid1 num-devices=4 
UUID=1d691642:baed26df:1d197496:4fb00ff8
ARRAY /dev/md1 level=raid10 num-devices=4 
UUID=b7475879:c95d9a47:c5043c02:0c5ae720
ARRAY /dev/md2 level=raid10 num-devices=4 
UUID=ea253255:cb915401:f32794ad:ce0fe396
---------------------

"localhost.localdomain" is intentional, there is a second machine with 
identical UUIDs all over the place cloned by move two disks, resync on 
both and adjust runtime config like networking and so on

> My Question is how this names come up? How does mdadm, the kernel or
> what ever component is responsible here, decide about the "name" of a
> raid device?
> And which factors influence the re-nameing of such devices (e.g.
> between boot or kernel-updates)?

devices are always a problem because it's unknown when they appear, it's 
even unknown *if* they appear so nothing can wait for them

that's why you should always work with UUID

Re: naming system of raid devices

[Phil Turmel]

On 1/1/21 10:28 AM, c.buhtz@posteo.jp wrote:
> In the last weeks I played around with mdadm - on VMs, Odroid-Devices,
> PCs - all with Debian 10.
> 
> I observed a (for me) curious behavior about the naming of RAID
> devices.
> 
> I did "--create /dev/md0" all the time.
> 
> But sometimes it results in /dev/md127.
> Sometimes it is /dev/md/0.
> Sometimes it switches between upgrade of the kernel image (/dev/md127
> become /dev/md0 and booting fails).
> 
> Googleing this topic brings up a lot of other users discussing about
> that problem.
> 
> My current solution is to ignore the /dev/*-name and mount the
> device/partition by its UUID.
> 
> Another often reported "solution" is to edit the mdadm.conf. But this
> is not a good solution for me. mdadm looks in the superblock and knows
> (nearly) everything. Each conf-file I need to edit keeps the
> possibility for errors/mistakes/faults (because I am not a sysop/admin
> but a simple home-server-wannabe-admin).

Most distros incrementally assemble raid arrays during early boot, 
before the real root filesystem is mounted.  By itself, with no 
mdadm.conf guidance, mdadm will assign names to arrays starting with 
md127 and counting backwards.  And will process all arrays found.

I recommend creating an mdadm.conf file containing ARRAY entries for 
your desired setup.  Trim those lines to only have the desired name and 
UUID.

Have your distro then rebuild your initramfs (distro-specific) so the 
updated mdadm.conf is available during early boot.  Reboot to see that 
it worked.

Once you are sure it works, I also recommend adding AUTO=-all to 
mdadm.conf, so any extra arrays you might plug in temporarily won't 
auto-assemble if still plugged in during boot.

Phil

Re: naming system of raid devices

[antlists]

On 01/01/2021 15:28, c.buhtz@posteo.jp wrote:
> In the last weeks I played around with mdadm - on VMs, Odroid-Devices,
> PCs - all with Debian 10.
> 
> I observed a (for me) curious behavior about the naming of RAID
> devices.
> 
> I did "--create /dev/md0" all the time.
> 
> But sometimes it results in /dev/md127.
> Sometimes it is /dev/md/0.
> Sometimes it switches between upgrade of the kernel image (/dev/md127
> become /dev/md0 and booting fails).
> 
> Googleing this topic brings up a lot of other users discussing about
> that problem.
> 
> My current solution is to ignore the /dev/*-name and mount the
> device/partition by its UUID.
> 
> Another often reported "solution" is to edit the mdadm.conf. But this
> is not a good solution for me. mdadm looks in the superblock and knows
> (nearly) everything. Each conf-file I need to edit keeps the
> possibility for errors/mistakes/faults (because I am not a sysop/admin
> but a simple home-server-wannabe-admin).
> 
> My Question is how this names come up? How does mdadm, the kernel or
> what ever component is responsible here, decide about the "name" of a
> raid device?
> And which factors influence the re-nameing of such devices (e.g.
> between boot or kernel-updates)?
> 
The kernel and/or mdadm decides the same way it decides what drive is 
sda, sdb, etc, in that it is COMPLETELY RANDOM. Yes, the dice normally 
fall the same way and names are consistent, but numbers now normally 
always count down from 127. (I think that's so we can count down from 
32,000 if we need to... :-) For example, because I have just two, 
mirrored, drives, it doesn't matter which one appears first the arrays 
count down consistently so eg root is always md127. But that's a quirk 
of my system.

The recommended way with mdadm is to create named arrays, eg 
/dev/md/root or /dev/md/home.

Cheers,
Wol

Re: naming system of raid devices

[c.buhtz]

Thanks to you and all other answers. It helped me a lot to understand
the backgrounds.

On 2021-01-01 14:09 Phil Turmel <philip@turmel.org> wrote:
> I recommend creating an mdadm.conf file containing ARRAY entries for 
> your desired setup.  Trim those lines to only have the desired name
> and UUID.

Maybe I should open a new thread for that topic?
Please correct me, because I am not an usual admin nor an mdadm expert.

I do not want to convince you with the following! I just want to bring
up my point of view and my opinion to make it possible for you experts
to "correct" me. ;)

I do not see the advantage of creating mdadm.conf.
Via fstab I mount the devices by their UUID.
And all other information's mdadm needs to use the RAID is stored in the
superblock.

So information's in mdadm.conf would be redundant. And especially for
a non-routine home-admin like me each conf-file I modify keep the
possibility of misstakes/missconfigurations and more problems. Keeping
it as simple as possible is very important for my environment.

> Once you are sure it works, I also recommend adding AUTO=-all to 
> mdadm.conf, so any extra arrays you might plug in temporarily won't 
> auto-assemble if still plugged in during boot.

I do not understand this. What does "auto-assemble" means? You mean if
I plug in a SDD/HDD with a mdadm-created superblock?

Re: naming system of raid devices

[Wols Lists]

On 02/01/21 15:39, c.buhtz@posteo.jp wrote:
> Thanks to you and all other answers. It helped me a lot to understand
> the backgrounds.
> 
> On 2021-01-01 14:09 Phil Turmel <philip@turmel.org> wrote:
>> I recommend creating an mdadm.conf file containing ARRAY entries for 
>> your desired setup.  Trim those lines to only have the desired name
>> and UUID.
> 
> Maybe I should open a new thread for that topic?
> Please correct me, because I am not an usual admin nor an mdadm expert.
> 
> I do not want to convince you with the following! I just want to bring
> up my point of view and my opinion to make it possible for you experts
> to "correct" me. ;)
> 
> I do not see the advantage of creating mdadm.conf.
> Via fstab I mount the devices by their UUID.
> And all other information's mdadm needs to use the RAID is stored in the
> superblock.

As the guy who looks after the wiki, none of my systems (to my best
knowledge) have or have had an mdadm.conf. There's supposed to be a way
to create it automagically from the current running config, but I've
never managed to get to grips with it ...
> 
> So information's in mdadm.conf would be redundant. And especially for
> a non-routine home-admin like me each conf-file I modify keep the
> possibility of misstakes/missconfigurations and more problems. Keeping
> it as simple as possible is very important for my environment.
> 
>> Once you are sure it works, I also recommend adding AUTO=-all to 
>> mdadm.conf, so any extra arrays you might plug in temporarily won't 
>> auto-assemble if still plugged in during boot.
> 
> I do not understand this. What does "auto-assemble" means? You mean if
> I plug in a SDD/HDD with a mdadm-created superblock?
> 
It's what happens when a system boots - as each drive is recognised,
it's added to an array until the array has enough drives to function. In
other words, rather than doing an mdadm assemble command with all the
drives, it's doing an assemble with just one drive at a time saying "add
to the relevant array".

If you've disabled auto-assemble, presumably you do need mdadm.conf to
tell mdadm what to assemble, which presumably also means you can't have
root on a raid because boot needs the array to find mdadm.conf in order
to start the array ... Don't quote me on that ...

Cheers,
Wol

Re: naming system of raid devices

[Rudy Zijlstra]

On 02-01-2021 19:18, Wols Lists wrote:
> On 02/01/21 15:39, c.buhtz@posteo.jp wrote:
>>
>> I do not see the advantage of creating mdadm.conf.
>> Via fstab I mount the devices by their UUID.
>> And all other information's mdadm needs to use the RAID is stored in the
>> superblock.
> As the guy who looks after the wiki, none of my systems (to my best
> knowledge) have or have had an mdadm.conf. There's supposed to be a way
> to create it automagically from the current running config, but I've
> never managed to get to grips with it ...
It's easy and in the man page. Several options actually. I typically use:
mdadm --examine --scan > <output file>
I than check the outputfile, if i agree add it to the distro mdadm.conf 
and re-generate the initrd/initramfs or however the distro calls it.

As i often have my boot disk on an array, i want predictability... thus 
i use mdadm.conf in the initrd.

Without mdadm.conf there is no predictability, as order of disk 
detection can, and sometimes indeed does, change.
>> So information's in mdadm.conf would be redundant. And especially for
>> a non-routine home-admin like me each conf-file I modify keep the
>> possibility of misstakes/missconfigurations and more problems. Keeping
>> it as simple as possible is very important for my environment.
mdadm.conf is your friend :)
>>
>>> Once you are sure it works, I also recommend adding AUTO=-all to
>>> mdadm.conf, so any extra arrays you might plug in temporarily won't
>>> auto-assemble if still plugged in during boot.
This is the start of a philosophical discussion. In other words, 
different minds have different preferences.
>> I do not understand this. What does "auto-assemble" means? You mean if
>> I plug in a SDD/HDD with a mdadm-created superblock?
>>
> It's what happens when a system boots - as each drive is recognised,
> it's added to an array until the array has enough drives to function. In
> other words, rather than doing an mdadm assemble command with all the
> drives, it's doing an assemble with just one drive at a time saying "add
> to the relevant array".
>
> If you've disabled auto-assemble, presumably you do need mdadm.conf to
> tell mdadm what to assemble, which presumably also means you can't have
> root on a raid because boot needs the array to find mdadm.conf in order
> to start the array ... Don't quote me on that ...
nope. The suggestion here is to define the raids you need in mdadm.conf, 
and then disable auto-creation to prevent surprises from temporarily 
connected disks.

Cheers

Rudy

Re: naming system of raid devices

[Rudy Zijlstra]

On 02-01-2021 16:39, c.buhtz@posteo.jp wrote:
> Thanks to you and all other answers. It helped me a lot to understand
> the backgrounds.
>
> On 2021-01-01 14:09 Phil Turmel <philip@turmel.org> wrote:
>> I recommend creating an mdadm.conf file containing ARRAY entries for
>> your desired setup.  Trim those lines to only have the desired name
>> and UUID.
> Maybe I should open a new thread for that topic?
> Please correct me, because I am not an usual admin nor an mdadm expert.
>
> I do not want to convince you with the following! I just want to bring
> up my point of view and my opinion to make it possible for you experts
> to "correct" me. ;)
>
> I do not see the advantage of creating mdadm.conf.
> Via fstab I mount the devices by their UUID.
> And all other information's mdadm needs to use the RAID is stored in the
> superblock.
>
> So information's in mdadm.conf would be redundant. And especially for
> a non-routine home-admin like me each conf-file I modify keep the
> possibility of misstakes/missconfigurations and more problems. Keeping
> it as simple as possible is very important for my environment.

Nope, it is not redundant. It tells the init system that your UUID is on 
a raid device, which means it has to wait for the raid devices to be 
online. Without mdadm.conf it cannot know this.

Re: naming system of raid devices

[Anthony Youngman]

On 02/01/2021 22:12, Rudy Zijlstra wrote:
>> I do not see the advantage of creating mdadm.conf.
>> Via fstab I mount the devices by their UUID.
>> And all other information's mdadm needs to use the RAID is stored in the
>> superblock.
>>
>> So information's in mdadm.conf would be redundant. And especially for
>> a non-routine home-admin like me each conf-file I modify keep the
>> possibility of misstakes/missconfigurations and more problems. Keeping
>> it as simple as possible is very important for my environment.
> 
> Nope, it is not redundant. It tells the init system that your UUID is on 
> a raid device, which means it has to wait for the raid devices to be 
> online. Without mdadm.conf it cannot know this.

So it just waits until that UUID appears ...

Dunno whether my system is configured with UUID or device name, but root 
is on an array, and there's no mdadm.conf. Works fine ...

That's why the suggestion is "use eg /dev/md/root".

I'm with the OP. You don't need an mdadm.conf, so don't bother with it.

Cheers,
Wol