md prefered minor has been renumbered

md prefered minor has been renumbered

[Randall C. Grimshaw]

I have made the mistake of booting a centos-6.8 live cd to manipulate a centos 5 system.
as a result md5 has become md122, md2 has become md125, md4 has become md126, md0 has become md127, and most unfortunately md3 has also become md122.

smartctl shows that the WD brand drives do support SCT
mdadm --examine was used to reveal the mish-mash using uuid numbers compared with the file /etc/mdadm.conf also referencing /etc/fstab.

can someone kindly tell me the mdadm command to put the correct numbers back.

sda7,sdb7 (122) /dev/md5 uuid=11b7e8b7:ca181503:cf45af46:db2a5d9d /
sda1,sdb1 (127) /dev/md0 uuid=d9a5774d:d930fbd8:cf45af46:db2a5d9d /boot
sda3,sdb2 (125) /dev/md2 uuid=23a4116f:1d43390b:cf45af46:db2a5d9d /tmp
sda5,sdb5 (122) /dev/md3 uuid=c89f6357:f1c7a0ef:cf45af46:db2a5d9d /var
sda6,sdb6 (126) /dev/md4 uuid=67ad84bc:05153ebe:cf45af46:db2a5d9d /var/lib/mysql/cmdaemon_mon
sda2,sdb2 (   1) /dev/md1 uuid=5d9f0b00:24d890eb:cf45af46:db2a5d9d swap
sdc1                                                            /home

much appreciated
Randall Grimshaw

Re: md prefered minor has been renumbered

[Phil Turmel]

On 11/28/2016 12:26 PM, Randall C. Grimshaw wrote:
> I have made the mistake of booting a centos-6.8 live cd to manipulate a centos 5 system.
> as a result md5 has become md122, md2 has become md125, md4 has become md126, md0 has become md127, and most unfortunately md3 has also become md122.

This is normal and expected for a livecd -- the numbering comes from the
mdadm.conf file in the initramfs, unless that mdadm.conf file excludes
arrays for later assembly (when the rootfs is available).

Many livecd initramfs scripts try to assemble everything, so the numbers
are allocated from 127 down.  Usually you can find a boot parameter to
suppress assembly.

> smartctl shows that the WD brand drives do support SCT
> mdadm --examine was used to reveal the mish-mash using uuid numbers compared with the file /etc/mdadm.conf also referencing /etc/fstab.

Eww.  You should not be counting on /dev/mdX numbering in your fstab.
Specifically for this reason.  Use LABEL= or UUID= syntax in fstab.

> can someone kindly tell me the mdadm command to put the correct numbers back.

No need.  When you boot back into your normal system, its initramfs will
supply this information.  If you are changing the info, make sure you
rebuild your initramfs, so the updated mdadm.conf file is included.

Phil

Re: md prefered minor has been renumbered

[Wols Lists]

On 28/11/16 17:26, Randall C. Grimshaw wrote:
> I have made the mistake of booting a centos-6.8 live cd to manipulate a centos 5 system.
> as a result md5 has become md122, md2 has become md125, md4 has become md126, md0 has become md127, and most unfortunately md3 has also become md122.
> 
> smartctl shows that the WD brand drives do support SCT
> mdadm --examine was used to reveal the mish-mash using uuid numbers compared with the file /etc/mdadm.conf also referencing /etc/fstab.
> 
> can someone kindly tell me the mdadm command to put the correct numbers back.

In My Experience (I'll probably get told I'm wrong :-) mdadm.conf seems
to be pretty much ignored by modern systems. And md numbers now by
default count down from 127, not up from 0.

I think if you reboot back in to CentOS 5, you'll get the right numbers
back...
> 
> sda7,sdb7 (122) /dev/md5 uuid=11b7e8b7:ca181503:cf45af46:db2a5d9d /
> sda1,sdb1 (127) /dev/md0 uuid=d9a5774d:d930fbd8:cf45af46:db2a5d9d /boot
> sda3,sdb2 (125) /dev/md2 uuid=23a4116f:1d43390b:cf45af46:db2a5d9d /tmp
> sda5,sdb5 (122) /dev/md3 uuid=c89f6357:f1c7a0ef:cf45af46:db2a5d9d /var
> sda6,sdb6 (126) /dev/md4 uuid=67ad84bc:05153ebe:cf45af46:db2a5d9d /var/lib/mysql/cmdaemon_mon
> sda2,sdb2 (   1) /dev/md1 uuid=5d9f0b00:24d890eb:cf45af46:db2a5d9d swap
> sdc1                                                            /home
> 
That said, you should not be using fixed numbers any more - the linux
kernel does not guarantee discovery order or device names. It may
shuffle sda, sdb, sdc etc. It definitely shuffles mdnnn (my other gentoo
system causes me grief with this :-) because it's broken in other ways
too...)

What do you need your mdN names to be constant for? If you're just doing
emergency maintenance, then you might be better off using a rescue CD,
or just manually mounting those partitions for maintenance.

And if you're planning on upgrading, either convert the arrays to named
arrays, or refer to them by UUID.

Cheers,
Wol

Re: md prefered minor has been renumbered

[Phil Turmel]

On 11/28/2016 03:02 PM, Wols Lists wrote:

> In My Experience (I'll probably get told I'm wrong :-) mdadm.conf seems
> to be pretty much ignored by modern systems. And md numbers now by
> default count down from 127, not up from 0.

mdadm.conf is *not* ignored.  Defaults are taken if it isn't found, or
doesn't contain the info needed.

Most modern systems have two copies of mdadm.conf.  One in the initramfs
so you can boot a root filesystem that is inside an array, and the other
in your root filesystem to assemble everything else.

Most distros copy the latter to the former when updating the initramfs.
You may have to manually trigger this operation when you are
re-arranging your arrays.

You also have to manually intervene if you want your initramfs to have
a different mdadm.conf, like a two-phase assembly.

mdadm has been counting down from 127 for as long as I've used it.

Phil

RE: md prefered minor has been renumbered

[Randall C. Grimshaw]

Phil:
  Thank you for responding, it is much appreciated.
  This is a recently assigned system so I must apologize for not having all of the history re fstab.
  But when I try to boot into the 'normal' system it continues to use the new numbering, the mounts fail, (the duplicate md122 obviously fails), and the system panics.
  I suspect the answer will have something to do with the command I googled: mdadm -A --update=superminor /dev/mdNEWNUMBER  /dev/sd...
.. but I have zero experience with it.
Randall

________________________________________
From: Phil Turmel [philip@turmel.org]
Sent: Monday, November 28, 2016 2:58 PM
To: Randall C. Grimshaw; linux-raid@vger.kernel.org
Subject: Re: md prefered minor has been renumbered

On 11/28/2016 12:26 PM, Randall C. Grimshaw wrote:
> I have made the mistake of booting a centos-6.8 live cd to manipulate a centos 5 system.
> as a result md5 has become md122, md2 has become md125, md4 has become md126, md0 has become md127, and most unfortunately md3 has also become md122.

This is normal and expected for a livecd -- the numbering comes from the
mdadm.conf file in the initramfs, unless that mdadm.conf file excludes
arrays for later assembly (when the rootfs is available).

Many livecd initramfs scripts try to assemble everything, so the numbers
are allocated from 127 down.  Usually you can find a boot parameter to
suppress assembly.

> smartctl shows that the WD brand drives do support SCT
> mdadm --examine was used to reveal the mish-mash using uuid numbers compared with the file /etc/mdadm.conf also referencing /etc/fstab.

Eww.  You should not be counting on /dev/mdX numbering in your fstab.
Specifically for this reason.  Use LABEL= or UUID= syntax in fstab.

> can someone kindly tell me the mdadm command to put the correct numbers back.

No need.  When you boot back into your normal system, its initramfs will
supply this information.  If you are changing the info, make sure you
rebuild your initramfs, so the updated mdadm.conf file is included.

Phil

Re: md prefered minor has been renumbered

[Wols Lists]

On 28/11/16 20:22, Phil Turmel wrote:
> On 11/28/2016 03:02 PM, Wols Lists wrote:
> 
>> In My Experience (I'll probably get told I'm wrong :-) mdadm.conf seems
>> to be pretty much ignored by modern systems. And md numbers now by
>> default count down from 127, not up from 0.
> 
> mdadm.conf is *not* ignored.  Defaults are taken if it isn't found, or
> doesn't contain the info needed.

I did say IME :-)
> 
> Most modern systems have two copies of mdadm.conf.  One in the initramfs
> so you can boot a root filesystem that is inside an array, and the other
> in your root filesystem to assemble everything else.

Mine don't appear to have any ... and when I've tried to create one, it
was ignored. Of course, that doesn't rule out incompetence on my part
:-) But it means the defaults are perfectly okay for me :-) and probably
most people most of the time ...
> 
> Most distros copy the latter to the former when updating the initramfs.
> You may have to manually trigger this operation when you are
> re-arranging your arrays.
> 
> You also have to manually intervene if you want your initramfs to have
> a different mdadm.conf, like a two-phase assembly.
> 
> mdadm has been counting down from 127 for as long as I've used it.

I think I created several of my arrays as md0/1/2. And they stayed that
way for a little while before deciding to renumber.
> 
> Phil
> 
Cheers,
Wol

Re: md prefered minor has been renumbered

[Phil Turmel]

On 11/28/2016 03:39 PM, Wols Lists wrote:
> I did say IME :-)

(-:

>> Most modern systems have two copies of mdadm.conf.  One in the initramfs
>> so you can boot a root filesystem that is inside an array, and the other
>> in your root filesystem to assemble everything else.
> 
> Mine don't appear to have any ... and when I've tried to create one, it
> was ignored. Of course, that doesn't rule out incompetence on my part
> :-) But it means the defaults are perfectly okay for me :-) and probably
> most people most of the time ...

If you create an mdadm.conf file, and *don't* put it in your initramfs,
and are using a distro that assembles in the initramfs, your mdadm.conf
file will be ignored *during* *boot*.  But then used any other time
after boot.  If you just left it there, the next kernel upgrade or
package that triggered a new initramfs would have picked it up.  And
"magically" applied those array numbers on your next boot.

> I think I created several of my arrays as md0/1/2. And they stayed that
> way for a little while before deciding to renumber.

Sounds like your distro changed its initramfs around that time.

Phil

RE: md prefered minor has been renumbered

[Randall C. Grimshaw]

Update: So I was really hopeful given the google find of this link:
http://superuser.com/questions/346719/how-to-change-the-name-of-an-md-device-mdadm
as instructed I stopped my devices, that went well.
mdadm --stop /dev/md122
mdadm --assemble --update=super-minor /dev/md5 /dev/sdb7 /dev/sda7
but the --assemble command errors with the message: no devices found for /dev/md5
note: the devices /dev/sda7 and /dev/sda7 *DO* appear with fdisk -l

do I need to --force the command or something?

Randall
________________________________________
From: linux-raid-owner@vger.kernel.org [linux-raid-owner@vger.kernel.org] on behalf of Randall C. Grimshaw [rgrimsha@esf.edu]
Sent: Monday, November 28, 2016 3:34 PM
To: Phil Turmel; linux-raid@vger.kernel.org
Subject: RE: md prefered minor has been renumbered

Phil:
  Thank you for responding, it is much appreciated.
  This is a recently assigned system so I must apologize for not having all of the history re fstab.
  But when I try to boot into the 'normal' system it continues to use the new numbering, the mounts fail, (the duplicate md122 obviously fails), and the system panics.
  I suspect the answer will have something to do with the command I googled: mdadm -A --update=superminor /dev/mdNEWNUMBER  /dev/sd...
. but I have zero experience with it.
Randall

________________________________________
From: Phil Turmel [philip@turmel.org]
Sent: Monday, November 28, 2016 2:58 PM
To: Randall C. Grimshaw; linux-raid@vger.kernel.org
Subject: Re: md prefered minor has been renumbered

On 11/28/2016 12:26 PM, Randall C. Grimshaw wrote:
> I have made the mistake of booting a centos-6.8 live cd to manipulate a centos 5 system.
> as a result md5 has become md122, md2 has become md125, md4 has become md126, md0 has become md127, and most unfortunately md3 has also become md122.

This is normal and expected for a livecd -- the numbering comes from the
mdadm.conf file in the initramfs, unless that mdadm.conf file excludes
arrays for later assembly (when the rootfs is available).

Many livecd initramfs scripts try to assemble everything, so the numbers
are allocated from 127 down.  Usually you can find a boot parameter to
suppress assembly.

> smartctl shows that the WD brand drives do support SCT
> mdadm --examine was used to reveal the mish-mash using uuid numbers compared with the file /etc/mdadm.conf also referencing /etc/fstab.

Eww.  You should not be counting on /dev/mdX numbering in your fstab.
Specifically for this reason.  Use LABEL= or UUID= syntax in fstab.

> can someone kindly tell me the mdadm command to put the correct numbers back.

No need.  When you boot back into your normal system, its initramfs will
supply this information.  If you are changing the info, make sure you
rebuild your initramfs, so the updated mdadm.conf file is included.

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: md prefered minor has been renumbered

[Phil Turmel]

On 11/28/2016 03:34 PM, Randall C. Grimshaw wrote:
> Phil:
>   Thank you for responding, it is much appreciated.
>   This is a recently assigned system so I must apologize for not having all of the history re fstab.
>   But when I try to boot into the 'normal' system it continues to use the new numbering, the mounts fail, (the duplicate md122 obviously fails), and the system panics.
>   I suspect the answer will have something to do with the command I googled: mdadm -A --update=superminor /dev/mdNEWNUMBER  /dev/sd...
> .. but I have zero experience with it.
> Randall

Use 'blkid' to find out the UUIDs of your filesystems.  Replace /dev/mdX
in your fstab with UUID="......"

then "mount -av"

Then you don't care what number shows up, and can resolve it at your
leisure.

Whatever mdadm.conf you end up with, run your distro's utility to get it
into your initramfs.  On ubuntu, it is "update-initramfs".

Phil

Re: md prefered minor has been renumbered

[Phil Turmel]

Hi Randall,

{ Convention on kernel.org is reply-to-all and to avoid top-posting. I
added the list back. }

On 11/28/2016 05:56 PM, Randall C. Grimshaw wrote:
> Thank you Phil:
>   the linux rescue mode didn't have blkid but I had the uuid's from mdadm --examine,
> so I tried uuid=xxxxx in the fstab file. no joy. it is failing in the load of the ramfs, after assembly but before fstab.
>   it appears that I am caught in the catch22 where I cannot update initramfs until I minimally fix the duplicate md122 problem so that I can assemble /var and mount it.
> Randall

You must use the form UUID="xxxx" with caps and quotes.

you can test with mount UUID="....." /path/to/mount/point to check.

Phil

RE: md prefered minor has been renumbered

[Randall C. Grimshaw]

________________________________________
From: Phil Turmel [philip@turmel.org]
Sent: Monday, November 28, 2016 6:10 PM
To: Randall C. Grimshaw
Cc: Linux-RAID
Subject: Re: md prefered minor has been renumbered

Hi Randall,

{ Convention on kernel.org is reply-to-all and to avoid top-posting. I
added the list back. }

On 11/28/2016 05:56 PM, Randall C. Grimshaw wrote:
> Thank you Phil:
>   the linux rescue mode didn't have blkid but I had the uuid's from mdadm --examine,
> so I tried uuid=xxxxx in the fstab file. no joy. it is failing in the load of the ramfs, after assembly but before fstab.
>   it appears that I am caught in the catch22 where I cannot update initramfs until I minimally fix the duplicate md122 problem so that I can assemble /var and mount it.
> Randall

You must use the form UUID="xxxx" with caps and quotes.

you can test with mount UUID="....." /path/to/mount/point to check.

Phil
----------------------------------

Phil:
  Sorry for the typos and the top posts. Even with Caps and Quotes I was still in trouble.
sequentially it complains that md122 is already running - it cannot run /sdb5 (the duplicate)
then it tries to resume from /dev/md1 -- (md1 remains as swap) then a bunch of indicators that the initramfs
is unable to mount everything... but the post I found was basically correct.
I suspect it was the conflicting devices that made my situation different but the solution was to specify the 
device to mdadm using uuid= (thank you for that tip)
so my assembly command looked like: mdadm -A --uuid=xxxx --update=super-minor /dev/mdx
it solved the conflicting devices and returned my system to running state.
[SOLVED]