Confused about UUID mounting and mirrors

Confused about UUID mounting and mirrors

[michael]

Hello,

I am making the transition from hardware raid to software raid and 
things have gone pretty well.

I am trying to mount everything by UUID to account for devices moving 
around, etc.

I have moved all mounts in /etc/fstab to uuid except /boot which is a 
mirror sitting on /dev/md0.

Since it is a mirror, blkid returns the same uuid for /dev/md0, 
/dev/sda1, and /dev/sdb1.

This confuses mount at boot time.

How to I get it to correctly find md0 and not the components of the mirror?

Michael

Re: Confused about UUID mounting and mirrors

[Michal Soltys]

michael@kmaclub.com wrote:
> Hello,
> 
> This confuses mount at boot time.
> 
> How to I get it to correctly find md0 and not the components of the mirror?
> 

You mean you mount by /dev/disk/by-uuid/* ?

The 0.9x or 1.0 superblocks (the ones working nicely with grub and 
raid1) are placed at the end of the device, thus existing filesystem can 
be detected both from /dev/sd{a,b}1 and /dev/md0.

In such case, you will have to adjust udev rules, so sd{a,b}1 filesystem 
uuid symlinks will not be created, or will be overwritten by the symlink 
to /dev/md0.

You can achieve the latter with higher link_priority in OPTIONS, in your 
udev rules.

Re: Confused about UUID mounting and mirrors

[michael]

Michal Soltys wrote:
> michael@kmaclub.com wrote:
>> Hello,
>>
>> This confuses mount at boot time.
>>
>> How to I get it to correctly find md0 and not the components of the 
>> mirror?
>>
> 
> You mean you mount by /dev/disk/by-uuid/* ?


In /etc/fstab,  I have entries like this:

UUID=8e651838-38c1-4783-8bde-4174ec484d52 /     ext3    defaults  1 1
UUID=aaeca70c-f0fe-470c-b631-87248648d275 /export xfs defaults,nobarrier 1 2
UUID=6e22c5b0-2874-4826-a871-ed733f8da643 swap swap    defaults   0 0
UUID=323cd094-4cbe-4c3b-9096-366c05465e7c /export/services xfs defaults 1 2

so it is referring to the UUID of the filesystem and not necessarily the 
device.  Some are single partitions, some md devices, and some sit on 
top of LVM.


> The 0.9x or 1.0 superblocks (the ones working nicely with grub and 
> raid1) are placed at the end of the device, thus existing filesystem can 
> be detected both from /dev/sd{a,b}1 and /dev/md0.
> 
> In such case, you will have to adjust udev rules, so sd{a,b}1 filesystem 
> uuid symlinks will not be created, or will be overwritten by the symlink 
> to /dev/md0.
> 
> You can achieve the latter with higher link_priority in OPTIONS, in your 
> udev rules.

Do you have an example?  I am not sure I follow.

So what happens if someone sticks in a USB key and boots the machine. 
The key bcomes sda, and each following drive shifts.  Is UDEV in this 
case going to know to account for this?  The goal is to be able to 
always find the filesystem regardless of what the disk name might be.

Michael

Re: Confused about UUID mounting and mirrors

[David Greaves]

michael@kmaclub.com wrote:
> UUID=8e651838-38c1-4783-8bde-4174ec484d52 /     ext3    defaults  1 1
> UUID=aaeca70c-f0fe-470c-b631-87248648d275 /export xfs defaults,nobarrier
> 1 2
> UUID=6e22c5b0-2874-4826-a871-ed733f8da643 swap swap    defaults   0 0
> UUID=323cd094-4cbe-4c3b-9096-366c05465e7c /export/services xfs defaults 1 2
> 
> so it is referring to the UUID of the filesystem and not necessarily the
> device.  Some are single partitions, some md devices, and some sit on
> top of LVM.

What does :
  mdadm --detail /dev/md0 | grep UUID
show?

I think it will give you the UUID of the md device, not the fs on the md device
(which as you have seen is also the UUID of the fs visible on the md components)

David

Re: Confused about UUID mounting and mirrors

[michael]

David Greaves wrote:
> michael@kmaclub.com wrote:
>> UUID=8e651838-38c1-4783-8bde-4174ec484d52 /     ext3    defaults  1 1
>> UUID=aaeca70c-f0fe-470c-b631-87248648d275 /export xfs defaults,nobarrier
>> 1 2
>> UUID=6e22c5b0-2874-4826-a871-ed733f8da643 swap swap    defaults   0 0
>> UUID=323cd094-4cbe-4c3b-9096-366c05465e7c /export/services xfs defaults 1 2
>>
>> so it is referring to the UUID of the filesystem and not necessarily the
>> device.  Some are single partitions, some md devices, and some sit on
>> top of LVM.
> 
> What does :
>   mdadm --detail /dev/md0 | grep UUID
> show?
> 
> I think it will give you the UUID of the md device, not the fs on the md device
> (which as you have seen is also the UUID of the fs visible on the md components)


That does return a UUID, but it is a slightly different format

mdadm --detail /dev/md0 | grep UUID
            UUID : 3531a48d:1180bdf3:bfcd9e76:5b7e5957

versus something like:

6e22c5b0-2874-4826-a871-ed733f8da643

I tried adding it to /etc/fstab as well as just using mount -U <uuid> 
and both fail complaining that:

mount -U 3531a48d:1180bdf3:bfcd9e76:5b7e5957 /mnt
mount: no such partition found

I tried replacing the : with - to match the format in /etc/fstab with no 
success.

mount -U 3531a48d-1180bdf3-bfcd9e76-5b7e5957 /mnt
mount: no such partition found


I also tried formatting it the same in terms of the - location

mount -U  3531a48d-1180-bdf3-bfcd-9e765b7e5957 /mnt
mount: no such partition found


Is anyone mounting a mirrored /boot using UUID's?

Michael

Re: Confused about UUID mounting and mirrors

[Michael Tokarev]

michael@kmaclub.com wrote:
[]
> mdadm --detail /dev/md0 | grep UUID
>            UUID : 3531a48d:1180bdf3:bfcd9e76:5b7e5957

This is a RAID (md) -- i.e., device, UUID.  Think of it
as of disk drive serial number for example -- look at
/dev/disk/by-id/.

> mount -U 3531a48d:1180bdf3:bfcd9e76:5b7e5957 /mnt
> mount: no such partition found

And mount expects *filesystem* UUID, not a device UUID.
I.e., the UUID which was written by mkfs.  For a ext[234]fs,
tune2fs -l /dev/md0 will show it; for other filesystems
use appropriate tools.

/mjt

Re: Confused about UUID mounting and mirrors

[michael]

Michael Tokarev wrote:
> michael@kmaclub.com wrote:
> []
>> mdadm --detail /dev/md0 | grep UUID
>>            UUID : 3531a48d:1180bdf3:bfcd9e76:5b7e5957
> 
> This is a RAID (md) -- i.e., device, UUID.  Think of it
> as of disk drive serial number for example -- look at
> /dev/disk/by-id/.
> 
>> mount -U 3531a48d:1180bdf3:bfcd9e76:5b7e5957 /mnt
>> mount: no such partition found
> 
> And mount expects *filesystem* UUID, not a device UUID.
> I.e., the UUID which was written by mkfs.  For a ext[234]fs,
> tune2fs -l /dev/md0 will show it; for other filesystems
> use appropriate tools.


I understand.  I was trying this based on a suggestion further back in 
this thread.

Using the filesystem ID doesn't work either because /dev/sda, /dev/sdb, 
and /dev/md0 all have the same UUID since it is a mirror.  How do you 
tell the system to mount /dev/md0 and not the individual drives which is 
what it is trying to do now.

Michael

Re: Confused about UUID mounting and mirrors

[Clive Messer]

On Tue, 2008-06-17 at 07:19 -0700, michael@kmaclub.com wrote:

> Using the filesystem ID doesn't work either because /dev/sda, /dev/sdb, 
> and /dev/md0 all have the same UUID since it is a mirror.  How do you 
> tell the system to mount /dev/md0 and not the individual drives which is 
> what it is trying to do now.

Michael,

I'm not sure I understand. My Fedora fstab and grub.conf uses UUID's.

The UUID on the underlying raid devices is different to the filesystem
device. eg. /dev/md0 is RAID1 comprised of /dev/sd[a-d]1

/dev/md0: UUID="d046b3c4-8f1b-4c60-8982-9621cf2ac34c" SEC_TYPE="ext2"
TYPE="ext3" LABEL="BOOT"

/dev/sda1: UUID="bb2b88b1-b1a0-c8f7-2e40-e00de48ff896" TYPE="mdraid"
/dev/sdb1: UUID="bb2b88b1-b1a0-c8f7-2e40-e00de48ff896" TYPE="mdraid" 
/dev/sdc1: UUID="bb2b88b1-b1a0-c8f7-2e40-e00de48ff896" TYPE="mdraid" 
/dev/sdd1: UUID="bb2b88b1-b1a0-c8f7-2e40-e00de48ff896" TYPE="mdraid" 

/etc/fstab:
UUID=d046b3c4-8f1b-4c60-8982-9621cf2ac34c /boot ext3 defaults 1 2

Regards

Clive
-  
Clive Messer <clive@vacuumtube.org.uk>

Re: Confused about UUID mounting and mirrors

[Peter Grandi]

[ ... absurd question about disambiguating storage entities with
the same UUID but which are otherwise distinct ... ]

> Is anyone mounting a mirrored /boot using UUID's?

I hope it is only you, because it is a complete absurdity: UUID
stands for Universally Unique ID. If someone gives the same UUID
(for example by mirroring) to two entities that are in fact not
absolutely equivalent in every respect (including convenience)
they are misusing UUIDs (but then how many people understand the
difference between "equal" and "identically the same", even
among programmers: modality can be subtle).

In practice current storage entity naming schemes have not been
designed to cover some common cases, and you have two options:

* Use relative (storage path) names, and those will change, but
  you can distinguish between mirror copies.

* Use absolute (UUIDs, labels) names, and those won't change,
  but you cannot distinguish between mirror copies.

An approach that would be suitable for mirroring would be to use
semi-relative naming: absolute naming for the containing storage
entity (e.g. disk) and relative naming for the contained one
(e.g.  partition or filesystem), assuming that the latter are
static and not subject to change.

Then you can mirror the contained storage entities and
ambiguities do not arise because the container UUID is
unambiguous and the relative path from it is static.  For
example something like '/dev/${VOLID}_p0' where the '${VOLID}'
part can be a UUID or a storage media label or "something".

The problem is that none of the commonly available software uses
this kind of naming scheme, even if to some extend it would not
be too hard to introduce the basic outline.

I use something like that *manually* where I use the "disk
signature" (4 bytes) of a PC-DOS MBR to uniquely identify
physical volumes, and then I mirror-copy not the whole physical
volume (which would duplicate the disk signature), but each
partition defined by the MBR. I don't use 'udev' (I think it is
awful and then I dislike the ugly, petty politics surrounding
its inception).

Re: Confused about UUID mounting and mirrors

[Michal Soltys]

michael@kmaclub.com wrote:
> 
> In /etc/fstab,  I have entries like this:
> 
> UUID=8e651838-38c1-4783-8bde-4174ec484d52 /     ext3    defaults  1 1
> UUID=aaeca70c-f0fe-470c-b631-87248648d275 /export xfs defaults,nobarrier 
> 1 2
> UUID=6e22c5b0-2874-4826-a871-ed733f8da643 swap swap    defaults   0 0
> UUID=323cd094-4cbe-4c3b-9096-366c05465e7c /export/services xfs defaults 1 2
> 
> 
> Do you have an example?  I am not sure I follow.
> 

Taken fragment from stock udev rules:

IMPORT{program}="vol_id --export $tempnode"

OPTIONS="link_priority=100"

ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*",
SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"

ENV{ID_FS_USAGE}=="filesystem|other", ENV{ID_FS_LABEL_ENC}=="?*",
SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"

- vol_id will export filesystem metadata, which udev will import and
make available for further rules in current uevent

- link_priority will prioritize uuid and label symlinks to the
filesystem found on md0, over existing or to-be-made symlinks
to mirror components. (actully - vol_id called for components will 
detect they are part of raid and return md's uuid along with USAGE=raid 
- so the link won't be even created in persistant storage rules).

- the rest will create appropriate symlinks

That's the general idea. Of coruse it's only a fragment. Check
60-persistent-storage.rules and 64-md-raid.rules from udev sources, to
see how things get done there.

E.g. in one fstab of mine (not uuids, the same idea though), there're:

/dev/disk/by-id/scsi-SATA_Maxtor_6Y120P0_Y32JMMQE-part1         /boot
ext2    defaults 0 0
/dev/disk/by-id/scsi-SATA_Maxtor_6Y120P0_Y32JMMQE-part2         swap
swap    defaults 0 0

> So what happens if someone sticks in a USB key and boots the machine. 
> The key bcomes sda, and each following drive shifts.  Is UDEV in this 
> case going to know to account for this?  The goal is to be able to 
> always find the filesystem regardless of what the disk name might be.
> 

That depends on the udev rules. With typical ones - your names would
shift, but symlinks created from /dev/disk/by-* directories would be proper.

You can even set sd{a,b,c,...} or any other names you want persistently
there, basing on the device's label, uuid or other traits - similary to
how network devices can be named persistently based on e.g. their mac
address. Still - with symlinks - there isn't much reason to.

When mdadm assembles your /dev/md0 - it usually looks for md's uuid
along other things (check your mdadm.conf) - so leaving /dev/md0 in
fstab should be perfectly fine. You can even name it /dev/grubboot - it
doesn't have to be md0.

Re: Confused about UUID mounting and mirrors

[Michael Tokarev]

michael@kmaclub.com wrote:
[]
> Using the filesystem ID doesn't work either because /dev/sda, /dev/sdb, 
> and /dev/md0 all have the same UUID since it is a mirror.  How do you 
> tell the system to mount /dev/md0 and not the individual drives which is 
> what it is trying to do now.

The tools/library that looks up filesystem UUIDs/labels are smarter than
that.  Or are supposed to be smarter.  That it - the tools are able to
recognize and skip components of a raid array, since raid superblock is
present on the device - which is not present on the array itself.

/mjt

Re: Confused about UUID mounting and mirrors

[michael]

Michael Tokarev wrote:
> michael@kmaclub.com wrote:
> []
>> Using the filesystem ID doesn't work either because /dev/sda, 
>> /dev/sdb, and /dev/md0 all have the same UUID since it is a mirror.  
>> How do you tell the system to mount /dev/md0 and not the individual 
>> drives which is what it is trying to do now.
> 
> The tools/library that looks up filesystem UUIDs/labels are smarter than
> that.  Or are supposed to be smarter.  That it - the tools are able to
> recognize and skip components of a raid array, since raid superblock is
> present on the device - which is not present on the array itself.


Ok.  Thanks,

I will play with this more tonight and see I did something wrong the 
first time I tried it.

Thanks for all the suggestions.

Michael

Re: Confused about UUID mounting and mirrors

[Neil Brown]

On Monday June 16, michael@kmaclub.com wrote:
> Hello,
> 
> I am making the transition from hardware raid to software raid and 
> things have gone pretty well.
> 
> I am trying to mount everything by UUID to account for devices moving 
> around, etc.
> 
> I have moved all mounts in /etc/fstab to uuid except /boot which is a 
> mirror sitting on /dev/md0.
> 
> Since it is a mirror, blkid returns the same uuid for /dev/md0, 
> /dev/sda1, and /dev/sdb1.
> 
> This confuses mount at boot time.
> 
> How to I get it to correctly find md0 and not the components of the mirror?

One option could be to not use UUIDs for mount md devices.  md devices
do not suffer from random name changes when devices config changes
because they are named based on content rather than location.

I realise that it might be more elegant to use UUIDs for everything,
but if you are just after "practical", but don't bother with UUIDs to
mount md arrays.

NeilBrown

Re: Confused about UUID mounting and mirrors

[michael]

Michael Tokarev wrote:
> michael@kmaclub.com wrote:
> []
>> Using the filesystem ID doesn't work either because /dev/sda, 
>> /dev/sdb, and /dev/md0 all have the same UUID since it is a mirror.  
>> How do you tell the system to mount /dev/md0 and not the individual 
>> drives which is what it is trying to do now.
> 
> The tools/library that looks up filesystem UUIDs/labels are smarter than
> that.  Or are supposed to be smarter.  That it - the tools are able to
> recognize and skip components of a raid array, since raid superblock is
> present on the device - which is not present on the array itself.


Well, the tools just might be smarter than me.

I tried boot via UUID again tonight using the filesystem UUID and it is 
working.  I haven't messed with the udev rules or changed anything I can 
think of to account for the change in behaviour.  I have also rebooted a 
number of times and it seems to work every time.

Good news it is fixed and I appreciate all the suggestions from 
everyone.  I have learned a lot about linux software raid ( and LVM ) in 
the course of this little project.  Bad news I don't know why it is fixed.

Thanks everyone!

Michael

Re: Confused about UUID mounting and mirrors

[Bill Davidsen]

michael@kmaclub.com wrote:
> Hello,
>
> I am making the transition from hardware raid to software raid and 
> things have gone pretty well.
>
> I am trying to mount everything by UUID to account for devices moving 
> around, etc.
>
> I have moved all mounts in /etc/fstab to uuid except /boot which is a 
> mirror sitting on /dev/md0.
>
> Since it is a mirror, blkid returns the same uuid for /dev/md0, 
> /dev/sda1, and /dev/sdb1.
>
> This confuses mount at boot time.
>
> How to I get it to correctly find md0 and not the components of the 
> mirror?

After reading the whole thread, I'm not sure anyone has explained what's 
happening. Te reason for using a UUID is to get a mount to work even 
when the path to data changes dues to physical device renaming. However, 
it's perfectly safe to mount bd devices by device name, providing that 
the UUID is used in the mdadm.conf file to assure that the right 
physical components are using in the md device.

There may also be come confusion due to two kinfs of UUID, the one for 
the filesystem, which you know is replicated on all mirrors, and the one 
on the md array, which is used to assemble to array. So if you use the 
array UUID in tour mdadm.conf file, the name of the array will remain 
constant, and you can safely use that for mounting.

As you have been told, the tools are supposed to be smart enough to use 
the filesystem on the md array and not on the components when a 
filesystem UUID is in the fstab. But you have found the hard way that 
this is not always going to work correctly, due to factors I haven't 
identified yet, other than "every once in a while this ***'s up!"

I'm not making a recommendation, just trying to clarify what's happening 
and why.

-- 
Bill Davidsen <davidsen@tmr.com>
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark