LVM on dmraid breakage

LVM on dmraid breakage

[Phillip Susi]

I am investigating a bug report involving the combination of lvm and 
dmraid, and it seems to me that the problem is that lvm is detecting the 
underlying partition on the primary disk and accessing it directly 
rather than going through the raid device made by dmraid.  I think that 
the problem is there is no facility for dmraid to "claim" the physical 
disk so that lvm does not look at it, and also to have lvm scan the raid 
device for physical volumes.

Does anyone have any thoughts on how this might be accomplished?

Re: LVM on dmraid breakage

[Jonathan Brassow]

I'm not aware of a mechanism that LVM used to skip over owned  
devices... there may exist one.  However, you could use lvm filters  
to skip those underlying devices.

  brassow

On Aug 1, 2007, at 3:19 PM, Phillip Susi wrote:

> I am investigating a bug report involving the combination of lvm  
> and dmraid, and it seems to me that the problem is that lvm is  
> detecting the underlying partition on the primary disk and  
> accessing it directly rather than going through the raid device  
> made by dmraid.  I think that the problem is there is no facility  
> for dmraid to "claim" the physical disk so that lvm does not look  
> at it, and also to have lvm scan the raid device for physical volumes.
>
> Does anyone have any thoughts on how this might be accomplished?
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

Re: LVM on dmraid breakage

[Phillip Susi]

Jonathan Brassow wrote:
> I'm not aware of a mechanism that LVM used to skip over owned devices... 
> there may exist one.  However, you could use lvm filters to skip those 
> underlying devices.
> 
>  brassow

LVM filters?

P.S. I would like to point out that because this mailing list munges the 
reply-to header, this conversation has been split even though I cross 
posted it to ataraid-list.

Re: LVM on dmraid breakage

[Jonathan Brassow]

On Aug 1, 2007, at 4:19 PM, Phillip Susi wrote:

> Jonathan Brassow wrote:
>> I'm not aware of a mechanism that LVM used to skip over owned  
>> devices... there may exist one.  However, you could use lvm  
>> filters to skip those underlying devices.
>>  brassow
>
> LVM filters?

Look in /etc/lvm/lvm.conf for "filter".  There are some examples  
there.  Be careful though, if you exclude your root partion you may  
be in for future pain.

  brassow

Re: LVM on dmraid breakage

[Chandra Seetharaman]

On Wed, 2007-08-01 at 16:44 -0500, Jonathan Brassow wrote:
> On Aug 1, 2007, at 4:19 PM, Phillip Susi wrote:
> 
> > Jonathan Brassow wrote:
> >> I'm not aware of a mechanism that LVM used to skip over owned  
> >> devices... there may exist one.  However, you could use lvm  
> >> filters to skip those underlying devices.
> >>  brassow
> >
> > LVM filters?
> 
> Look in /etc/lvm/lvm.conf for "filter".  There are some examples  
> there.  Be careful though, if you exclude your root partion you may  
> be in for future pain.
> 

To avoid this problem, put your root disk at the start of the filter
with a "a" (like "a|/dev/sda.*|"), and have the "r"ejects after it.

Basically, when a device is found, the filter is walked from left to
write and action is taken when the first match happen. IOW, all your
scsi disks will be "r"ejected with the following filter,

["r|/dev/sd.*|", "a|/dev/sdgp.*|", "a|/dev/sdgo.*|", "a|/dev/sda.*" ]

in effect the later "a"ccepts are useless.

>   brassow
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan@us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

[linux-lvm] Re: LVM on dmraid breakage

[Luca Berra]

On Wed, Aug 01, 2007 at 04:19:59PM -0400, Phillip Susi wrote:
>I am investigating a bug report involving the combination of lvm and 
>dmraid, and it seems to me that the problem is that lvm is detecting the 
>underlying partition on the primary disk and accessing it directly 
>rather than going through the raid device made by dmraid.  I think that 
>the problem is there is no facility for dmraid to "claim" the physical 
>disk so that lvm does not look at it, and also to have lvm scan the raid 
>device for physical volumes.
>
>Does anyone have any thoughts on how this might be accomplished?

Two things come to mind
1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
partition table on component devices

2) lvm tool filter could be modified to check if a device has been
already claimed by device-mapper.

something along the lines of:
ioctl DM_LIST_DEVICES
while (...)
  ioctl DM_TABLE_DEPS

the above would probably require a cache.


i believe that manual use of partx or lvm filters is sub-optimal
L.



-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: LVM on dmraid breakage

[Luca Berra]

On Wed, Aug 01, 2007 at 04:19:59PM -0400, Phillip Susi wrote:
>I am investigating a bug report involving the combination of lvm and 
>dmraid, and it seems to me that the problem is that lvm is detecting the 
>underlying partition on the primary disk and accessing it directly 
>rather than going through the raid device made by dmraid.  I think that 
>the problem is there is no facility for dmraid to "claim" the physical 
>disk so that lvm does not look at it, and also to have lvm scan the raid 
>device for physical volumes.
>
>Does anyone have any thoughts on how this might be accomplished?

Two things come to mind
1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
partition table on component devices

2) lvm tool filter could be modified to check if a device has been
already claimed by device-mapper.

something along the lines of:
ioctl DM_LIST_DEVICES
while (...)
  ioctl DM_TABLE_DEPS

the above would probably require a cache.


i believe that manual use of partx or lvm filters is sub-optimal
L.



-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[linux-lvm] Re: LVM on dmraid breakage

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Luca Berra wrote:
> Two things come to mind
> 1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
> partition table on component devices
> 
> 2) lvm tool filter could be modified to check if a device has been
> already claimed by device-mapper.
> 

But device-mapper already opens devices exclusively when a target claims
them.

The problem here sounds more like it's down to ordering; LVM2 is being
given a chance to scan the devices & build LVs / claim devices before
dmraid gets a look at them.


Kind regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFGsbXh6YSQoMYUY94RAp5aAKDHrjX/LIoM5XPkeD/3hkd+oQ9vHgCZAXHS
Ykme+1gwvYuRat0fZwBALJs=
=mIsK
-----END PGP SIGNATURE-----

Re: LVM on dmraid breakage

[Bryn M. Reeves]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Luca Berra wrote:
> Two things come to mind
> 1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
> partition table on component devices
> 
> 2) lvm tool filter could be modified to check if a device has been
> already claimed by device-mapper.
> 

But device-mapper already opens devices exclusively when a target claims
them.

The problem here sounds more like it's down to ordering; LVM2 is being
given a chance to scan the devices & build LVs / claim devices before
dmraid gets a look at them.


Kind regards,
Bryn.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFGsbXh6YSQoMYUY94RAp5aAKDHrjX/LIoM5XPkeD/3hkd+oQ9vHgCZAXHS
Ykme+1gwvYuRat0fZwBALJs=
=mIsK
-----END PGP SIGNATURE-----

Re: LVM on dmraid breakage

[Phillip Susi]

Bryn M. Reeves wrote:
> But device-mapper already opens devices exclusively when a target claims
> them.
> 
> The problem here sounds more like it's down to ordering; LVM2 is being
> given a chance to scan the devices & build LVs / claim devices before
> dmraid gets a look at them.


I think the "exclusive" open only prevents you from mounting a 
filesystem on the device.  It does not stop you from opening the device 
and accessing it from user mode, nor does it apparently prevent dm from 
(re)using the device.

The problem is that lvm is looking at the physical disks at all.  It 
needs to scan the dmraid device, not the underlying disks.

Re: LVM on dmraid breakage

[Phillip Susi]

Luca Berra wrote:
> Two things come to mind
> 1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
> partition table on component devices

How will this play out in terms of udev plug events?  Can it be relied 
on that udev will process the disk first, before the partitions?  If the 
partitions are deleted during the processing of the disk, does udev 
still try to process their add events?  Will it then process the delete 
events?  If so then this is problematic.

> 2) lvm tool filter could be modified to check if a device has been
> already claimed by device-mapper.
> 
> something along the lines of:
> ioctl DM_LIST_DEVICES
> while (...)
>  ioctl DM_TABLE_DEPS
> 
> the above would probably require a cache.

I'd rather not see it that tightly coupled to dm.  I was thinking 
perhaps of something involving udev attributes so that udev would know 
not to run pvscan on that device.  Though it looks like pvscan also 
needs reworked so it plays nice with udev, being called to scan a given 
device as detected rather than looking for all well known physical 
device names in /dev.

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[David Robinson]

> The problem is that lvm is looking at the physical disks at all.  It 
> needs to scan the dmraid device, not the underlying disks.

If you need LVM to only scan certain devices you should modify the 
filter line in lvm.conf.

--Dave

Re: Re: LVM on dmraid breakage

[David Robinson]

> The problem is that lvm is looking at the physical disks at all.  It 
> needs to scan the dmraid device, not the underlying disks.

If you need LVM to only scan certain devices you should modify the 
filter line in lvm.conf.

--Dave

Re: [linux-lvm] Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 02, 2007 at 11:45:53AM +0100, Bryn M. Reeves wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Luca Berra wrote:
>> Two things come to mind
>> 1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
>> partition table on component devices
>> 
>> 2) lvm tool filter could be modified to check if a device has been
>> already claimed by device-mapper.
>> 
>
>But device-mapper already opens devices exclusively when a target claims
>them.

No, it does not,
sorry

[root@localhost ~]# dmsetup create foobar
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup table
foobar: 0 281329664 linear 9:2 0
[root@localhost ~]# dmsetup create foobar1
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup create foobar2
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup create foobar2
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup table
foobar3: 0 281329664 linear 9:2 0
foobar2: 0 281329664 linear 9:2 0
foobar1: 0 281329664 linear 9:2 0
foobar: 0 281329664 linear 9:2 0

L.
-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: [linux-lvm] Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 02, 2007 at 11:45:53AM +0100, Bryn M. Reeves wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Luca Berra wrote:
>> Two things come to mind
>> 1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
>> partition table on component devices
>> 
>> 2) lvm tool filter could be modified to check if a device has been
>> already claimed by device-mapper.
>> 
>
>But device-mapper already opens devices exclusively when a target claims
>them.

No, it does not,
sorry

[root@localhost ~]# dmsetup create foobar
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup table
foobar: 0 281329664 linear 9:2 0
[root@localhost ~]# dmsetup create foobar1
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup create foobar2
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup create foobar2
0 281329664 linear /dev/md2 0
[root@localhost ~]# dmsetup table
foobar3: 0 281329664 linear 9:2 0
foobar2: 0 281329664 linear 9:2 0
foobar1: 0 281329664 linear 9:2 0
foobar: 0 281329664 linear 9:2 0

L.
-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 02, 2007 at 06:04:20PM -0400, Phillip Susi wrote:
>Luca Berra wrote:
>>Two things come to mind
>>1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
>>partition table on component devices
>
>How will this play out in terms of udev plug events?  Can it be relied 
>on that udev will process the disk first, before the partitions?  If the 
>partitions are deleted during the processing of the disk, does udev 
>still try to process their add events?  Will it then process the delete 
>events?  If so then this is problematic.
I really have no clue, fortunately dmraid is usually used for boot
disks, so it should be started before udev has a chance of messing with
devices. btw, md already does this, and redhat's initrd does this as
well.
The real solution to this is so simple.
Just ditch the partition detection code from the kernel, and move it to
userspace where it belongs.
For the time being use BLKPG_DEL_PARTITION, to undo what the kernel
should not have done.

>>2) lvm tool filter could be modified to check if a device has been
>>already claimed by device-mapper.
>>
>>something along the lines of:
>>ioctl DM_LIST_DEVICES
>>while (...)
>>  ioctl DM_TABLE_DEPS
>>
>>the above would probably require a cache.
>
>I'd rather not see it that tightly coupled to dm.  I was thinking 
>perhaps of something involving udev attributes so that udev would know 
>not to run pvscan on that device.  Though it looks like pvscan also 
>needs reworked so it plays nice with udev, being called to scan a given 
>device as detected rather than looking for all well known physical 
>device names in /dev.
i'd rather not see it coupled with udev :P
maybe i am limited, but i really fail to see how an event driven model
could be at all useful in this cases, and i am really convinced that the
effort needed to make i work is too high compared to possible benefits.
The biggest question being: How do you know you have scanned all
possible PV for a given volume group, and you have to activate it.
Anyway i still think my proposal is sensible and adapts to the current
paradigm of lvm2.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 02, 2007 at 06:04:20PM -0400, Phillip Susi wrote:
>Luca Berra wrote:
>>Two things come to mind
>>1) dmraid should use ioctl BLKPG_DEL_PARTITION, to clean up any eventual
>>partition table on component devices
>
>How will this play out in terms of udev plug events?  Can it be relied 
>on that udev will process the disk first, before the partitions?  If the 
>partitions are deleted during the processing of the disk, does udev 
>still try to process their add events?  Will it then process the delete 
>events?  If so then this is problematic.
I really have no clue, fortunately dmraid is usually used for boot
disks, so it should be started before udev has a chance of messing with
devices. btw, md already does this, and redhat's initrd does this as
well.
The real solution to this is so simple.
Just ditch the partition detection code from the kernel, and move it to
userspace where it belongs.
For the time being use BLKPG_DEL_PARTITION, to undo what the kernel
should not have done.

>>2) lvm tool filter could be modified to check if a device has been
>>already claimed by device-mapper.
>>
>>something along the lines of:
>>ioctl DM_LIST_DEVICES
>>while (...)
>>  ioctl DM_TABLE_DEPS
>>
>>the above would probably require a cache.
>
>I'd rather not see it that tightly coupled to dm.  I was thinking 
>perhaps of something involving udev attributes so that udev would know 
>not to run pvscan on that device.  Though it looks like pvscan also 
>needs reworked so it plays nice with udev, being called to scan a given 
>device as detected rather than looking for all well known physical 
>device names in /dev.
i'd rather not see it coupled with udev :P
maybe i am limited, but i really fail to see how an event driven model
could be at all useful in this cases, and i am really convinced that the
effort needed to make i work is too high compared to possible benefits.
The biggest question being: How do you know you have scanned all
possible PV for a given volume group, and you have to activate it.
Anyway i still think my proposal is sensible and adapts to the current
paradigm of lvm2.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Luca Berra wrote:
> I really have no clue, fortunately dmraid is usually used for boot
> disks, so it should be started before udev has a chance of messing with
> devices. btw, md already does this, and redhat's initrd does this as
> well.

I don't know about redhat, but Ubuntu uses udev in the initramfs. 
Currently the boot scripts wait for udevsettle, then run dmraid and 
pvscan to search out all detected devices, then tries to mount the root 
fs, but this setup is less than ideal.  Our goal is to move towards full 
udev plug and play, where devices are detected by the kernel, processed, 
and as soon as the one(s) needed for the root have been enumerated, 
mount the root fs and run-init.

> The real solution to this is so simple.
> Just ditch the partition detection code from the kernel, and move it to
> userspace where it belongs.
> For the time being use BLKPG_DEL_PARTITION, to undo what the kernel
> should not have done.

I agree with moving the partition detection code to user space, but 
trying to undo it after the fact doesn't help because udev is already 
processing the add events.  Also you do not need to remove the 
partitions so long as pvscan understands that it shouldn't be using them.

> i'd rather not see it coupled with udev :P
> maybe i am limited, but i really fail to see how an event driven model
> could be at all useful in this cases, and i am really convinced that the
> effort needed to make i work is too high compared to possible benefits.
> The biggest question being: How do you know you have scanned all
> possible PV for a given volume group, and you have to activate it.
> Anyway i still think my proposal is sensible and adapts to the current
> paradigm of lvm2.

Udev is supposed to be the new model for enumerating devices and 
performing plug and play actions on them, rather than "ls /dev/hd?".  To 
answer your specific question, lvm would know it has enumerated all the 
required pvs because the vg information tells it how many pvs are 
supposed to be there so it can check the udevdb to see if they have all 
been added yet.  If they have, activate the vg, otherwise do nothing 
until another pv is detected.

The event driven model is useful in that it allows faster boot times ( 
don't need to detect ALL devices before activating and mounting the root 
), and allows runtime plug and play.  Where does lvm store state 
information right now?  In conf files in /etc isn't it?  Why use its own 
database for that when it could just keep the information in udevdb?
Keeping the information in udevdb makes it easily available to things 
like hal in gnome, which in turn allows things like desktop popups when 
a mirror has a drive fail, which changes the state of the vg in udevdb, 
which pushes that update to any monitoring applications.  It also allows 
lvm to share information such as whether or not the device is "claimed" 
with other components, without those components having to be 
specifically aware of lvm and mess with its conf files.  Specifically, 
both dmraid and mdraid ( nor any other future components ) do not have 
to be taught how to edit lvm's conf files to tell it not to access the 
underlying physical disks but to use the virtual devices instead.

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
>I agree with moving the partition detection code to user space, but 
>trying to undo it after the fact doesn't help because udev is already 
>processing the add events.  Also you do not need to remove the 
>partitions so long as pvscan understands that it shouldn't be using them.
which is the modification i proposed to lvm tools, isn't it?


>Udev is supposed to be the new model for enumerating devices and 
i know that, and i will withdraw from this discussion, since it might
get to an useless flame war.

Is there any technical reason for not having lvm tools filter out devices that
are used by device mapper?

besides dmraid, think of multipath.

Regards,
L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Luca Berra]

On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
>I agree with moving the partition detection code to user space, but 
>trying to undo it after the fact doesn't help because udev is already 
>processing the add events.  Also you do not need to remove the 
>partitions so long as pvscan understands that it shouldn't be using them.
which is the modification i proposed to lvm tools, isn't it?


>Udev is supposed to be the new model for enumerating devices and 
i know that, and i will withdraw from this discussion, since it might
get to an useless flame war.

Is there any technical reason for not having lvm tools filter out devices that
are used by device mapper?

besides dmraid, think of multipath.

Regards,
L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Luca Berra wrote:
> On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
>> I agree with moving the partition detection code to user space, but 
>> trying to undo it after the fact doesn't help because udev is already 
>> processing the add events.  Also you do not need to remove the 
>> partitions so long as pvscan understands that it shouldn't be using them.
> which is the modification i proposed to lvm tools, isn't it?

You suggested deleting the partition table after it has already been 
detected.  I am saying that while I agree in principal that partition 
detection should be moved out of the kernel, for now, it is in there and 
deleting them after they have already been detected doesn't help matters 
because pvscan may already be running on them.

>> Udev is supposed to be the new model for enumerating devices and 
> i know that, and i will withdraw from this discussion, since it might
> get to an useless flame war.
> 
> Is there any technical reason for not having lvm tools filter out 
> devices that
> are used by device mapper?
> 
> besides dmraid, think of multipath.

None that I can see at the moment, but that doesn't mean there isn't 
one, or won't be one in the future.  The other problem is that there are 
likely other factors besides being used already as a dm target that 
might give reason for lvm to not scan the volume.  These kind of policy 
decisions seem like they should be made by udev rather than hard coded 
into lvm.  If the admin wants a policy where lvm should look at volumes, 
or indeed, maybe only certain volumes, that already happen to be dm 
targets, he should be able to do that.  Likewise, there may be some 
other reason to not look at a disk for lvm pvs.  Editing a conf file to 
specify a filter list of devices by name is all well and good for a 
static system, but it does not play well in the modern udev managed plug 
and play world.

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Fri, Aug 03, 2007 at 05:30:00PM -0400, Phillip Susi wrote:
>Luca Berra wrote:
>>On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
>>>I agree with moving the partition detection code to user space, but 
>>>trying to undo it after the fact doesn't help because udev is already 
>>>processing the add events.  Also you do not need to remove the 
>>>partitions so long as pvscan understands that it shouldn't be using them.
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>which is the modification i proposed to lvm tools, isn't it?
>
>You suggested deleting the partition table after it has already been 
>detected.  I am saying that while I agree in principal that partition 
I was referring to the modification to lvm2, not the one to dmraid.

>detection should be moved out of the kernel, for now, it is in there and 
>deleting them after they have already been detected doesn't help matters 
>because pvscan may already be running on them.
Actually i am not worried too much about vgscan/pvscan, i am much more
worried for mount, swapon, one particular piece of crap called
hibernate-cleanup.sh etc.
besides, i am positive that users get confused having both /dev/sda1 and
/dev/mapper/via_aggehhahyue1

>>>Udev is supposed to be the new model for enumerating devices and 
>>i know that, and i will withdraw from this discussion, since it might
>>get to an useless flame war.
>>
>>Is there any technical reason for not having lvm tools filter out 
>>devices that
>>are used by device mapper?
>>
>>besides dmraid, think of multipath.
>
>None that I can see at the moment, but that doesn't mean there isn't 
>one, or won't be one in the future.  The other problem is that there are 
the above is called FUD

>likely other factors besides being used already as a dm target that 
>might give reason for lvm to not scan the volume.  These kind of policy 
probably, we strive to be perfect, but we still have a long way to
go...

>decisions seem like they should be made by udev rather than hard coded 
>into lvm.  If the admin wants a policy where lvm should look at volumes, 
>or indeed, maybe only certain volumes, that already happen to be dm 
>targets, he should be able to do that.  Likewise, there may be some 
udev is not the only thing on earth that wants to activate a volume
group. what if i wanted to do it manually?

>other reason to not look at a disk for lvm pvs.  Editing a conf file to 
>specify a filter list of devices by name is all well and good for a 
>static system, but it does not play well in the modern udev managed plug 
>and play world.
whoever wrote about editing the conf file?
i wrote about detecting that a device is already in user by
device-mapper and skipping that.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Fri, Aug 03, 2007 at 05:30:00PM -0400, Phillip Susi wrote:
>Luca Berra wrote:
>>On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
>>>I agree with moving the partition detection code to user space, but 
>>>trying to undo it after the fact doesn't help because udev is already 
>>>processing the add events.  Also you do not need to remove the 
>>>partitions so long as pvscan understands that it shouldn't be using them.
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>which is the modification i proposed to lvm tools, isn't it?
>
>You suggested deleting the partition table after it has already been 
>detected.  I am saying that while I agree in principal that partition 
I was referring to the modification to lvm2, not the one to dmraid.

>detection should be moved out of the kernel, for now, it is in there and 
>deleting them after they have already been detected doesn't help matters 
>because pvscan may already be running on them.
Actually i am not worried too much about vgscan/pvscan, i am much more
worried for mount, swapon, one particular piece of crap called
hibernate-cleanup.sh etc.
besides, i am positive that users get confused having both /dev/sda1 and
/dev/mapper/via_aggehhahyue1

>>>Udev is supposed to be the new model for enumerating devices and 
>>i know that, and i will withdraw from this discussion, since it might
>>get to an useless flame war.
>>
>>Is there any technical reason for not having lvm tools filter out 
>>devices that
>>are used by device mapper?
>>
>>besides dmraid, think of multipath.
>
>None that I can see at the moment, but that doesn't mean there isn't 
>one, or won't be one in the future.  The other problem is that there are 
the above is called FUD

>likely other factors besides being used already as a dm target that 
>might give reason for lvm to not scan the volume.  These kind of policy 
probably, we strive to be perfect, but we still have a long way to
go...

>decisions seem like they should be made by udev rather than hard coded 
>into lvm.  If the admin wants a policy where lvm should look at volumes, 
>or indeed, maybe only certain volumes, that already happen to be dm 
>targets, he should be able to do that.  Likewise, there may be some 
udev is not the only thing on earth that wants to activate a volume
group. what if i wanted to do it manually?

>other reason to not look at a disk for lvm pvs.  Editing a conf file to 
>specify a filter list of devices by name is all well and good for a 
>static system, but it does not play well in the modern udev managed plug 
>and play world.
whoever wrote about editing the conf file?
i wrote about detecting that a device is already in user by
device-mapper and skipping that.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Luca Berra wrote:
> the above is called FUD

No, it is called extensible design.  I didn't make claims that the sky 
is falling or any other such nonsense, I simply pointed out what new 
requirements may be added in the future, and that if it isn't too hard, 
we should design the system now to be able to handle those new 
requirements.

> udev is not the only thing on earth that wants to activate a volume
> group. what if i wanted to do it manually?

Then you do so manually.  This discussion is about what the system does 
automatically.

> whoever wrote about editing the conf file?
> i wrote about detecting that a device is already in user by
> device-mapper and skipping that.

Johnathan Brassow suggested adding the devices claimed by dmraid to 
lvm's filter spec in its conf file.

What I am saying is to generalize your suggestion.  Rather than 
specifically code the lvm tools to use the dm ioctls to check if a 
device is in use and avoid using it, take a more general approach that 
will work on similar problems as well, that don't involve device mapper 
targeting the underlying device, and may be easier to implement.

If udev uses pvscan on each disk to find out if it is a member of a vg, 
it can then note which vg it is a member of in its db.  Then it can 
invoke lvm to attempt to activate that vg, explicitly telling lvm which 
devices comprise the known pvs of that vg ( since it knows this 
information ), rather than letting lvm scan /dev/sd*.  When dmraid 
activates a raid set, udev can note that the physical disk is claimed by 
dmraid, and it will never ask lvm to do anything with it.

Not only does that solve the lvm/dmraid problem, but any future reasons 
that arise for lvm NOT to scan a given volume can be solved using the 
same udev attributes rather than having to patch lvm ( and dmraid ).

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Mon, Aug 06, 2007 at 10:45:02AM -0400, Phillip Susi wrote:
>>udev is not the only thing on earth that wants to activate a volume
>>group. what if i wanted to do it manually?
>
>Then you do so manually.  This discussion is about what the system does 
>automatically.
And by activating a vg manually i don't benefit from the filter? why?

What about if it is a ha-cluster suite that needs to activate the volume
group. I can't have udev activate that on sight, it will probably
corrupt data.

>>whoever wrote about editing the conf file?
>>i wrote about detecting that a device is already in user by
>>device-mapper and skipping that.
>
>Johnathan Brassow suggested adding the devices claimed by dmraid to 
>lvm's filter spec in its conf file.

probably because this is the only available way now.
anyway this method tends to suck when you have a complex setup.
and lvm device filters are not very intuitive to setup.

>What I am saying is to generalize your suggestion.  Rather than 
>specifically code the lvm tools to use the dm ioctls to check if a 
>device is in use and avoid using it, take a more general approach that 
>will work on similar problems as well, that don't involve device mapper 
>targeting the underlying device, and may be easier to implement.

Let's make a deal, you code that in udev, and in the meantime try to
convince Linus in ditching partition detection code from kernel.
I will code the filter for current lvm.

>If udev uses pvscan on each disk to find out if it is a member of a vg, 
>it can then note which vg it is a member of in its db.  Then it can 
>invoke lvm to attempt to activate that vg, explicitly telling lvm which 
>devices comprise the known pvs of that vg ( since it knows this 
>information ), rather than letting lvm scan /dev/sd*.  When dmraid 
>activates a raid set, udev can note that the physical disk is claimed by 
>dmraid, and it will never ask lvm to do anything with it.
doing this will require changing how lvm tools work.
1) disable the automatic vgscan performed by every lvm command
2) modify pvscan to be able to scan a single device

L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Luca Berra]

On Mon, Aug 06, 2007 at 10:45:02AM -0400, Phillip Susi wrote:
>>udev is not the only thing on earth that wants to activate a volume
>>group. what if i wanted to do it manually?
>
>Then you do so manually.  This discussion is about what the system does 
>automatically.
And by activating a vg manually i don't benefit from the filter? why?

What about if it is a ha-cluster suite that needs to activate the volume
group. I can't have udev activate that on sight, it will probably
corrupt data.

>>whoever wrote about editing the conf file?
>>i wrote about detecting that a device is already in user by
>>device-mapper and skipping that.
>
>Johnathan Brassow suggested adding the devices claimed by dmraid to 
>lvm's filter spec in its conf file.

probably because this is the only available way now.
anyway this method tends to suck when you have a complex setup.
and lvm device filters are not very intuitive to setup.

>What I am saying is to generalize your suggestion.  Rather than 
>specifically code the lvm tools to use the dm ioctls to check if a 
>device is in use and avoid using it, take a more general approach that 
>will work on similar problems as well, that don't involve device mapper 
>targeting the underlying device, and may be easier to implement.

Let's make a deal, you code that in udev, and in the meantime try to
convince Linus in ditching partition detection code from kernel.
I will code the filter for current lvm.

>If udev uses pvscan on each disk to find out if it is a member of a vg, 
>it can then note which vg it is a member of in its db.  Then it can 
>invoke lvm to attempt to activate that vg, explicitly telling lvm which 
>devices comprise the known pvs of that vg ( since it knows this 
>information ), rather than letting lvm scan /dev/sd*.  When dmraid 
>activates a raid set, udev can note that the physical disk is claimed by 
>dmraid, and it will never ask lvm to do anything with it.
doing this will require changing how lvm tools work.
1) disable the automatic vgscan performed by every lvm command
2) modify pvscan to be able to scan a single device

L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Luca Berra wrote:
> And by activating a vg manually i don't benefit from the filter? why?

Huh?  I fail to see the direction you are going in with this and what it 
has to do with udev auto configuring things.

> What about if it is a ha-cluster suite that needs to activate the volume
> group. I can't have udev activate that on sight, it will probably
> corrupt data.

Right, which is why you wouldn't be using udev in such a set up.  At 
least not configured to auto mount disks when plugged in.

> probably because this is the only available way now.
> anyway this method tends to suck when you have a complex setup.
> and lvm device filters are not very intuitive to setup.

You just made my point for me.

> Let's make a deal, you code that in udev, and in the meantime try to
> convince Linus in ditching partition detection code from kernel.
> I will code the filter for current lvm.

The partition table code already can be disabled in the kernel config 
iirc.

> doing this will require changing how lvm tools work.
> 1) disable the automatic vgscan performed by every lvm command
> 2) modify pvscan to be able to scan a single device

Correct.  The commands need to be able to take a list of devices to 
manipulate rather than going after /dev/sd*.

Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Wed, Aug 01, 2007 at 04:19:59PM -0400, Phillip Susi wrote:
> I am investigating a bug report involving the combination of lvm and 
> dmraid, and it seems to me that the problem is that lvm is detecting the 
> underlying partition on the primary disk and accessing it directly 
> rather than going through the raid device made by dmraid.  I think that 
> the problem is there is no facility for dmraid to "claim" the physical 
> disk so that lvm does not look at it, and also to have lvm scan the raid 
> device for physical volumes.

LVM tries hard to 'do the right thing' in cases like this.  If it's
getting it wrong in some particular case, we need to fix it.  I presume
it gives you a warning message about the duplicate UUID?

If this can be reproduced using the latest upstream source code release,
please supply relevant details (lvm.conf, -vvvv traces & dmsetup output)
- you can run 'lvm_dump.sh -a' (in the scripts dir) to capture this.
 
Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 08:50:12AM +0200, Luca Berra wrote:
> 2) lvm tool filter could be modified to check if a device has been
> already claimed by device-mapper.
> something along the lines of:
> ioctl DM_LIST_DEVICES
> while (...)
>  ioctl DM_TABLE_DEPS

There are already some checks in the code, but they are have not been completed.

But a dm device should already take precedence over a scsi device.

Alasdair
-- 
agk@redhat.com

Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 08:50:12AM +0200, Luca Berra wrote:
> 2) lvm tool filter could be modified to check if a device has been
> already claimed by device-mapper.
> something along the lines of:
> ioctl DM_LIST_DEVICES
> while (...)
>  ioctl DM_TABLE_DEPS

There are already some checks in the code, but they are have not been completed.

But a dm device should already take precedence over a scsi device.

Alasdair
-- 
agk@redhat.com

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 05:50:51PM -0400, Phillip Susi wrote:
> The problem is that lvm is looking at the physical disks at all.  It 
> needs to scan the dmraid device, not the underlying disks.
 
Indeed.  Please send the evidence showing where it is going wrong so
we can fix it.

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 05:50:51PM -0400, Phillip Susi wrote:
> The problem is that lvm is looking at the physical disks at all.  It 
> needs to scan the dmraid device, not the underlying disks.
 
Indeed.  Please send the evidence showing where it is going wrong so
we can fix it.

Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 06:04:20PM -0400, Phillip Susi wrote:
> I'd rather not see it that tightly coupled to dm.  I was thinking 
> perhaps of something involving udev attributes so that udev would know 
> not to run pvscan on that device.  Though it looks like pvscan also 
> needs reworked so it plays nice with udev, being called to scan a given 
> device as detected rather than looking for all well known physical 
> device names in /dev.

It's been fairly clear how this should all work from the lvm2 side for
some time now, but it will still take a while to implement.

As I've said many times before, in my view the current problems stem
from some system components unilaterally switching to an event-driven
model while others (including lvm2 and initscripts) haven't - and
combining two incompatible models on a system was "brave".

If lvm2 is run in an event-driven environment, the idea is for it to
give up all its device scanning and filtering.  It will hand over
control of what devices to use to an external module shared by
everything that needs to know about devices - including mdadm,
initscripts, mount, hal, udev etc.  [What owns this module is irrelevant
but udev is an obvious candidate.]

When udev sees a new device, it will inform each subsystem of the
device.

lvm2 will provide an interface that is given a device and, if a PV is
present on it, lvm2 will pass information about it (including its UUID)
to the external module for storage.

The external module will have an interface capable of being told
by lvm2 (at any time) 'Devices with UUIDs X, Y and Z form a volume
group called VGn'.

Triggers can be defined so that when all the PV UUIDs are present
for a volume group with name VGn, an lvm2 command (like 'vgchange -ay
VGn') gets invoked.
 
You can see how md, mount etc. can work in a similar event-driven
fashion.

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 02, 2007 at 06:04:20PM -0400, Phillip Susi wrote:
> I'd rather not see it that tightly coupled to dm.  I was thinking 
> perhaps of something involving udev attributes so that udev would know 
> not to run pvscan on that device.  Though it looks like pvscan also 
> needs reworked so it plays nice with udev, being called to scan a given 
> device as detected rather than looking for all well known physical 
> device names in /dev.

It's been fairly clear how this should all work from the lvm2 side for
some time now, but it will still take a while to implement.

As I've said many times before, in my view the current problems stem
from some system components unilaterally switching to an event-driven
model while others (including lvm2 and initscripts) haven't - and
combining two incompatible models on a system was "brave".

If lvm2 is run in an event-driven environment, the idea is for it to
give up all its device scanning and filtering.  It will hand over
control of what devices to use to an external module shared by
everything that needs to know about devices - including mdadm,
initscripts, mount, hal, udev etc.  [What owns this module is irrelevant
but udev is an obvious candidate.]

When udev sees a new device, it will inform each subsystem of the
device.

lvm2 will provide an interface that is given a device and, if a PV is
present on it, lvm2 will pass information about it (including its UUID)
to the external module for storage.

The external module will have an interface capable of being told
by lvm2 (at any time) 'Devices with UUIDs X, Y and Z form a volume
group called VGn'.

Triggers can be defined so that when all the PV UUIDs are present
for a volume group with name VGn, an lvm2 command (like 'vgchange -ay
VGn') gets invoked.
 
You can see how md, mount etc. can work in a similar event-driven
fashion.

Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 10:40:45AM +1000, David Robinson wrote:
> If you need LVM to only scan certain devices you should modify the 
> filter line in lvm.conf.
 
Yes, this is the current approach: a well-configured system would be
actively modifying the filters (with the help of udev) both on boot and
if device configurations change subsequently.

Alasdair
-- 
agk@redhat.com

Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 10:40:45AM +1000, David Robinson wrote:
> If you need LVM to only scan certain devices you should modify the 
> filter line in lvm.conf.
 
Yes, this is the current approach: a well-configured system would be
actively modifying the filters (with the help of udev) both on boot and
if device configurations change subsequently.

Alasdair
-- 
agk@redhat.com

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 10:11:34AM +0200, Luca Berra wrote:
> i'd rather not see it coupled with udev :P
> maybe i am limited, but i really fail to see how an event driven model
> could be at all useful in this cases, and i am really convinced that the
> effort needed to make i work is too high compared to possible benefits.

This is out of our hands - people are already experimenting with this
and lvm2 needs to support it with enthusiasm!

But that does not mean we'll stop supporting the existing mechanisms, as
it'll just become a configuration option (I suspect compile-time rather
than run-time).

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 10:11:34AM +0200, Luca Berra wrote:
> i'd rather not see it coupled with udev :P
> maybe i am limited, but i really fail to see how an event driven model
> could be at all useful in this cases, and i am really convinced that the
> effort needed to make i work is too high compared to possible benefits.

This is out of our hands - people are already experimenting with this
and lvm2 needs to support it with enthusiasm!

But that does not mean we'll stop supporting the existing mechanisms, as
it'll just become a configuration option (I suspect compile-time rather
than run-time).

Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
> Where does lvm store state 
> information right now?  In conf files in /etc isn't it?  

It uses an untrusted cache in /etc/lvm as hints to speed up its
processing if the hints prove to be still correct.  It can't ever trust
because things can change on the system without lvm2's knowledge.  And
device scanning causes us no end of problems - particularly in a
clustered environment.  The new model should provide us with a
completely-reliable cache.

And why does lvm2 today have to include code to check for partition
tables, md devices, filesystems etc.?

If all these components co-operated through common interfaces there'd be
no need!

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 02:10:02PM -0400, Phillip Susi wrote:
> Where does lvm store state 
> information right now?  In conf files in /etc isn't it?  

It uses an untrusted cache in /etc/lvm as hints to speed up its
processing if the hints prove to be still correct.  It can't ever trust
because things can change on the system without lvm2's knowledge.  And
device scanning causes us no end of problems - particularly in a
clustered environment.  The new model should provide us with a
completely-reliable cache.

And why does lvm2 today have to include code to check for partition
tables, md devices, filesystems etc.?

If all these components co-operated through common interfaces there'd be
no need!

Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 05:30:00PM -0400, Phillip Susi wrote:
> Editing a conf file to 
> specify a filter list of devices by name is all well and good for a 
> static system, but it does not play well in the modern udev managed plug 
> and play world.
 
BTW The filters could of course be generated dynamically (from a database) and
supplied on the command line (with --config) - or lvm2 could be extended to
read them from a database.

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Fri, Aug 03, 2007 at 05:30:00PM -0400, Phillip Susi wrote:
> Editing a conf file to 
> specify a filter list of devices by name is all well and good for a 
> static system, but it does not play well in the modern udev managed plug 
> and play world.
 
BTW The filters could of course be generated dynamically (from a database) and
supplied on the command line (with --config) - or lvm2 could be extended to
read them from a database.

Alasdair
-- 
agk@redhat.com

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Tue, Aug 07, 2007 at 11:17:47AM +0200, Luca Berra wrote:
> 1) disable the automatic vgscan performed by every lvm command

With access to a 100% accurate external device database instead of the
existing .cache this is easy.
The --trustcache argument would be enabled by default on such a system.

> 2) modify pvscan to be able to scan a single device

This would actually just be something like 'pvs -o uuid <device>' with
a subtle modification so it notices there are no VG fields required
and so it doesn't attempt to access the VG metadata (which might
require accessing disks other than the one specified).

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Tue, Aug 07, 2007 at 11:17:47AM +0200, Luca Berra wrote:
> 1) disable the automatic vgscan performed by every lvm command

With access to a 100% accurate external device database instead of the
existing .cache this is easy.
The --trustcache argument would be enabled by default on such a system.

> 2) modify pvscan to be able to scan a single device

This would actually just be something like 'pvs -o uuid <device>' with
a subtle modification so it notices there are no VG fields required
and so it doesn't attempt to access the VG metadata (which might
require accessing disks other than the one specified).

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> On Fri, Aug 03, 2007 at 10:40:45AM +1000, David Robinson wrote:
>> If you need LVM to only scan certain devices you should modify the 
>> filter line in lvm.conf.
>  
> Yes, this is the current approach: a well-configured system would be
> actively modifying the filters (with the help of udev) both on boot and
> if device configurations change subsequently.

No.  The config files are for administrators to modify.  Udev should not 
be automagically messing with things in /etc.

Re: [dm-devel] Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> On Thu, Aug 02, 2007 at 05:50:51PM -0400, Phillip Susi wrote:
>> The problem is that lvm is looking at the physical disks at all.  It 
>> needs to scan the dmraid device, not the underlying disks.
>  
> Indeed.  Please send the evidence showing where it is going wrong so
> we can fix it.

https://bugs.launchpad.net/ubuntu/+source/dmraid/+bug/129285

Note the dm table listing:

nvidia_ajafeffc: 0 625142446 mirror core 2 131072 nosync 2 8:0 0 8:16 0

nvidia_ajafeffc4: 0 464776515 linear 254:2 160360830

nvidia_ajafeffc3: 0 195313 linear 254:2 160156250

nvidia_ajafeffc2: 0 78236550 linear 254:2 81915435

nvidia_ajafeffc1: 0 81915372 linear 254:2 63

volgroup-swap: 0 1843200 linear 8:2 76390784

volgroup-root: 0 76390400 linear 8:2 384

This is do to the fact that the lvm tools open /dev/sd* and look for lvm 
partitions, which it finds.

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> On Tue, Aug 07, 2007 at 11:17:47AM +0200, Luca Berra wrote:
>> 1) disable the automatic vgscan performed by every lvm command
> 
> With access to a 100% accurate external device database instead of the
> existing .cache this is easy.
> The --trustcache argument would be enabled by default on such a system.

It doesn't even need to look somewhere special for a cache, just needs 
to be passed the list of devices on the command line it should open 
rather than generating a list from a readdir() of /dev.

Re: [dm-devel] Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> As I've said many times before, in my view the current problems stem
> from some system components unilaterally switching to an event-driven
> model while others (including lvm2 and initscripts) haven't - and
> combining two incompatible models on a system was "brave".

Actually, none of the involved components here are event driven yet. 
Right now dmraid and lvm are just run during boot time in the init 
scripts.

> If lvm2 is run in an event-driven environment, the idea is for it to
> give up all its device scanning and filtering.  It will hand over
> control of what devices to use to an external module shared by
> everything that needs to know about devices - including mdadm,
> initscripts, mount, hal, udev etc.  [What owns this module is irrelevant
> but udev is an obvious candidate.]

Yes.

> When udev sees a new device, it will inform each subsystem of the
> device.
> 
> lvm2 will provide an interface that is given a device and, if a PV is
> present on it, lvm2 will pass information about it (including its UUID)
> to the external module for storage.
> 
> The external module will have an interface capable of being told
> by lvm2 (at any time) 'Devices with UUIDs X, Y and Z form a volume
> group called VGn'.
> 
> Triggers can be defined so that when all the PV UUIDs are present
> for a volume group with name VGn, an lvm2 command (like 'vgchange -ay
> VGn') gets invoked.
>  
> You can see how md, mount etc. can work in a similar event-driven
> fashion.

Exactly.  Then policy decisions like whether lvm should be allowed to 
auto detect pvs in disks being claimed by dmraid can be easily made with 
udev scripts by the distro rather than having to modify lvm, and 
presumably, all of the other components that are interacting in the 
system.

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Wed, Aug 08, 2007 at 05:04:12PM -0400, Phillip Susi wrote:
> https://bugs.launchpad.net/ubuntu/+source/dmraid/+bug/129285
 
There's something not quite right there:
  Why doesn't the 'lvdisplay' issue a message about the duplicate PV?
  - run 'vgscan -vvvv' to see what lvm2 is really doing with all the
    relevant devices

  What lvm2 version? 
   - could be old and not containing the code to do the detection
     correctly?

  Are there non-upstream-default config settings that had the effect of turning
off the detection?  
   - see 'lvm dumpconfig' output or /etc/lvm/lvm.conf plus any compile-time
     default changes


Standard procedure these days is to ask people to run 'lvmdump -a'
(scripts/lvm_dump.sh in the source tree) to capture the relevant information
for examination.

> This is do to the fact that the lvm tools open /dev/sd* and look for lvm 
> partitions, which it finds.
 
But I don't yet see why it didn't also open the other devices, write a warning
message about duplicate IDs, and do the right thing and give them
precedence.
 
Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Wed, Aug 08, 2007 at 05:04:12PM -0400, Phillip Susi wrote:
> https://bugs.launchpad.net/ubuntu/+source/dmraid/+bug/129285
 
There's something not quite right there:
  Why doesn't the 'lvdisplay' issue a message about the duplicate PV?
  - run 'vgscan -vvvv' to see what lvm2 is really doing with all the
    relevant devices

  What lvm2 version? 
   - could be old and not containing the code to do the detection
     correctly?

  Are there non-upstream-default config settings that had the effect of turning
off the detection?  
   - see 'lvm dumpconfig' output or /etc/lvm/lvm.conf plus any compile-time
     default changes


Standard procedure these days is to ask people to run 'lvmdump -a'
(scripts/lvm_dump.sh in the source tree) to capture the relevant information
for examination.

> This is do to the fact that the lvm tools open /dev/sd* and look for lvm 
> partitions, which it finds.
 
But I don't yet see why it didn't also open the other devices, write a warning
message about duplicate IDs, and do the right thing and give them
precedence.
 
Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> But I don't yet see why it didn't also open the other devices, write a warning
> message about duplicate IDs, and do the right thing and give them
> precedence.

Why would it open the other devices?  LVM does not scan /dev/mapper/*, 
only /dev/sd*.  Since it does not scan the /dev/mapper devices for pvs, 
it sees no duplicates and can not "do the right thing".

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 09, 2007 at 01:27:34PM -0400, Phillip Susi wrote:
>Alasdair G Kergon wrote:
>>But I don't yet see why it didn't also open the other devices, write a 
>>warning
>>message about duplicate IDs, and do the right thing and give them
>>precedence.
>
>Why would it open the other devices?  LVM does not scan /dev/mapper/*, 
>only /dev/sd*.  Since it does not scan the /dev/mapper devices for pvs, 
>it sees no duplicates and can not "do the right thing".

LVM scans the whole /dev directory, unless you configure it otherwise.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: Re: LVM on dmraid breakage

[Luca Berra]

On Thu, Aug 09, 2007 at 01:27:34PM -0400, Phillip Susi wrote:
>Alasdair G Kergon wrote:
>>But I don't yet see why it didn't also open the other devices, write a 
>>warning
>>message about duplicate IDs, and do the right thing and give them
>>precedence.
>
>Why would it open the other devices?  LVM does not scan /dev/mapper/*, 
>only /dev/sd*.  Since it does not scan the /dev/mapper devices for pvs, 
>it sees no duplicates and can not "do the right thing".

LVM scans the whole /dev directory, unless you configure it otherwise.

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

[linux-lvm] Re: [dm-devel] Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 09, 2007 at 01:27:34PM -0400, Phillip Susi wrote:
> Why would it open the other devices?  LVM does not scan /dev/mapper/*, 
> only /dev/sd*.

Upstream lvm2 has done for a long time (since 2.02.00 with quite a few
fixes and tweaks after that, and one known problem still outstanding).
I can't speak for any customisation done in ubuntu or on the machine
with the problem - hence my request for full version and configuration
information.

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Alasdair G Kergon]

On Thu, Aug 09, 2007 at 01:27:34PM -0400, Phillip Susi wrote:
> Why would it open the other devices?  LVM does not scan /dev/mapper/*, 
> only /dev/sd*.

Upstream lvm2 has done for a long time (since 2.02.00 with quite a few
fixes and tweaks after that, and one known problem still outstanding).
I can't speak for any customisation done in ubuntu or on the machine
with the problem - hence my request for full version and configuration
information.

Alasdair
-- 
agk@redhat.com

Re: Re: LVM on dmraid breakage

[Phillip Susi]

Alasdair G Kergon wrote:
> Upstream lvm2 has done for a long time (since 2.02.00 with quite a few
> fixes and tweaks after that, and one known problem still outstanding).
> I can't speak for any customisation done in ubuntu or on the machine
> with the problem - hence my request for full version and configuration
> information.

It scans everything in /dev and all subdirectories?  Wow that would be 
colossally slow and prone to hang.  Imagine trying to access a nd device 
whose server is down.

In any case, if it did scan the dmraid device, it would just complain 
about finding duplicates; it has no way of knowing which one it should 
use.

Also it looks like feisty used 2.2.06 and the config files shipped in 
the package can be found here:

http://packages.ubuntu.com/feisty/admin/lvm2