[linux-lvm] Hide volume group during startup

[linux-lvm] Hide volume group during startup

[Erik Schwalbe]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

Hi, 

we have a problem with hiding/deactivating a stacked volume group during startup. 

System is a Ubuntu 10.4.4 LTS (NAS). 
lvm2: 2.02.54-1ubuntu4.1ppa5 

lvs: 
backup-all data -wi-ao 3.00t 
fileserver-forBackup data -wi-ao 50.00g 
.... 
kvm_disk0 data -wi-ao 400.00g 
vm-203-disk-1 kvm_disk0 -wi-ao- 28.01g 
.... 

kvm_disk0 is used as device for iscsitarget. 

The iscsiinitiator created a new volume group (kvm_disk0) on the blockdevice and new virtual disks (vm-203-disk-1) . 

Our problem is, that the iscsitarget host found not only the VG "data" during system startup but also the other VG (kvm_disk0) and activate it. 
So DRBD can't get exclusive access to the LV kvm_disk0. 

We try to filter in lvm.conf, but after startup, the VG kvm_disk0 is allways activated. 

Thank you for help. 

Regards, 
Erik 








[-- Attachment #2: Type: text/html, Size: 2871 bytes --]

Re: [linux-lvm] Hide volume group during startup

[Bryn M. Reeves]

On 04/11/2012 11:04 AM, Erik Schwalbe wrote:
> The iscsiinitiator created a new volume group (kvm_disk0) on the
> blockdevice and new virtual disks (vm-203-disk-1).
> 
> Our problem is, that the iscsitarget host found not only the VG "data"
> during system startup but also the other VG (kvm_disk0) and activate it.
> So DRBD can't get exclusive access to the LV kvm_disk0.
> 
> We try to filter in lvm.conf, but after startup, the VG kvm_disk0 is
> allways activated.

Please post the filter you are using as well as the full path name of
the disk you are attempting to filter (a complete recursive listing of
/dev as generated by lvmdump or run by hand would be useful. You can use
a service like pastebin for this - please don't send large attachments
to the list).

There are other ways to control activation if for some reason filtering
will not work (e.g. the device name is not stable).

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Hi,

filter was:

filter = [ "r|/dev/data/kvm_disk0|" ]

path:
/dev/data/kvm_disk0 -> ../mapper/data-kvm_disk0

The device name is stable, but what other way of control activation are possible??

Regards,
Erik

----- Original Message -----
From: "Bryn M. Reeves" <bmr@redhat.com>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Cc: "Erik Schwalbe" <erik.schwalbe@canoo.com>
Sent: Thursday, 12 April, 2012 1:11:44 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On 04/11/2012 11:04 AM, Erik Schwalbe wrote:
> The iscsiinitiator created a new volume group (kvm_disk0) on the
> blockdevice and new virtual disks (vm-203-disk-1).
> 
> Our problem is, that the iscsitarget host found not only the VG "data"
> during system startup but also the other VG (kvm_disk0) and activate it.
> So DRBD can't get exclusive access to the LV kvm_disk0.
> 
> We try to filter in lvm.conf, but after startup, the VG kvm_disk0 is
> allways activated.

Please post the filter you are using as well as the full path name of
the disk you are attempting to filter (a complete recursive listing of
/dev as generated by lvmdump or run by hand would be useful. You can use
a service like pastebin for this - please don't send large attachments
to the list).

There are other ways to control activation if for some reason filtering
will not work (e.g. the device name is not stable).

Regards,
Bryn.

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Hide volume group during startup

[Bryn M. Reeves]

On 04/12/2012 12:41 PM, Erik Schwalbe wrote:
> Hi,
> 
> filter was:
> 
> filter = [ "r|/dev/data/kvm_disk0|" ]
> 
> path:
> /dev/data/kvm_disk0 -> ../mapper/data-kvm_disk0

To filter a device-mapper device you also have to filter out the
/dev/dm-N device nodes that udev manages. There's also a symlink from
/dev/$VG_NAME/$LV_NAME to the dm-N node (although I think that should be
handled by filtering /dev/dm-N).

I'd be using something like:

    [ "r|/dev/dm.*|", "r|/dev/$VG_NAME|", "a|.*|"]

Alternately (and I prefer this for many configurations) you can accept
("a|..|") the device you need for the data VG and reject all other
devices ("r|.*|").

> The device name is stable, but what other way of control activation are possible??

Tags but since you're not currently filtering dm-N nodes I would try
that first.

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Lars Ellenberg]

On Thu, Apr 12, 2012 at 01:41:31PM +0200, Erik Schwalbe wrote:
> Hi,
> 
> filter was:
> 
> filter = [ "r|/dev/data/kvm_disk0|" ]
> 
> path:
> /dev/data/kvm_disk0 -> ../mapper/data-kvm_disk0
> 
> The device name is stable, but what other way of control activation are possible??

Forgot to update the initramfs/initrd,
and the VGs are activated from there
already, even before pivoting to the real root?

Run update-initramfs or
mkinitrd /boot/initramfs-`uname -r`.img `uname -r`
or something like that.

> 
> Regards,
> Erik
> 
> ----- Original Message -----
> From: "Bryn M. Reeves" <bmr@redhat.com>
> To: "LVM general discussion and development" <linux-lvm@redhat.com>
> Cc: "Erik Schwalbe" <erik.schwalbe@canoo.com>
> Sent: Thursday, 12 April, 2012 1:11:44 PM
> Subject: Re: [linux-lvm] Hide volume group during startup
> 
> On 04/11/2012 11:04 AM, Erik Schwalbe wrote:
> > The iscsiinitiator created a new volume group (kvm_disk0) on the
> > blockdevice and new virtual disks (vm-203-disk-1).
> > 
> > Our problem is, that the iscsitarget host found not only the VG "data"
> > during system startup but also the other VG (kvm_disk0) and activate it.
> > So DRBD can't get exclusive access to the LV kvm_disk0.
> > 
> > We try to filter in lvm.conf, but after startup, the VG kvm_disk0 is
> > allways activated.
> 
> Please post the filter you are using as well as the full path name of
> the disk you are attempting to filter (a complete recursive listing of
> /dev as generated by lvmdump or run by hand would be useful. You can use
> a service like pastebin for this - please don't send large attachments
> to the list).
> 
> There are other ways to control activation if for some reason filtering
> will not work (e.g. the device name is not stable).
> 
> Regards,
> Bryn.

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Hi,

there is only one dm-device in /dev:

...
drwxr-xr-x  5 root root         100 2012-04-05 12:50 disk
brw-rw----  1 root disk    251,  18 2012-04-05 12:51 dm-18
drwxr-xr-x  4 root root          80 2012-04-05 12:51 drbd
...

/dev/data/kvm_disk0 is a blockdevice for drbd, so there are many drbd files in /dev

...
brw-rw----  1 root disk    147,  20 2012-04-08 11:48 drbd20
...

drbd20 is the drbd resource using kvm_disk0 as device.

The symlink in /dev/data/ is:

lrwxrwxrwx  1 root root   24 2012-04-05 12:51 kvm_disk0 -> ../mapper/data-kvm_disk0

Should I be using this:

filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|", "a|.*|"]
 
I know, it is a little bit confusing.
The LV in VG data is called kvm_disk0 and the VG initiated on this device is also called kvm_disk0.

Sorry, but thank you for help.

Regards,
Erik


----- Original Message -----
From: "Bryn M. Reeves" <bmr@redhat.com>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Cc: "Erik Schwalbe" <erik.schwalbe@canoo.com>
Sent: Thursday, 12 April, 2012 1:52:20 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On 04/12/2012 12:41 PM, Erik Schwalbe wrote:
> Hi,
> 
> filter was:
> 
> filter = [ "r|/dev/data/kvm_disk0|" ]
> 
> path:
> /dev/data/kvm_disk0 -> ../mapper/data-kvm_disk0

To filter a device-mapper device you also have to filter out the
/dev/dm-N device nodes that udev manages. There's also a symlink from
/dev/$VG_NAME/$LV_NAME to the dm-N node (although I think that should be
handled by filtering /dev/dm-N).

I'd be using something like:

    [ "r|/dev/dm.*|", "r|/dev/$VG_NAME|", "a|.*|"]

Alternately (and I prefer this for many configurations) you can accept
("a|..|") the device you need for the data VG and reject all other
devices ("r|.*|").

> The device name is stable, but what other way of control activation are possible??

Tags but since you're not currently filtering dm-N nodes I would try
that first.

Regards,
Bryn.

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Hide volume group during startup

[Bryn M. Reeves]

On 04/12/2012 01:14 PM, Erik Schwalbe wrote:
> The symlink in /dev/data/ is:
> 
> lrwxrwxrwx  1 root root   24 2012-04-05 12:51 kvm_disk0 -> ../mapper/data-kvm_disk0

OK, you're using an older LVM2/device-mapper that does not use udev to
manage the device nodes and symlinks in /dev.

> Should I be using this:
> 
> filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|", "a|.*|"]

Yes. Or use the alternate style I showed and only accept the PVs
belonging to the data VG.

As Lars said you may also need to rebuild your initramfs although if the
data VG doesn't contain your root file system, swap or other resources
normally set up in the initramfs this shouldn't be needed.

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Ok, I will try a:

update-initramfs -u -k all 

Hope this helps.

Thank you for the hint.

Regards,
Erik

----- Original Message -----
From: "Lars Ellenberg" <lars.ellenberg@linbit.com>
To: linux-lvm@redhat.com
Sent: Thursday, 12 April, 2012 2:01:39 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On Thu, Apr 12, 2012 at 01:41:31PM +0200, Erik Schwalbe wrote:
> Hi,
> 
> filter was:
> 
> filter = [ "r|/dev/data/kvm_disk0|" ]
> 
> path:
> /dev/data/kvm_disk0 -> ../mapper/data-kvm_disk0
> 
> The device name is stable, but what other way of control activation are possible??

Forgot to update the initramfs/initrd,
and the VGs are activated from there
already, even before pivoting to the real root?

Run update-initramfs or
mkinitrd /boot/initramfs-`uname -r`.img `uname -r`
or something like that.

> 
> Regards,
> Erik
> 
> ----- Original Message -----
> From: "Bryn M. Reeves" <bmr@redhat.com>
> To: "LVM general discussion and development" <linux-lvm@redhat.com>
> Cc: "Erik Schwalbe" <erik.schwalbe@canoo.com>
> Sent: Thursday, 12 April, 2012 1:11:44 PM
> Subject: Re: [linux-lvm] Hide volume group during startup
> 
> On 04/11/2012 11:04 AM, Erik Schwalbe wrote:
> > The iscsiinitiator created a new volume group (kvm_disk0) on the
> > blockdevice and new virtual disks (vm-203-disk-1).
> > 
> > Our problem is, that the iscsitarget host found not only the VG "data"
> > during system startup but also the other VG (kvm_disk0) and activate it.
> > So DRBD can't get exclusive access to the LV kvm_disk0.
> > 
> > We try to filter in lvm.conf, but after startup, the VG kvm_disk0 is
> > allways activated.
> 
> Please post the filter you are using as well as the full path name of
> the disk you are attempting to filter (a complete recursive listing of
> /dev as generated by lvmdump or run by hand would be useful. You can use
> a service like pastebin for this - please don't send large attachments
> to the list).
> 
> There are other ways to control activation if for some reason filtering
> will not work (e.g. the device name is not stable).
> 
> Regards,
> Bryn.

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Hi,

sorry but it did not work:

I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter and do a update-initramfs -u -k all
But after reboot the VG is always activate. 

...
kvm_disk0                   data      -wi-ao 400.00g                                                                            
  vm-203-disk-1               kvm_disk0 -wi-a-  28.01g                                      
  vm-203-disk-2               kvm_disk0 -wi-a-  80.00g
...

Regards,
Erik


----- Original Message -----
From: "Bryn M. Reeves" <bmr@redhat.com>
To: "LVM general discussion and development" <linux-lvm@redhat.com>
Cc: "Erik Schwalbe" <erik.schwalbe@canoo.com>
Sent: Thursday, 12 April, 2012 2:26:34 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On 04/12/2012 01:14 PM, Erik Schwalbe wrote:
> The symlink in /dev/data/ is:
> 
> lrwxrwxrwx  1 root root   24 2012-04-05 12:51 kvm_disk0 -> ../mapper/data-kvm_disk0

OK, you're using an older LVM2/device-mapper that does not use udev to
manage the device nodes and symlinks in /dev.

> Should I be using this:
> 
> filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|", "a|.*|"]

Yes. Or use the alternate style I showed and only accept the PVs
belonging to the data VG.

As Lars said you may also need to rebuild your initramfs although if the
data VG doesn't contain your root file system, swap or other resources
normally set up in the initramfs this shouldn't be needed.

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Bryn M. Reeves]

On 04/12/2012 02:31 PM, Erik Schwalbe wrote:
> Hi,
> 
> sorry but it did not work:
> 
> I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter

Test it at this point. There is no need to reboot. If
pvs/vgchange/vgdisplay etc. still display the nested VG and its PV your
filters are wrong or insufficient. Examine the -vvv output from the
tools to see the filter decisions that are being made.

Since you haven't mentioned any of the versions you are using you may
also want to remove the LVM2 cache after editing the filter (current
versions clear it automatically but older releases would still show
filtered devices if they exist in the cache.

> and do a update-initramfs -u -k all

You need to know whether this is the correct command for your
distribution and that it's affecting whatever kernel/initramfs
combination the machine boots by default (and also that it actually
works and creates a new initramfs).

But for now I would ignore boot problems and focus on the filter. Don't
try to solve all the problems at once as that just makes it more confusing.

When you have confirmed that your filter settings correctly exclude the
nested VG you can move on to making sure those settings are applied at boot.

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Ok, I found a solution:

I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter:

filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|"]

without "a|.*|" !!!!

and do a update-initramfs -u -k all

Perfect :-)

It seems that the allow filter overrules the remove option.

Is my filter correct?? 

ps: lvm version: 2.02.54-1ubuntu4.1ppa5

Regards,
Erik


----- Original Message -----
From: "Bryn M. Reeves" <bmr@redhat.com>
To: "Erik Schwalbe" <erik.schwalbe@canoo.com>
Cc: "LVM general discussion and development" <linux-lvm@redhat.com>
Sent: Thursday, 12 April, 2012 3:39:55 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On 04/12/2012 02:31 PM, Erik Schwalbe wrote:
> Hi,
> 
> sorry but it did not work:
> 
> I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter

Test it at this point. There is no need to reboot. If
pvs/vgchange/vgdisplay etc. still display the nested VG and its PV your
filters are wrong or insufficient. Examine the -vvv output from the
tools to see the filter decisions that are being made.

Since you haven't mentioned any of the versions you are using you may
also want to remove the LVM2 cache after editing the filter (current
versions clear it automatically but older releases would still show
filtered devices if they exist in the cache.

> and do a update-initramfs -u -k all

You need to know whether this is the correct command for your
distribution and that it's affecting whatever kernel/initramfs
combination the machine boots by default (and also that it actually
works and creates a new initramfs).

But for now I would ignore boot problems and focus on the filter. Don't
try to solve all the problems at once as that just makes it more confusing.

When you have confirmed that your filter settings correctly exclude the
nested VG you can move on to making sure those settings are applied at boot.

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Bryn M. Reeves]

On 04/12/2012 03:14 PM, Erik Schwalbe wrote:
> Ok, I found a solution:
> 
> I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter:
> 
> filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|"]
> 
> without "a|.*|" !!!!

Shouldn't make any difference. The filter line has an implicit "a|.*|"
at the end. From the lvm.conf man page:

  "Devices that  don't  match  any patterns are accepted."

> Is my filter correct?? 

If it works for you then that's the main thing.

Just be aware that it will need adjustment if you add any more LVs that
need to be treated this way (that's why I prefer to accept the PVs I
need and reject everything else with a final "r|.*|").

Regards,
Bryn.

Re: [linux-lvm] Hide volume group during startup

[Erik Schwalbe]

Thank you very much for your help.

Regards,
Erik

----- Original Message -----
From: "Bryn M. Reeves" <bmr@redhat.com>
To: "Erik Schwalbe" <erik.schwalbe@canoo.com>
Cc: "LVM general discussion and development" <linux-lvm@redhat.com>
Sent: Thursday, 12 April, 2012 4:30:16 PM
Subject: Re: [linux-lvm] Hide volume group during startup

On 04/12/2012 03:14 PM, Erik Schwalbe wrote:
> Ok, I found a solution:
> 
> I deactivate the VG kvm_disk0 with vgchange -an kvm_disk0, after that I set the filter:
> 
> filter = [ "r|/dev/mapper/data-kvm_disk0|", "r|/dev/kvm_disk0|"]
> 
> without "a|.*|" !!!!

Shouldn't make any difference. The filter line has an implicit "a|.*|"
at the end. From the lvm.conf man page:

  "Devices that  don't  match  any patterns are accepted."

> Is my filter correct?? 

If it works for you then that's the main thing.

Just be aware that it will need adjustment if you add any more LVs that
need to be treated this way (that's why I prefer to accept the PVs I
need and reject everything else with a final "r|.*|").

Regards,
Bryn.