[linux-lvm] Removing VG mappings using dmsetup tool

[linux-lvm] Removing VG mappings using dmsetup tool

[Vojtech Juranek]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

Hi,
I'm working on storage part of oVirt project [1]. Besides other options, we 
provide iSCSI storage for VMs. We create PV(s) on attached LUN(s), create 
volume group from this PV(s) and then create LVs on this VG as needed (each 
disk on its LV, similar there's dedicated LV for our metadata etc.). VG is 
what we call (block) storage domain. 

When we remove storage domain we deactivate all LVs and remove LVs and VG from 
all hosts. However, when this fails (e.g. storage is for some reason 
unavailable), hosts are left with stale LVs and VG which can cause various 
issues for us. What we are going to do is to remove these stale VG mappings 
using dmsetup remove command (first we try to remove LVs using lvm, but if 
there are still some mappings, we will try to remove it using dmsetup). 

I'd like to head your opinion on this approach. Is it wise to use dmsetup in 
such case? Or is there any better way how to handle such situations or remove 
stale mappings?

Thanks
Vojta

[1] https://www.ovirt.org/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Zdenek Kabelac]

Dne 23. 06. 20 v 16:26 Vojtech Juranek napsal(a):
> Hi,
> I'm working on storage part of oVirt project [1]. Besides other options, we
> provide iSCSI storage for VMs. We create PV(s) on attached LUN(s), create
> volume group from this PV(s) and then create LVs on this VG as needed (each
> disk on its LV, similar there's dedicated LV for our metadata etc.). VG is
> what we call (block) storage domain.
> 
> When we remove storage domain we deactivate all LVs and remove LVs and VG from
> all hosts. However, when this fails (e.g. storage is for some reason
> unavailable), hosts are left with stale LVs and VG which can cause various
> issues for us. What we are going to do is to remove these stale VG mappings
> using dmsetup remove command (first we try to remove LVs using lvm, but if
> there are still some mappings, we will try to remove it using dmsetup).
> 
> I'd like to head your opinion on this approach. Is it wise to use dmsetup in
> such case? Or is there any better way how to handle such situations or remove
> stale mappings?

Hi

Of course you can remove 'stale' mapping with 'dmsetup remove' -
(after all lvm2 is doing basically very similar job like
'dmsetup create/remove/load/suspend/resume'....

BUT I'd probably focus rather on fixing your oVirt workflow - you should
not be 'losing' devices while you have activate mappings on them - that's
IMHO major desing flaw - you can very easily cause very SERIOUS damages
to your on disk data consistency - so disk would require serious fsck before 
reuse...

Note - you cannot 'remove' mappings 'in-use'  (aka open count of a device
is higher then 0  - see 'dmsetup info -c' output for this).
However you can replace such mapping with 'error' target - so the
underlaying device is relaxed - although we do not support this
in lvm2 - you would need to use 'dmsetup' for this (and open lvm2 RFE
if there would be some serious justifaction).

Regards

Zdenek

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Gionatan Danti]

Il 2020-06-23 22:28 Zdenek Kabelac ha scritto:
> Note - you cannot 'remove' mappings 'in-use'  (aka open count of a 
> device
> is higher then 0  - see 'dmsetup info -c' output for this).
> However you can replace such mapping with 'error' target - so the
> underlaying device is relaxed - although we do not support this
> in lvm2 - you would need to use 'dmsetup' for this (and open lvm2 RFE
> if there would be some serious justifaction).

Not related to oVirt, but I would find a means to set an error target 
with LVM quite useful - think about immediately stopping IO to/from 
device in case some serious error happended. Using lvm rather than 
dmsetup would be easier and less error prone.

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it [1]
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Zdenek Kabelac]

Dne 23. 06. 20 v 22:37 Gionatan Danti napsal(a):
> Il 2020-06-23 22:28 Zdenek Kabelac ha scritto:
>> Note - you cannot 'remove' mappings 'in-use'� (aka open count of a device
>> is higher then 0� - see 'dmsetup info -c' output for this).
>> However you can replace such mapping with 'error' target - so the
>> underlaying device is relaxed - although we do not support this
>> in lvm2 - you would need to use 'dmsetup' for this (and open lvm2 RFE
>> if there would be some serious justifaction).
> 
> Not related to oVirt, but I would find a means to set an error target with LVM 
> quite useful - think about immediately stopping IO to/from device in case some 
> serious error happended. Using lvm rather than dmsetup would be easier and 
> less error prone.

Hi

ATM skilled admin can always easily enforce:

'dmsetup remove --force   vg-lv'

for i.e. linear devices to achieve this goal - however resolving this at lvm2 
is actually way more complex task when you start to consider the situation
should be at least 'somehow' recoverable - it's quite complicated and
not really highly demanded functionality.
It's more simple if you have constrained world of known types of devices
and known use-case you are targeting to solve.

Regards

Zdenek

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Gionatan Danti]

Il 2020-06-23 23:02 Zdenek Kabelac ha scritto:
> Hi
> 
> ATM skilled admin can always easily enforce:
> 
> 'dmsetup remove --force   vg-lv'

Hi Zdenek,
sure, but I find messing with dmsetup more error prone than using an LVM 
command.

> for i.e. linear devices to achieve this goal - however resolving this
> at lvm2 is actually way more complex task when you start to consider
> the situation
> should be at least 'somehow' recoverable - it's quite complicated and
> not really highly demanded functionality.
> It's more simple if you have constrained world of known types of 
> devices
> and known use-case you are targeting to solve.

Would be an initial minimal support userful? For example, something as 
"lvchange --error vg/lvm" which will only work on simple setup, while on 
more complex ones would simply print a warning and return with exit code 
1.

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it [1]
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Zdenek Kabelac]

Dne 23. 06. 20 v 23:34 Gionatan Danti napsal(a):
> Il 2020-06-23 23:02 Zdenek Kabelac ha scritto:
>> Hi
>>
>> ATM skilled admin can always easily enforce:
>>
>> 'dmsetup remove --force�� vg-lv'
> 
> Hi Zdenek,
> sure, but I find messing with dmsetup more error prone than using an LVM command.
> 
>> for i.e. linear devices to achieve this goal - however resolving this
>> at lvm2 is actually way more complex task when you start to consider
>> the situation
>> should be at least 'somehow' recoverable - it's quite complicated and
>> not really highly demanded functionality.
>> It's more simple if you have constrained world of known types of devices
>> and known use-case you are targeting to solve.
> 
> Would be an initial minimal support userful? For example, something as 
> "lvchange --error vg/lvm" which will only work on simple setup, while on more 
> complex ones would simply print a warning and return with exit code 1.
> 

There are 'couple' issue -

a) it would be nice to have this 'change' permanent ->  metadata update ->
but likely this user runs when some devices got missing - so lots of related 
trouble with 'MISSING' devices and metadata update.

b) when this would be only '--replacewitherror  y|n'  lvchange modifier - then
with next i.e. lvchange --refresh (or any other 'transient' table update) you 
get very same problematic reference of original PV.

c) it's unclear if we want to actually provide this as 'lvremove' feature - as
IMHO when 'error' replacement is required  - the most cleanest proceed is to 
never use this device again -  and ATM you can't  remove  'device' in use -
but with this kind of support we could probably migrate such device into
just 'error' placeholders (removable once devices are unused)  and space would
be relased in VG  (this would be probably my favorite path to add support
for this kind of forcible removal of used LV).

Regards

Zdenek

Re: [linux-lvm] Removing VG mappings using dmsetup tool

[Vojtech Juranek]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

> BUT I'd probably focus rather on fixing your oVirt workflow - you should
> not be 'losing' devices while you have activate mappings on them - that's
> IMHO major desing flaw - you can very easily cause very SERIOUS damages
> to your on disk data consistency - so disk would require serious fsck before
> reuse...

Thanks for your answer. Just to clarify - we are not going to loose devices 
intentionally, we are considering this option only for recovery from situation 
when storage is suddenly lost (e.g. broken network, storage sever crashed 
etc.). In these situations disk data consistency would be an issue anyway.

Thanks
Vojta

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]