[linux-lvm] confused with lvm2 filter rules

[linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello,

I met below filter action when executed 'vgextend'.
why the filter take no effect on executing pvcreate or vgcreate?

```
# ls -l /dev/disk/by-id/scsi-*
lrwxrwxrwx 1 root root 9 Jun  3 20:44 /dev/disk/by-id/scsi-360073a707 -> 
../../sda
lrwxrwxrwx 1 root root 9 Jun  3 20:44 /dev/disk/by-id/scsi-36007c0036 -> 
../../sdb
lrwxrwxrwx 1 root root 9 Jun  3 20:27 /dev/disk/by-id/scsi-3600b99ec4 -> 
../../sdc
# pvcreate /dev/disk/by-id/scsi-36073a707 /dev/disk/by-id/scsi-36007c0036
# pvs
   PV         VG Fmt  Attr PSize PFree
   /dev/sda      lvm2 ---  2.00g 2.00g
   /dev/sdb      lvm2 ---  2.00g 2.00g
# vgcreate vgtst /dev/disk/by-id/scsi-36073a707 
/dev/disk/by-id/scsi-36007c0036hy the filter take no effect on executing 
pvcreate or vgcreate?
# vgs
   VG    #PV #LV #SN Attr   VSize VFree
   vgtst   2   0   0 wz--n- 3.98g 3.98g
# vgextend vgtst /dev/disk/by-id/scsi-3600b99ec4
   Device /dev/disk/by-id/scsi-3600b99ec4 excluded by a filter.
# vgs
   VG    #PV #LV #SN Attr   VSize VFree
   vgtst   2   0   0 wz--n- 3.98g 3.98g

my enviroment:
# rpm -qa | grep lvm2
lvm2-clvm-2.02.180-8.16.x86_64
lvm2-cmirrord-2.02.180-8.16.x86_64
lvm2-2.02.180-8.16.x86_64

the filter rules:  (you can see all the disk in /dev/disk/by-id/ are 
rejected)
# grep filter /etc/lvm/lvm.conf | grep -v "#"
     filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|", 
"r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
```


Thank you.
zhm

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello list,

Can anybody help me for this question?
I tried on suse & centos with lvm2-2.02.180, both system have this issue.

The filter in /etc/lvm/lvm.conf only works with vgextend command.
It take no effect on pvcreate, vgcreate & vgreduce.

Thank you.

On 6/3/19 9:03 PM, Heming Zhao wrote:
> Hello,
> 
> I met below filter action when executed 'vgextend'.
> why the filter take no effect on executing pvcreate or vgcreate?
> 
> ```
> # ls -l /dev/disk/by-id/scsi-*
> lrwxrwxrwx 1 root root 9 Jun  3 20:44 /dev/disk/by-id/scsi-360073a707 ->
> ../../sda
> lrwxrwxrwx 1 root root 9 Jun  3 20:44 /dev/disk/by-id/scsi-36007c0036 ->
> ../../sdb
> lrwxrwxrwx 1 root root 9 Jun  3 20:27 /dev/disk/by-id/scsi-3600b99ec4 ->
> ../../sdc
> # pvcreate /dev/disk/by-id/scsi-36073a707 /dev/disk/by-id/scsi-36007c0036
> # pvs
>     PV         VG Fmt  Attr PSize PFree
>     /dev/sda      lvm2 ---  2.00g 2.00g
>     /dev/sdb      lvm2 ---  2.00g 2.00g
> # vgcreate vgtst /dev/disk/by-id/scsi-36073a707
> /dev/disk/by-id/scsi-36007c0036hy the filter take no effect on executing
> pvcreate or vgcreate?
> # vgs
>     VG    #PV #LV #SN Attr   VSize VFree
>     vgtst   2   0   0 wz--n- 3.98g 3.98g
> # vgextend vgtst /dev/disk/by-id/scsi-3600b99ec4
>     Device /dev/disk/by-id/scsi-3600b99ec4 excluded by a filter.
> # vgs
>     VG    #PV #LV #SN Attr   VSize VFree
>     vgtst   2   0   0 wz--n- 3.98g 3.98g
> 
> my enviroment:
> # rpm -qa | grep lvm2
> lvm2-clvm-2.02.180-8.16.x86_64
> lvm2-cmirrord-2.02.180-8.16.x86_64
> lvm2-2.02.180-8.16.x86_64
> 
> the filter rules:  (you can see all the disk in /dev/disk/by-id/ are
> rejected)
> # grep filter /etc/lvm/lvm.conf | grep -v "#"
>       filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
> ```
> 
> 
> Thank you.
> zhm
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Zdenek Kabelac]

Dne 03. 06. 19 v 15:03 Heming Zhao napsal(a):
> Hello,
> 
> I met below filter action when executed 'vgextend'.
> why the filter take no effect on executing pvcreate or vgcreate?

> # rpm -qa | grep lvm2
> lvm2-clvm-2.02.180-8.16.x86_64
> lvm2-cmirrord-2.02.180-8.16.x86_64
> lvm2-2.02.180-8.16.x86_64
> 
> the filter rules:  (you can see all the disk in /dev/disk/by-id/ are
> rejected)
> # grep filter /etc/lvm/lvm.conf | grep -v "#"
>       filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]


Hi

Filter with 'a|.*|' at the end will almost always never work.
As devices do have several names so if you reject it by one name,
you will likely accept it with another name.


So I'd highly recommend only these 2 variants that are 'easy to follow'.


1. White-list devices you want to see and add r|.*|  as the last rule.
^^^^^^^^^^^^^^^ most recommended.

2. Pure list of reject rules (do not add any 'a' rule).

Regards

Zdenek

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello Zdenek,

Thank you very much for your explanation.

Regards,
zhm


On 6/5/19 6:20 PM, Zdenek Kabelac wrote:
> Dne 03. 06. 19 v 15:03 Heming Zhao napsal(a):
>> Hello,
>>
>> I met below filter action when executed 'vgextend'.
>> why the filter take no effect on executing pvcreate or vgcreate?
> 
>> # rpm -qa | grep lvm2
>> lvm2-clvm-2.02.180-8.16.x86_64
>> lvm2-cmirrord-2.02.180-8.16.x86_64
>> lvm2-2.02.180-8.16.x86_64
>>
>> the filter rules:� (you can see all the disk in /dev/disk/by-id/ are
>> rejected)
>> # grep filter /etc/lvm/lvm.conf | grep -v "#"
>> ����� filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
>> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
> 
> 
> Hi
> 
> Filter with 'a|.*|' at the end will almost always never work.
> As devices do have several names so if you reject it by one name,
> you will likely accept it with another name.
> 
> 
> So I'd highly recommend only these 2 variants that are 'easy to follow'.
> 
> 
> 1. White-list devices you want to see and add r|.*|� as the last rule.
> ^^^^^^^^^^^^^^^ most recommended.
> 
> 2. Pure list of reject rules (do not add any 'a' rule).
> 
> Regards
> 
> Zdenek
> 
> 
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello,

BTW,
Only vgextend doesn't work, which must be a bug. It looks the filter 
handling codes have bug.

Regards,
zhm

On 6/5/19 6:20 PM, Zdenek Kabelac wrote:
> Dne 03. 06. 19 v 15:03 Heming Zhao napsal(a):
>> Hello,
>>
>> I met below filter action when executed 'vgextend'.
>> why the filter take no effect on executing pvcreate or vgcreate?
> 
>> # rpm -qa | grep lvm2
>> lvm2-clvm-2.02.180-8.16.x86_64
>> lvm2-cmirrord-2.02.180-8.16.x86_64
>> lvm2-2.02.180-8.16.x86_64
>>
>> the filter rules:� (you can see all the disk in /dev/disk/by-id/ are
>> rejected)
>> # grep filter /etc/lvm/lvm.conf | grep -v "#"
>> ����� filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
>> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
> 
> 
> Hi
> 
> Filter with 'a|.*|' at the end will almost always never work.
> As devices do have several names so if you reject it by one name,
> you will likely accept it with another name.
> 
> 
> So I'd highly recommend only these 2 variants that are 'easy to follow'.
> 
> 
> 1. White-list devices you want to see and add r|.*|� as the last rule.
> ^^^^^^^^^^^^^^^ most recommended.
> 
> 2. Pure list of reject rules (do not add any 'a' rule).
> 
> Regards
> 
> Zdenek
> 
> 
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Zdenek Kabelac]

Dne 06. 06. 19 v 10:16 Heming Zhao napsal(a):
> Hello,
> 
> BTW,
> Only vgextend doesn't work, which must be a bug. It looks the filter
> handling codes have bug.
> 

Hi

Please provide full 'vgextend -vvvv' trace  with some explanation of what you 
believe is a bug in handling code.


Regards


Zdenek

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello,

the filter is:
filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|", 
"r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]

if filter doesn't contain "a/.*/":
- pvcreate, vgcreate & vgextend use regex filter to reject the disk. 
(correct logic)

if filter contains "a/.*/":
- regex fileter pass the disk under pvcreate/vgcreate, create successfully.
- regex filter reject the disk under vgextend. (wrong. should create 
successfuly)
- vgextend should do the same action as pvcreate/vgcreate.

log as below (with filter contain: "a/.*/"):
------------------------
# vgextend -vvvv vgtst /dev/disk/by-id/scsi-360014051ec87260bf98462eb448
0b3b3
#lvmcmdline.c:2814          Parsing: vgextend -vvvv vgtst 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3
#lvmcmdline.c:1868          Recognised command vgextend_general (id 128 
/ enum 113).
#config/config.c:1480        devices/global_filter not found in config: 
defaulting to global_filter = [ "a|.*/|" ]
#libdm-config.c:1002        global/lvmetad_update_wait_time not found in 
config: defaulting to 10
#daemon-client.c:33            /run/lvm/lvmetad.socket: Opening daemon 
socket to lvmetad for protocol lvmetad version 1.
#daemon-client.c:52            Sending daemon lvmetad: hello
#cache/lvmetad.c:141           Successfully connected to lvmetad on fd 3.
#libdm-config.c:1074        global/use_lvmpolld not found in config: 
defaulting to 1
#filters/filter-sysfs.c:327           Sysfs filter initialised.
#filters/filter-internal.c:77            Internal filter initialised.
#filters/filter-type.c:56            LVM type filter initialised.
#filters/filter-usable.c:183           Usable device filter initialised.
#filters/filter-mpath.c:291           mpath filter initialised.
#filters/filter-partitioned.c:69            Partitioned filter initialised.
#filters/filter-signature.c:84            signature filter initialised.
#filters/filter-md.c:169           MD filter initialised.
#libdm-config.c:1074        devices/fw_raid_component_detection not 
found in config: defaulting to 0
#filters/filter-composite.c:109           Composite filter initialised.
#config/config.c:1483        Setting devices/filter to filter = [ 
"r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|", "r|/dev/fd.*|", 
"r|/dev/cdrom|", "a/.*/" ]
#filters/filter-regex.c:216           Regex filter initialised.
#filters/filter-usable.c:183           Usable device filter initialised.
#filters/filter-composite.c:109           Composite filter initialised.
#libdm-config.c:975         devices/cache not found in config: 
defaulting to /etc/lvm/cache/.cache
#filters/filter-persistent.c:415           Persistent filter initialised.
#filters/filter-composite.c:109           Composite filter initialised.
#libdm-config.c:1074        metadata/record_lvs_history not found in 
config: defaulting to 0
#lvmcmdline.c:2882          DEGRADED MODE. Incomplete RAID LVs will be 
processed.
#lvmcmdline.c:2888          Processing command: vgextend -vvvv vgtst 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3
#lvmcmdline.c:2889          Command pid: 17557
#lvmcmdline.c:2890          System ID:
#lvmcmdline.c:2893          O_DIRECT will be used
#locking/locking.c:129         File-based locking selected.
#libdm-config.c:1074        global/use_lvmlockd not found in config: 
defaulting to 0
#cache/lvmetad.c:254           Sending lvmetad get_global_info
#libdm-config.c:1074        metadata/pvmetadataignore not found in 
config: defaulting to 0
#libdm-config.c:1002        metadata/pvmetadatasize not found in config: 
defaulting to 255
#libdm-config.c:1002        metadata/pvmetadatacopies not found in 
config: defaulting to 1
#libdm-config.c:975         report/output_format not found in config: 
defaulting to basic
#libdm-config.c:1074        log/report_command_log not found in config: 
defaulting to 0
#cache/lvmcache.c:2535          Dropping VG info
#cache/lvmcache.c:751           lvmcache has no info for vgname 
"#orphans_lvm2" with VGID #orphans_lvm2.
#cache/lvmcache.c:751           lvmcache has no info for vgname 
"#orphans_lvm2".
#cache/lvmcache.c:2082          lvmcache: Initialised VG #orphans_lvm2.
#locking/locking.c:331           Dropping cache for #orphans.
#misc/lvm-flock.c:202         Locking /run/lvm/lock/P_orphans WB
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/P_orphans:aux WB
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/P_orphans WB
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/P_orphans:aux
#cache/lvmcache.c:751           lvmcache has no info for vgname "#orphans".
#device/dev-cache.c:723           Found dev 8:32 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3 - new.
#device/dev-io.c:609           Opened 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3 RO O_DIRECT
#device/dev-io.c:359 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3: size is 1024000 
sectors
#device/dev-io.c:658           Closed 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3
#device/dev-io.c:336 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3: using cached 
size 1024000 sectors
#filters/filter-regex.c:172 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3: Skipping (regex)
#filters/filter-persistent.c:346           filter caching bad 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3
#toollib.c:4451          Processing each PV
#cache/lvmetad.c:1445          Asking lvmetad for complete list of known 
VG ids/names
#toollib.c:3941          Getting list of all devices
#device/dev-cache.c:1212          Creating list of system devices.
#device/dev-cache.c:723           Found dev 253:0 /dev/vda - new.
#device/dev-cache.c:723           Found dev 253:1 /dev/vda1 - new.
#device/dev-cache.c:763           Found dev 253:1 
/dev/disk/by-partuuid/000c10c1-01 - new alias.
#device/dev-cache.c:763           Found dev 253:1 
/dev/disk/by-uuid/060ece02-7ac0-49bd-88cf-2853fe388671 - new alias.
#device/dev-cache.c:723           Found dev 253:2 /dev/vda2 - new.
#device/dev-cache.c:763           Found dev 253:2 
/dev/disk/by-partuuid/000c10c1-02 - new alias.
#device/dev-cache.c:763           Found dev 253:2 
/dev/disk/by-uuid/f545996e-247f-4778-ab81-7c7b6140d23a - new alias.
#device/dev-cache.c:763           Found dev 253:2 /dev/root - new alias.
#device/dev-cache.c:723           Found dev 8:0 /dev/sda - new.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-id/lvm-pv-uuid-2n9LLa-sWow-ZYA3-jiRX-5yfc-ghcq-119083 - new 
alias.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-id/scsi-1LIO-ORG_FILEIO:78c8c971-44e0-4791-aa89-97ad40cb48d1 
- new alias.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-id/scsi-3600140578c8c97144e04791aa8997ad4 - new alias.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-id/scsi-SLIO-ORG_FILEIO_78c8c971-44e0-4791-aa89-97ad40cb48d1 
- new alias.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-id/wwn-0x600140578c8c97144e04791aa8997ad4 - new alias.
#device/dev-cache.c:763           Found dev 8:0 
/dev/disk/by-path/ip-10.67.17.201:3260-iscsi-iqn.2019-06.com.example:iscsi-500m-disk01-lun-0 
- new alias.
#device/dev-cache.c:723           Found dev 8:16 /dev/sdb - new.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-id/lvm-pv-uuid-FQqmv2-8mTk-5r4r-cwvn-8HJ1-GDBH-svklEf - new 
alias.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-id/scsi-1LIO-ORG_FILEIO:a7ec3a1e-70d4-4a47-b16d-2e404f27b30c 
- new alias.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-id/scsi-36001405a7ec3a1e70d44a47b16d2e404 - new alias.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-id/scsi-SLIO-ORG_FILEIO_a7ec3a1e-70d4-4a47-b16d-2e404f27b30c 
- new alias.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-id/wwn-0x6001405a7ec3a1e70d44a47b16d2e404 - new alias.
#device/dev-cache.c:763           Found dev 8:16 
/dev/disk/by-path/ip-10.67.17.201:3260-iscsi-iqn.2019-06.com.example:iscsi-500m-disk02-lun-0 
- new alias.
#device/dev-cache.c:763           Found dev 8:32 /dev/sdc - new alias.
#device/dev-cache.c:763           Found dev 8:32 
/dev/disk/by-id/scsi-1LIO-ORG_FILEIO:1ec87260-bf98-462e-b448-0b3b32f1e1c4 
- new alias.
#device/dev-cache.c:714           Found dev 8:32 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3 - exists.
#device/dev-cache.c:763           Found dev 8:32 
/dev/disk/by-id/scsi-SLIO-ORG_FILEIO_1ec87260-bf98-462e-b448-0b3b32f1e1c4 
- new alias.
#device/dev-cache.c:763           Found dev 8:32 
/dev/disk/by-id/wwn-0x60014051ec87260bf98462eb4480b3b3 - new alias.
#device/dev-cache.c:763           Found dev 8:32 
/dev/disk/by-path/ip-10.67.17.201:3260-iscsi-iqn.2019-06.com.example:iscsi-500m-disk03-lun-0 
- new alias.
#cache/lvmetad.c:1420          Asking lvmetad for complete list of known PVs
#filters/filter-persistent.c:346           filter caching good /dev/sdb
#cache/lvmcache.c:751           lvmcache has no info for vgname "vgtst" 
with VGID PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0.
#cache/lvmcache.c:751           lvmcache has no info for vgname "vgtst".
#cache/lvmcache.c:2080          lvmcache /dev/sdb: now in VG vgtst with 
0 mda(s).
#cache/lvmcache.c:1906          lvmcache /dev/sdb: VG vgtst: set VGID to 
PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0.
#filters/filter-persistent.c:346           filter caching good /dev/sda
#cache/lvmcache.c:2080          lvmcache /dev/sda: now in VG vgtst 
(PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0) with 0 mda(s).
#device/dev-io.c:609           Opened /dev/sda RO O_DIRECT
#device/dev-io.c:359         /dev/sda: size is 1024000 sectors
#device/dev-io.c:658           Closed /dev/sda
#device/dev-io.c:336         /dev/sda: using cached size 1024000 sectors
#filters/filter-persistent.c:312           /dev/sda: filter cache using 
(cached good)
#device/dev-io.c:609           Opened /dev/vda RO O_DIRECT
#device/dev-io.c:359         /dev/vda: size is 83886080 sectors
#device/dev-io.c:658           Closed /dev/vda
#filters/filter-partitioned.c:37            /dev/vda: Skipping: 
Partition table signature found
#device/dev-io.c:609           Opened /dev/vda1 RO O_DIRECT
#device/dev-io.c:359         /dev/vda1: size is 3002368 sectors
#device/dev-io.c:658           Closed /dev/vda1
#device/dev-io.c:336         /dev/vda1: using cached size 3002368 sectors
#filters/filter-persistent.c:346           filter caching good /dev/vda1
#device/dev-io.c:609           Opened /dev/root RO O_DIRECT
#device/dev-io.c:359         /dev/root: size is 80881664 sectors
#device/dev-io.c:658           Closed /dev/root
#device/dev-io.c:336         /dev/root: using cached size 80881664 sectors
#filters/filter-persistent.c:346           filter caching good /dev/root
#device/dev-io.c:609           Opened /dev/sdb RO O_DIRECT
#device/dev-io.c:359         /dev/sdb: size is 1024000 sectors
#device/dev-io.c:658           Closed /dev/sdb
#device/dev-io.c:336         /dev/sdb: using cached size 1024000 sectors
#filters/filter-persistent.c:312           /dev/sdb: filter cache using 
(cached good)
#device/dev-io.c:336         /dev/sdc: using cached size 1024000 sectors
#device/dev-io.c:336         /dev/sdc: using cached size 1024000 sectors
#filters/filter-persistent.c:346           filter caching good /dev/sdc
#toollib.c:4377          Processing PVs in VG vgtst
#locking/locking.c:331           Dropping cache for vgtst.
#misc/lvm-flock.c:202         Locking /run/lvm/lock/V_vgtst RB
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/V_vgtst:aux WB
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/V_vgtst:aux
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/V_vgtst RB
#metadata/metadata.c:3778        Reading VG vgtst 
PFEUFq-Y9UO-1SFY-oNiI-gPEk-90Th-VmDDx0
#cache/lvmetad.c:983           Asking lvmetad for VG 
PFEUFq-Y9UO-1SFY-oNiI-gPEk-90Th-VmDDx0 vgtst
#filters/filter-persistent.c:312           /dev/sdb: filter cache using 
(cached good)
#cache/lvmcache.c:2080          lvmcache /dev/sdb: now in VG 
#orphans_lvm2 (#orphans_lvm2) with 1 mda(s).
#filters/filter-persistent.c:312           /dev/sda: filter cache using 
(cached good)
#cache/lvmcache.c:2080          lvmcache /dev/sda: now in VG 
#orphans_lvm2 (#orphans_lvm2) with 1 mda(s).
#metadata/vg.c:68            Allocated VG vgtst at 0x55c563bc5980.
#cache/lvmcache.c:751           lvmcache has no info for vgname "vgtst" 
with VGID PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0.
#cache/lvmcache.c:751           lvmcache has no info for vgname "vgtst".
#cache/lvmcache.c:2080          lvmcache /dev/sdb: now in VG vgtst with 
1 mda(s).
#cache/lvmcache.c:1906          lvmcache /dev/sdb: VG vgtst: set VGID to 
PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0.
#cache/lvmcache.c:2080          lvmcache /dev/sda: now in VG vgtst 
(PFEUFqY9UO1SFYoNiIgPEk90ThVmDDx0) with 1 mda(s).
#cache/lvmetad.c:2462          Sending lvmetad vg_clear_outdated_pvs
#device/dev-io.c:336         /dev/sdb: using cached size 1024000 sectors
#device/dev-io.c:336         /dev/sda: using cached size 1024000 sectors
#metadata/pv_manip.c:417           /dev/sdb 0:      0    123: NULL(0:0)
#metadata/pv_manip.c:417           /dev/sda 0:      0    123: NULL(0:0)
#metadata/vg.c:68            Allocated VG vgtst at 0x55c563bb2a90.
#toollib.c:4269        Processing PV /dev/sdb in VG vgtst.
#toollib.c:4269        Processing PV /dev/sda in VG vgtst.
#mm/memlock.c:594           Unlock: Memlock counters: prioritized:0 
locked:0 critical:0 daemon:0 suspended:0
#activate/fs.c:491           Syncing device names
#locking/locking.c:331           Dropping cache for vgtst.
#misc/lvm-flock.c:70          Unlocking /run/lvm/lock/V_vgtst
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/V_vgtst
#metadata/vg.c:83            Freeing VG vgtst at 0x55c563bb2a90.
#metadata/vg.c:83            Freeing VG vgtst at 0x55c563bc5980.
#toollib.c:4377          Processing PVs in VG #orphans_lvm2
#metadata/metadata.c:5495        Locking #orphans_lvm2 already done
#metadata/metadata.c:3764        Reading VG #orphans_lvm2
#locking/locking.c:331           Dropping cache for #orphans.
#misc/lvm-flock.c:70          Unlocking /run/lvm/lock/P_orphans
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/P_orphans
#cache/lvmcache.c:751           lvmcache has no info for vgname "#orphans".
#locking/locking.c:331           Dropping cache for #orphans.
#misc/lvm-flock.c:202         Locking /run/lvm/lock/P_orphans WB
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/P_orphans:aux WB
#misc/lvm-flock.c:100           _do_flock /run/lvm/lock/P_orphans WB
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/P_orphans:aux
#cache/lvmcache.c:751           lvmcache has no info for vgname "#orphans".
#metadata/metadata.c:3764        Reading VG #orphans_lvm2
#toollib.c:4056          Processing devices that are not PVs
#toollib.c:4072        Processing device /dev/vda1.
#toollib.c:4072        Processing device /dev/root.
#toollib.c:4072        Processing device /dev/sdc.
#toollib.c:5539    Device 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3 excluded by a filter.
#toollib.c:5545          <backtrace>
#locking/locking.c:331           Dropping cache for #orphans.
#misc/lvm-flock.c:70          Unlocking /run/lvm/lock/P_orphans
#misc/lvm-flock.c:47            _undo_flock /run/lvm/lock/P_orphans
#cache/lvmcache.c:751           lvmcache has no info for vgname "#orphans".
#vgextend.c:177           <backtrace>
#libdm-config.c:1074        global/notify_dbus not found in config: 
defaulting to 1
#cache/lvmcache.c:2535          Dropping VG info
#cache/lvmcache.c:751           lvmcache has no info for vgname 
"#orphans_lvm2" with VGID #orphans_lvm2.
#cache/lvmcache.c:751           lvmcache has no info for vgname 
"#orphans_lvm2".
#cache/lvmcache.c:2082          lvmcache: Initialised VG #orphans_lvm2.
#lvmcmdline.c:3028          Completed: vgextend -vvvv vgtst 
/dev/disk/by-id/scsi-360014051ec87260bf98462eb4480b3b3

On 6/6/19 4:43 PM, Zdenek Kabelac wrote:
> Dne 06. 06. 19 v 10:16 Heming Zhao napsal(a):
>> Hello,
>>
>> BTW,
>> Only vgextend doesn't work, which must be a bug. It looks the filter
>> handling codes have bug.
>>
> 
> Hi
> 
> Please provide full 'vgextend -vvvv' trace� with some explanation of 
> what you believe is a bug in handling code.
> 
> 
> Regards
> 
> 
> Zdenek
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Zdenek Kabelac]

Dne 06. 06. 19 v 15:30 Heming Zhao napsal(a):
> Hello,
> 
> the filter is:
> filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
> 
> if filter doesn't contain "a/.*/":
> - pvcreate, vgcreate & vgextend use regex filter to reject the disk.
> (correct logic)
>  > if filter contains "a/.*/":
> - regex fileter pass the disk under pvcreate/vgcreate, create successfully.
> - regex filter reject the disk under vgextend. (wrong. should create
> successfuly)
> - vgextend should do the same action as pvcreate/vgcreate.

Hi

As has been said -   when you put  a|.*| as the last rule - it's doing 
something different then you may think. So please do NOT test with such filter 
(I've been even planning to add 'WARNING:' message when lvm2 would spot such 
filter....)

> log as below (with filter contain: "a/.*/"):

Not really interesting trace - the filter works correctly for this case.

The best advice is - DO NOT USE IT set this way.

Code is correct and there is no bug - it's just non-trivial to understand what 
just filter is doing. Behavior cannot be changed without causing regression 
for other users that set filters correctly.

So do you have any buggy trace for correctly set filter ?

Regards

Zdenek

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello Zdenek,

I totally got your point. Let's close this session.
Thank you for you reply.

Regards,
zhm

On 6/6/19 9:51 PM, Zdenek Kabelac wrote:
> Dne 06. 06. 19 v 15:30 Heming Zhao napsal(a):
>> Hello,
>>
>> the filter is:
>> filter = [ "r|/dev/.*/by-path/.*|", "r|/dev/.*/by-id/.*|",
>> "r|/dev/fd.*|", "r|/dev/cdrom|", "a/.*/" ]
>>
>> if filter doesn't contain "a/.*/":
>> - pvcreate, vgcreate & vgextend use regex filter to reject the disk.
>> (correct logic)
>> �> if filter contains "a/.*/":
>> - regex fileter pass the disk under pvcreate/vgcreate, create 
>> successfully.
>> - regex filter reject the disk under vgextend. (wrong. should create
>> successfuly)
>> - vgextend should do the same action as pvcreate/vgcreate.
> 
> Hi
> 
> As has been said -�� when you put� a|.*| as the last rule - it's doing 
> something different then you may think. So please do NOT test with such 
> filter (I've been even planning to add 'WARNING:' message when lvm2 
> would spot such filter....)
> 
>> log as below (with filter contain: "a/.*/"):
> 
> Not really interesting trace - the filter works correctly for this case.
> 
> The best advice is - DO NOT USE IT set this way.
> 
> Code is correct and there is no bug - it's just non-trivial to 
> understand what just filter is doing. Behavior cannot be changed without 
> causing regression for other users that set filters correctly.
> 
> So do you have any buggy trace for correctly set filter ?
> 
> Regards
> 
> Zdenek
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Martin Wilck]

Hello Zdenek,

On Tue, 2019-06-25 at 05:30 +0000, Heming Zhao wrote:
> Hello Zdenek,
> 
> I raise this topic again. Lvm should have bug in filter code.
> Let me show the example.
> 
> filter = [ "a|^/dev/sd.*|", "r|.*|" ]
> As document description, above filter rules:
>   deny all the devices except "/dev/sd.*"
> 
> issue:
>   pvcreate executes successfully with "/dev/disk/by-id/xxx", but 
> vgextend doesn't.
> 
> expect result:
>   pvcreate should deny the device "/dev/disk/by-id/xxx".
> 

I disagree with Heming in this point. IMO both pvcreate and vgextend
should accept the device because of the first "a" rule. In any case,
it's obviously wrong that the two tools behave differently.

Note also that this difference occurs only if lvmetad is used. Without
lvmetad, both commands accept the device. The reason is that in this
case, lvmcache_label_scan(), which also builds the alias tree, is
called before applying the filter. With lvmetad OTOH,
lvmcache_label_scan() is basically a noop.

IMO this should be fixed by adding a call to
lvmcache_seed_infos_from_lvmetad() before applying the device filter to
vgextend. vgcreate() calls it early on, right after
lvmcache_label_scan(); the same might work for vgextend as well.
Alternatively, it might be possible to add a call to 
lvmcache_seed_infos_from_lvmetad() to pvcreate_each_device(); in that
case it might even be possible to remove the early calls in vgcreate().
I don't understand the initialization sequence of LVM2 commands well
enough to create a patch myself. Besides vgextend, other LVM2 commands
that need to apply the filter may be affected, too, as 
lvmcache_seed_infos_from_lvmetad() seems to be used only in a few hand-
selected code paths.

I suspect that this problem came to be in David's "label_scan" patch
series in April 2018. But we haven't verified that yet. I've put David
on CC.

> 
> analysis:
> 
> vgextend log excerpt: the aliases DB is built up _after_ applying
> the I've put
> filter, which falsely rejects the device.
> ```log
>   lvmcmdline.c:2888    Processing command: vgextend -vvvv -dddddd -t 
> testvg /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>   device/dev-cache.c:723     Found dev 8:35 
> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3 - new.
>   filters/filter-regex.c:172 
> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3:
> Skipping 
> (regex)
>   filters/filter-persistent.c:346     filter caching bad 
> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>   ...
>   device/dev-cache.c:1212    Creating list of system devices.
>   device/dev-cache.c:763     Found dev 8:35 /dev/sdc3 - new alias
>   device/dev-cache.c:763     Found dev 8:35 
> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3 
> - new alias.
> ```
> 
> vgcreate: the aliases DB is built up before applying the filter,
> which 
> works correctly now.
> ```log
>   lvmcmdline.c:2888    Processing command: vgcreate -vvvv -dddddd -t 
> tvg1 /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>   device/dev-cache.c:1212    Creating list of system devices.
>   device/dev-cache.c:723     Found dev 8:35 /dev/sdc3 - new.
>   device/dev-cache.c:763     Found dev 8:35 
> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3 
> - new alias.
>   filters/filter-persistent.c:312     /dev/sdc3: filter cache using 
> (cached good)
> ```
> 
> pvcreate will convert "/dev/disk/by-id/xxx" into another name 
> "/dev/sdX", which can pass the filter rule.

A bit more precisely: when running pvcreate (or vgcreate), by the time
the filter is called, "/dev/sdX" has been added to the list of aliases
and thus the device is accepted, whereas in a vgextend run, the list of
aliases has not been built up yet, and contains only a single member
"/dev/disk/by-id/...", which is rejected.

Regards,
Martin

-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

Re: [linux-lvm] confused with lvm2 filter rules

[Zdenek Kabelac]

Dne 25. 06. 19 v 9:56 Martin Wilck napsal(a):
> Hello Zdenek,
> 
> On Tue, 2019-06-25 at 05:30 +0000, Heming Zhao wrote:
>> Hello Zdenek,
>>
>> I raise this topic again. Lvm should have bug in filter code.
>> Let me show the example.
>>
>> filter = [ "a|^/dev/sd.*|", "r|.*|" ]
>> As document description, above filter rules:
>>    deny all the devices except "/dev/sd.*"
>>
>> issue:
>>    pvcreate executes successfully with "/dev/disk/by-id/xxx", but
>> vgextend doesn't.
>>
>> expect result:
>>    pvcreate should deny the device "/dev/disk/by-id/xxx".
>>
> 
> I disagree with Heming in this point. IMO both pvcreate and vgextend
> should accept the device because of the first "a" rule. In any case,
> it's obviously wrong that the two tools behave differently.
> 
> Note also that this difference occurs only if lvmetad is used. Without
> lvmetad, both commands accept the device. The reason is that in this
> case, lvmcache_label_scan(), which also builds the alias tree, is
> called before applying the filter. With lvmetad OTOH,
> lvmcache_label_scan() is basically a noop.
> 
> IMO this should be fixed by adding a call to
> lvmcache_seed_infos_from_lvmetad() before applying the device filter to
> vgextend. vgcreate() calls it early on, right after
> lvmcache_label_scan(); the same might work for vgextend as well.
> Alternatively, it might be possible to add a call to
> lvmcache_seed_infos_from_lvmetad() to pvcreate_each_device(); in that
> case it might even be possible to remove the early calls in vgcreate().
> I don't understand the initialization sequence of LVM2 commands well
> enough to create a patch myself. Besides vgextend, other LVM2 commands
> that need to apply the filter may be affected, too, as
> lvmcache_seed_infos_from_lvmetad() seems to be used only in a few hand-
> selected code paths.
> 
> I suspect that this problem came to be in David's "label_scan" patch
> series in April 2018. But we haven't verified that yet. I've put David
> on CC.
> 
>>
>> analysis:
>>
>> vgextend log excerpt: the aliases DB is built up _after_ applying
>> the I've put
>> filter, which falsely rejects the device.
>> ```log
>>    lvmcmdline.c:2888    Processing command: vgextend -vvvv -dddddd -t
>> testvg /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>    device/dev-cache.c:723     Found dev 8:35
>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3 - new.
>>    filters/filter-regex.c:172
>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3:
>> Skipping
>> (regex)
>>    filters/filter-persistent.c:346     filter caching bad
>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>    ...
>>    device/dev-cache.c:1212    Creating list of system devices.
>>    device/dev-cache.c:763     Found dev 8:35 /dev/sdc3 - new alias
>>    device/dev-cache.c:763     Found dev 8:35
>> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
>> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3
>> - new alias.
>> ```
>>
>> vgcreate: the aliases DB is built up before applying the filter,
>> which
>> works correctly now.
>> ```log
>>    lvmcmdline.c:2888    Processing command: vgcreate -vvvv -dddddd -t
>> tvg1 /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>    device/dev-cache.c:1212    Creating list of system devices.
>>    device/dev-cache.c:723     Found dev 8:35 /dev/sdc3 - new.
>>    device/dev-cache.c:763     Found dev 8:35
>> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
>> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3
>> - new alias.
>>    filters/filter-persistent.c:312     /dev/sdc3: filter cache using
>> (cached good)
>> ```
>>
>> pvcreate will convert "/dev/disk/by-id/xxx" into another name
>> "/dev/sdX", which can pass the filter rule.
> 
> A bit more precisely: when running pvcreate (or vgcreate), by the time
> the filter is called, "/dev/sdX" has been added to the list of aliases
> and thus the device is accepted, whereas in a vgextend run, the list of
> aliases has not been built up yet, and contains only a single member
> "/dev/disk/by-id/...", which is rejected.
>


Please open bugzilla with your findings:

https://bugzilla.redhat.com/enter_bug.cgi?product=LVM%20and%20device-mapper

Thanks

Zdenek

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

Hello All,

I will fire this bug.

On 6/25/19 4:56 PM, Zdenek Kabelac wrote:
> Dne 25. 06. 19 v 9:56 Martin Wilck napsal(a):
>> Hello Zdenek,
>>
>> On Tue, 2019-06-25 at 05:30 +0000, Heming Zhao wrote:
>>> Hello Zdenek,
>>>
>>> I raise this topic again. Lvm should have bug in filter code.
>>> Let me show the example.
>>>
>>> filter = [ "a|^/dev/sd.*|", "r|.*|" ]
>>> As document description, above filter rules:
>>>    deny all the devices except "/dev/sd.*"
>>>
>>> issue:
>>>    pvcreate executes successfully with "/dev/disk/by-id/xxx", but
>>> vgextend doesn't.
>>>
>>> expect result:
>>>    pvcreate should deny the device "/dev/disk/by-id/xxx".
>>>
>>
>> I disagree with Heming in this point. IMO both pvcreate and vgextend
>> should accept the device because of the first "a" rule. In any case,
>> it's obviously wrong that the two tools behave differently.
>>
>> Note also that this difference occurs only if lvmetad is used. Without
>> lvmetad, both commands accept the device. The reason is that in this
>> case, lvmcache_label_scan(), which also builds the alias tree, is
>> called before applying the filter. With lvmetad OTOH,
>> lvmcache_label_scan() is basically a noop.
>>
>> IMO this should be fixed by adding a call to
>> lvmcache_seed_infos_from_lvmetad() before applying the device filter to
>> vgextend. vgcreate() calls it early on, right after
>> lvmcache_label_scan(); the same might work for vgextend as well.
>> Alternatively, it might be possible to add a call to
>> lvmcache_seed_infos_from_lvmetad() to pvcreate_each_device(); in that
>> case it might even be possible to remove the early calls in vgcreate().
>> I don't understand the initialization sequence of LVM2 commands well
>> enough to create a patch myself. Besides vgextend, other LVM2 commands
>> that need to apply the filter may be affected, too, as
>> lvmcache_seed_infos_from_lvmetad() seems to be used only in a few hand-
>> selected code paths.
>>
>> I suspect that this problem came to be in David's "label_scan" patch
>> series in April 2018. But we haven't verified that yet. I've put David
>> on CC.
>>
>>>
>>> analysis:
>>>
>>> vgextend log excerpt: the aliases DB is built up _after_ applying
>>> the I've put
>>> filter, which falsely rejects the device.
>>> ```log
>>>    lvmcmdline.c:2888    Processing command: vgextend -vvvv -dddddd -t
>>> testvg /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>>    device/dev-cache.c:723     Found dev 8:35
>>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3 - new.
>>>    filters/filter-regex.c:172
>>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3:
>>> Skipping
>>> (regex)
>>>    filters/filter-persistent.c:346     filter caching bad
>>> /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>>    ...
>>>    device/dev-cache.c:1212    Creating list of system devices.
>>>    device/dev-cache.c:763     Found dev 8:35 /dev/sdc3 - new alias
>>>    device/dev-cache.c:763     Found dev 8:35
>>> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
>>> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3
>>> - new alias.
>>> ```
>>>
>>> vgcreate: the aliases DB is built up before applying the filter,
>>> which
>>> works correctly now.
>>> ```log
>>>    lvmcmdline.c:2888    Processing command: vgcreate -vvvv -dddddd -t
>>> tvg1 /dev/disk/by-id/scsi-36001405bbbdf17a69ad46ffb287a3340-part3
>>>    device/dev-cache.c:1212    Creating list of system devices.
>>>    device/dev-cache.c:723     Found dev 8:35 /dev/sdc3 - new.
>>>    device/dev-cache.c:763     Found dev 8:35
>>> /dev/disk/by-id/iscsi-iqn.2018-06.de.suse.mwilck:sles15u-sp1-
>>> 01_i_iface:default-1--iqn.2018-06.de.suse.zeus:01_t_0x1-lun-3-part3
>>> - new alias.
>>>    filters/filter-persistent.c:312     /dev/sdc3: filter cache using
>>> (cached good)
>>> ```
>>>
>>> pvcreate will convert "/dev/disk/by-id/xxx" into another name
>>> "/dev/sdX", which can pass the filter rule.
>>
>> A bit more precisely: when running pvcreate (or vgcreate), by the time
>> the filter is called, "/dev/sdX" has been added to the list of aliases
>> and thus the device is accepted, whereas in a vgextend run, the list of
>> aliases has not been built up yet, and contains only a single member
>> "/dev/disk/by-id/...", which is rejected.
>>
> 
> 
> Please open bugzilla with your findings:
> 
> https://bugzilla.redhat.com/enter_bug.cgi?product=LVM%20and%20device-mapper
> 
> Thanks
> 
> Zdenek
> 

Re: [linux-lvm] confused with lvm2 filter rules

[Heming Zhao]

For record, I filed bug url:
https://bugzilla.redhat.com/show_bug.cgi?id=1723745

On 6/25/19 5:13 PM, Heming Zhao wrote:
> Hello All,
> 
> I will fire this bug.
> 
> On 6/25/19 4:56 PM, Zdenek Kabelac wrote:
>> Dne 25. 06. 19 v 9:56 Martin Wilck napsal(a):
>>> Hello Zdenek,
>>>
>>> On Tue, 2019-06-25 at 05:30 +0000, Heming Zhao wrote:
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>