[linux-lvm] " Failed to find physical volume <ZFS block device"

[linux-lvm] " Failed to find physical volume <ZFS block device"

[alessandro macuz]

[-- Attachment #1.1: Type: text/plain, Size: 1066 bytes --]

Hi all,

I sought the web for the error " Failed to find physical volume
"/dev/zd96p5"." but the closest discussion in the LVM archives I found was
one of mine back in 2014

https://listman.redhat.com/archives/linux-lvm/2014-June/msg00016.html

This time though it seems pvs fails to find the physical volume when
lvmdiskscan does recognize it. Do I have to really upgrade lvm2 or is there
another way to have pvs recognize the volume?

root@server:/etc# lvm version
  LVM version:     2.02.133(2) (2015-10-30)
  Library version: 1.02.110 (2015-10-30)
  Driver version:  4.34.0
root@ server :/etc# lvmdiskscan -l
  WARNING: only considering LVM devices
  /dev/sda5             [      74.05 GiB] LVM physical volume
  /dev/zd304p3          [      19.00 GiB] LVM physical volume
  /dev/zd96p5           [    1023.52 GiB] LVM physical volume
*  /dev/zd112p5          [    1023.52 GiB] LVM physical volume*
  0 LVM physical volume whole disks
  4 LVM physical volumes
root@ server :/etc# pvs -a /dev/zd96p5
  Failed to find physical volume "/dev/zd96p5".

Thanks, Alex.

[-- Attachment #1.2: Type: text/html, Size: 1936 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[alessandro macuz]

[-- Attachment #1.1: Type: text/plain, Size: 1529 bytes --]

fdisk correctly identifies the extended partition as 8e.
I wonder which kind of data lvmdiskscan and pvs use in order to list LVM
physical volumes.
Does PVS check some specific metadata within the partition without just
relying on the type of partition displayed by fdisk?

Alex

Le mer. 22 sept. 2021 à 01:36, alessandro macuz <alessandro.macuz@gmail.com>
a écrit :

> Hi all,
>
> I sought the web for the error " Failed to find physical volume
> "/dev/zd96p5"." but the closest discussion in the LVM archives I found
> was one of mine back in 2014
>
> https://listman.redhat.com/archives/linux-lvm/2014-June/msg00016.html
>
> This time though it seems pvs fails to find the physical volume when
> lvmdiskscan does recognize it. Do I have to really upgrade lvm2 or is there
> another way to have pvs recognize the volume?
>
> root@server:/etc# lvm version
>   LVM version:     2.02.133(2) (2015-10-30)
>   Library version: 1.02.110 (2015-10-30)
>   Driver version:  4.34.0
> root@ server :/etc# lvmdiskscan -l
>   WARNING: only considering LVM devices
>   /dev/sda5             [      74.05 GiB] LVM physical volume
>   /dev/zd304p3          [      19.00 GiB] LVM physical volume
>   /dev/zd96p5           [    1023.52 GiB] LVM physical volume
> *  /dev/zd112p5          [    1023.52 GiB] LVM physical volume*
>   0 LVM physical volume whole disks
>   4 LVM physical volumes
> root@ server :/etc# pvs -a /dev/zd96p5
>   Failed to find physical volume "/dev/zd96p5".
>
> Thanks, Alex.
>

[-- Attachment #1.2: Type: text/html, Size: 2670 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[Zdenek Kabelac]

Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
> fdisk correctly identifies the extended partition as 8e.
> I wonder which kind of data lvmdiskscan and pvs use in order to list LVM 
> physical volumes.
> Does PVS check some specific metadata within the partition without just 
> relying on the type of partition displayed by fdisk?
> 
>

Hi

Yes - PVs do have header signature keeping information about PV attributes
and also has the storage area to keep lvm2 metadata.

Partition flags known to fdisk are irrelevant.


Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[alessandro macuz]

[-- Attachment #1.1: Type: text/plain, Size: 989 bytes --]

Thanks Zdenek,

so it might be that metadata is corrupted somehow and hence the pvs program
doesn't recognize that partition as physical volume?
That may explain why lvmdiskscan reports physical disks (by just looking at
the partition type 8e) and pvs completely ignores it.
Am I correct?

Alex


Le jeu. 23 sept. 2021 à 15:52, Zdenek Kabelac <zkabelac@redhat.com> a
écrit :

> Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
> > fdisk correctly identifies the extended partition as 8e.
> > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM
> > physical volumes.
> > Does PVS check some specific metadata within the partition without just
> > relying on the type of partition displayed by fdisk?
> >
> >
>
> Hi
>
> Yes - PVs do have header signature keeping information about PV attributes
> and also has the storage area to keep lvm2 metadata.
>
> Partition flags known to fdisk are irrelevant.
>
>
> Regards
>
> Zdenek
>
>

[-- Attachment #1.2: Type: text/html, Size: 1428 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[Zdenek Kabelac]

[-- Attachment #1.1: Type: text/plain, Size: 1614 bytes --]

Dne 23. 09. 21 v 20:03 alessandro macuz napsal(a):
> Thanks Zdenek,
>
> so it might be that metadata is corrupted somehow and hence the pvs program 
> doesn't recognize that partition as physical volume?
> That may explain why lvmdiskscan reports physical disks (by just looking at 
> the partition type 8e) and pvs completely ignores it.
> Am I correct?
>

Hi


Yes - if your disk header part has lost/damaged its content - it will not be 
recognized as PV - thus completely ignored.

Note - the easiest is to check the verbose output of  'pvs -vvv' - where you 
could follow up what is command doing in relatively 'readable' form - if you 
can't follow it - just attach to the email for overlook why could be your disk 
eventually ignored.

Note - the other reason could be the device got filtered by some filter - but 
I assume you've not changed your configuration on your system ?


Zdenek


> Le jeu. 23 sept. 2021 à 15:52, Zdenek Kabelac <zkabelac@redhat.com> a écrit :
>
>     Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
>     > fdisk correctly identifies the extended partition as 8e.
>     > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM
>     > physical volumes.
>     > Does PVS check some specific metadata within the partition without just
>     > relying on the type of partition displayed by fdisk?
>     >
>     >
>
>     Hi
>
>     Yes - PVs do have header signature keeping information about PV attributes
>     and also has the storage area to keep lvm2 metadata.
>
>     Partition flags known to fdisk are irrelevant.
>
>
>     Regards
>
>     Zdenek
>

[-- Attachment #1.2: Type: text/html, Size: 3140 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[Roger Heflin]

If you have lvmetad running and in use then the lvm commands ask it
what the system has on it.

I have seen on random boots fairly separated systems (rhel7 versions,
and many years newer fedora systems) at random fail to find one or
more pv.s

I have disabled it at home, and in my day job we have also disabled
(across 20k+ systems) as we confirmed it had inconsistency issues
several times on a variety of our newest installs.

Stopping lvmetad and/or restarting it would generally fix it.    But
it was a source of enough random issues(often failure to mount on a
boot, so often issues that resulted in page-outs to debug)  and did
not speed things up much enough to be worth it even on devices with
>2000 SAN volumes.

On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac <zkabelac@redhat.com> wrote:
>
> Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
> > fdisk correctly identifies the extended partition as 8e.
> > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM
> > physical volumes.
> > Does PVS check some specific metadata within the partition without just
> > relying on the type of partition displayed by fdisk?
> >
> >
>
> Hi
>
> Yes - PVs do have header signature keeping information about PV attributes
> and also has the storage area to keep lvm2 metadata.
>
> Partition flags known to fdisk are irrelevant.
>
>
> Regards
>
> Zdenek
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[alessandro macuz]

[-- Attachment #1.1: Type: text/plain, Size: 2469 bytes --]

Thanks Roger, Zdenek,

I have my ZVOL on my NAS exposed as LUNs. The initiator were switched off
and for unknown reason I found my NAS switched off as well.
It had run for long and I feared the worst (CPU/motherboard/etc). Instead
once powered up everything started to work again but the LUNs that seemed
to jeopardized.
I have many ZVOLs used by ESXi in which I have EVE-NG who uses LVM and such
ZVOLs have the same size so I wanted to inspect them to check the hostname.

Now some LUNs started to work normally, some others still behave weirdly. I
will run pvs on them with extra debugs to see what's going on.

Many thanks,

Alex

Le jeu. 23 sept. 2021 à 23:48, Roger Heflin <rogerheflin@gmail.com> a
écrit :

> If you have lvmetad running and in use then the lvm commands ask it
> what the system has on it.
>
> I have seen on random boots fairly separated systems (rhel7 versions,
> and many years newer fedora systems) at random fail to find one or
> more pv.s
>
> I have disabled it at home, and in my day job we have also disabled
> (across 20k+ systems) as we confirmed it had inconsistency issues
> several times on a variety of our newest installs.
>
> Stopping lvmetad and/or restarting it would generally fix it.    But
> it was a source of enough random issues(often failure to mount on a
> boot, so often issues that resulted in page-outs to debug)  and did
> not speed things up much enough to be worth it even on devices with
> >2000 SAN volumes.
>
> On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac <zkabelac@redhat.com>
> wrote:
> >
> > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
> > > fdisk correctly identifies the extended partition as 8e.
> > > I wonder which kind of data lvmdiskscan and pvs use in order to list
> LVM
> > > physical volumes.
> > > Does PVS check some specific metadata within the partition without just
> > > relying on the type of partition displayed by fdisk?
> > >
> > >
> >
> > Hi
> >
> > Yes - PVs do have header signature keeping information about PV
> attributes
> > and also has the storage area to keep lvm2 metadata.
> >
> > Partition flags known to fdisk are irrelevant.
> >
> >
> > Regards
> >
> > Zdenek
> >
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@redhat.com
> > https://listman.redhat.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> >
>

[-- Attachment #1.2: Type: text/html, Size: 3358 bytes --]

[-- Attachment #2: Type: text/plain, Size: 201 bytes --]

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] " Failed to find physical volume <ZFS block device"

[Roger Heflin]

A possibility I just debugged for a non-booting system.

If there is a partition table on the underlying device then that
device is not detected as an LVM1/2 member in at least one version of
udevd, and won't be seen nor turned on automatically by the
systemd-udev code.

lvm vgchange -ay worked to enable it (emergency mode, it was the root
pv--no udevd involvement) and eventually I found the partition table
and removed it and the machine would then boot without needing a
manual intervention.

dd if=/dev/zero of=device bs=512 count=1 was used once we determined
there was a partition signature still left (after partition deletion
with fdisk, still had a header), examined with dd if=/dev/device
bs=512 count=1 | xxd found 4 non-zero bytes in the block.

On Sun, Sep 26, 2021 at 9:22 AM alessandro macuz
<alessandro.macuz@gmail.com> wrote:
>
> Thanks Roger, Zdenek,
>
> I have my ZVOL on my NAS exposed as LUNs. The initiator were switched off and for unknown reason I found my NAS switched off as well.
> It had run for long and I feared the worst (CPU/motherboard/etc). Instead once powered up everything started to work again but the LUNs that seemed to jeopardized.
> I have many ZVOLs used by ESXi in which I have EVE-NG who uses LVM and such ZVOLs have the same size so I wanted to inspect them to check the hostname.
>
> Now some LUNs started to work normally, some others still behave weirdly. I will run pvs on them with extra debugs to see what's going on.
>
> Many thanks,
>
> Alex
>
> Le jeu. 23 sept. 2021 à 23:48, Roger Heflin <rogerheflin@gmail.com> a écrit :
>>
>> If you have lvmetad running and in use then the lvm commands ask it
>> what the system has on it.
>>
>> I have seen on random boots fairly separated systems (rhel7 versions,
>> and many years newer fedora systems) at random fail to find one or
>> more pv.s
>>
>> I have disabled it at home, and in my day job we have also disabled
>> (across 20k+ systems) as we confirmed it had inconsistency issues
>> several times on a variety of our newest installs.
>>
>> Stopping lvmetad and/or restarting it would generally fix it.    But
>> it was a source of enough random issues(often failure to mount on a
>> boot, so often issues that resulted in page-outs to debug)  and did
>> not speed things up much enough to be worth it even on devices with
>> >2000 SAN volumes.
>>
>> On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac <zkabelac@redhat.com> wrote:
>> >
>> > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a):
>> > > fdisk correctly identifies the extended partition as 8e.
>> > > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM
>> > > physical volumes.
>> > > Does PVS check some specific metadata within the partition without just
>> > > relying on the type of partition displayed by fdisk?
>> > >
>> > >
>> >
>> > Hi
>> >
>> > Yes - PVs do have header signature keeping information about PV attributes
>> > and also has the storage area to keep lvm2 metadata.
>> >
>> > Partition flags known to fdisk are irrelevant.
>> >
>> >
>> > Regards
>> >
>> > Zdenek
>> >
>> > _______________________________________________
>> > linux-lvm mailing list
>> > linux-lvm@redhat.com
>> > https://listman.redhat.com/mailman/listinfo/linux-lvm
>> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>> >


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/