[linux-lvm] inconsistent metadata

[linux-lvm] inconsistent metadata

[Mike Williams]

[-- Attachment #1: Type: Text/Plain, Size: 3869 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi guys,

As you may have guessed, from my posting, I'm having some LVM trouble.

Basically this is what has happened.
I got a new 200gig drive, and created an lvm1 pv/vg/lv. Copied loads of data 
onto it, then put the drive into another machine.
After this worked I changed the metadata to lvm2.
Added another drive to the machine and extended the LV and extended the 
filesystem, I then added another drive but created a new LV in the same VG, 
then a third drive was added to a third LV.
And all was well, until...

Yesterday, even though everything was working, I was getting errors like:
Couldn't find device with uuid 'CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc'.
Had to swap out the PSU this evening and then upon restarting couldn't 
activate my VG :(

gandalf root # pvdisplay
  Couldn't find device with uuid 'CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc'.
  Couldn't find device with uuid 'CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc'.
  Couldn't find device with uuid 'CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc'.
  Physical volume "unknown device" of volume group "filestore" is exported
  --- Physical volume ---
  PV Name               unknown device
  VG Name               filestore (exported)
  PV Size               189.88 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              6076
  Free PE               0
  Allocated PE          6076
  PV UUID               CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc

"unknown device" is /dev/hde, my new 200gig drive, which IS working fine, and 
IS exactly where is was before. So, after some googling, and practice on 
another empty drive I decided to do:

gandalf root # pvcreate --uuid CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc /dev/hde
  No physical volume label read from /dev/hde
  Physical volume "/dev/hde" successfully created
Which fixed pvdisplay, woo (not)

gandalf root # pvdisplay |head
  Physical volume "/dev/hde" of volume group "filestore" is exported
  --- Physical volume ---
  PV Name               /dev/hde
  VG Name               filestore (exported)
  PV Size               189.88 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              6076
  Free PE               0
  Allocated PE          6076
gandalf root # vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "filestore" using metadata type lvm2
gandalf root # vgchange -a y
  Volume group filestore metadata is inconsistent
  Volume group for uuid not found: 
3InK3jOE71aeco40b7L5c1cfU78EJpsk00000000000000000000000000000000
  Volume group filestore metadata is inconsistent
  Volume group for uuid not found: 
3InK3jOE71aeco40b7L5c1cfU78EJpskPVV5#Hyv1z60d#OVfdcJC9TbA3tfsCV6
  Volume group filestore metadata is inconsistent
  Volume group for uuid not found: 
3InK3jOE71aeco40b7L5c1cfU78EJpskp3dZg9RLNdmdRZFx96sbzQ9PEsZ37Wbn
  0 logical volume(s) in volume group "filestore" now active

Aargggh!

Can someone suggest a course of action to get at my data, please?
I just need to iron out this kink in the road and my life long devotion to LVM 
will be complete! :)
Cheers

p.s. gandalf is an Athlon 1.4GHz (at 1GHz), with a gig of ram, on board ATA133 
and a Promise PCI ATA66, running 2.6.0-test6, and LVM 2.00.06. The 3 drives 
IDE drives are on the PCI card (with an additional drive), 4 drives on the 
on-board controller, and 2 SCSI disks is on an Adaptec AIC-7892A U160.
Attached is (hopefully) the full output of {pv,vg,lv}display and 
{pv,vg,lv}scan.

- -- 
Mike Williams
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/hb5KInuLMrk7bIwRAuiuAJ9Fc+jSWmwIz3M7ztHAyZVWA8JgxwCfb6Io
XgtkwLcYwFSPOHkAB+5GHGI=
=RukW
-----END PGP SIGNATURE-----

[-- Attachment #2: lvm-grr --]
[-- Type: text/x-diff, Size: 4276 bytes --]

gandalf root # pvdisplay
  Physical volume "/dev/hde" of volume group "filestore" is exported
  --- Physical volume ---
  PV Name               /dev/hde
  VG Name               filestore (exported)
  PV Size               189.88 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              6076
  Free PE               0
  Allocated PE          6076
  PV UUID               CrijJ6-qPj1-cSYX-HzQF-D4M8-TUkA-yK3FYc

  Physical volume "/dev/hdg" of volume group "filestore" is exported
  --- Physical volume ---
  PV Name               /dev/hdg
  VG Name               filestore (exported)
  PV Size               76.28 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              2441
  Free PE               0
  Allocated PE          2441
  PV UUID               PdVjWD-7uc7-G7oT-s55m-1JXo-UXIf-bkWZOv

  --- Physical volume ---
  PV Name               /dev/hdf
  VG Name               filestore
  PV Size               74.50 GB / not usable 0
  Allocatable           yes (but full)
  PE Size (KByte)       32768
  Total PE              2384
  Free PE               0
  Allocated PE          2384
  PV UUID               WYGR#I-IkRH-mwfV-nv2O-ga2B-P#zI-gm0etL

  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               filestore
  PV Size               16.22 GB / not usable 0
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              519
  Free PE               519
  Allocated PE          0
  PV UUID               g4n1Mw-5Pal-#Hjq-OvDd-RLUA-J06m-O8RV2f

gandalf root # vgdisplay
  --- Volume group ---
  VG Name               filestore
  System ID             legolas.home.gaima.co.uk1064954139
  Format                lvm2
  Metadata Areas        3
  Metadata Sequence No  9
  VG Access             read/write
  VG Status             resizable
  MAX LV                256
  Cur LV                3
  Open LV               0
  Max PV                256
  Cur PV                4
  Act PV                4
  VG Size               356.88 GB
  PE Size               32.00 MB
  Total PE              11420
  Alloc PE / Size       10901 / 340.66 GB
  Free  PE / Size       519 / 16.22 GB
  VG UUID               3InK3j-OE71-aeco-40b7-L5c1-cfU7-8EJpsk

gandalf root # lvdisplay
  --- Logical volume ---
  LV Name                /dev/filestore/movies
  VG Name                filestore
  LV UUID                000000-0000-0000-0000-0000-0000-000000
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                241.16 GB
  Current LE             7717
  Segments               2
  Allocation             next free
  Read ahead sectors     1024

  --- Logical volume ---
  LV Name                /dev/filestore/star-trek
  VG Name                filestore
  LV UUID                PVV5#H-yv1z-60d#-OVfd-cJC9-TbA3-tfsCV6
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                74.50 GB
  Current LE             2384
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0

  --- Logical volume ---
  LV Name                /dev/filestore/usr
  VG Name                filestore
  LV UUID                p3dZg9-RLNd-mdRZ-Fx96-sbzQ-9PEs-Z37Wbn
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                25.00 GB
  Current LE             800
  Segments               1
  Allocation             next free (default)
  Read ahead sectors     0


gandalf root # pvscan
  PV /dev/hde     is in exported VG filestore [189.88 GB / 0    free]
  PV /dev/hdg     is in exported VG filestore [76.28 GB / 0    free]
  PV /dev/hdf    VG filestore   lvm2 [74.50 GB / 0    free]
  PV /dev/sda2   VG filestore   lvm2 [16.22 GB / 16.22 GB free]
  Total: 4 [16.22 GB] / in use: 4 [16.22 GB] / in no VG: 0 [0   ]
gandalf root # vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "filestore" using metadata type lvm2
gandalf root # lvscan
  inactive          '/dev/filestore/movies' [241.16 GB] next free
  inactive          '/dev/filestore/star-trek' [74.50 GB] next free (default)
  inactive          '/dev/filestore/usr' [25.00 GB] next free (default)

Re: [linux-lvm] inconsistent metadata

[Alasdair G Kergon]

On Thu, Oct 09, 2003 at 09:00:10PM +0100, Mike Williams wrote:
> Can someone suggest a course of action to get at my data, please?

Something strange must have happened during that long sequence of
operations - perhaps adding disks containing an exported volume
group with the same name as one already on the machine, which makes
LVM2 confused, it seems?

You can look at the metadata history in /etc/lvm/archive to try to
work out the sequence of events and at what point things started
going wrong.  [If you'd like me to take a look, send me a tarball
of /etc/lvm.]

Filters in the config file can be used to hide the 2 devices that say 
they're exported while you rename the real VG.

Alasdair
-- 
agk@uk.sistina.com

Re: [linux-lvm] inconsistent metadata

[Mike Williams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 10 October 2003 19:42, Alasdair G Kergon wrote:

> You can look at the metadata history in /etc/lvm/archive to try to
> work out the sequence of events and at what point things started
> going wrong.  [If you'd like me to take a look, send me a tarball
> of /etc/lvm.]

OK, I've look through the archive files and can't really see anything odd, but 
I'm no expert.
I will send /etc/lvm to you separately, any help you could offer will be 
gratefully received.

> Filters in the config file can be used to hide the 2 devices that say
> they're exported while you rename the real VG.

Excuse my ignorance, but I don't understand :)
I assume this is a reference to the 2 PV's which pvscan is saying are exported 
in a VG, while the other 2 don't?

- -- 
Mike Williams
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/h+ZpInuLMrk7bIwRAk4YAJwJV/hIEksCzvcScYtbCVmUVGiwdACggMbF
ope9x3trhJLux4XpQNdA7WA=
=NuZ8
-----END PGP SIGNATURE-----

Re: [linux-lvm] inconsistent metadata

[Mike Williams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 09 October 2003 21:00, Mike Williams wrote:
> Can someone suggest a course of action to get at my data, please?
> I just need to iron out this kink in the road and my life long devotion to
> LVM will be complete! :)

I took a leap of faith and vgcfgbackup'd, and vgcfgrestore'd, and now I have 
my data back! w00t

- -- 
Mike Williams
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/jDsvInuLMrk7bIwRAuy8AJ43UCJxWsAVKlhPrUy7EWic3aWxNwCdHgE7
H6tb/+MGGYiBa7nIvXESOjo=
=dsOr
-----END PGP SIGNATURE-----

[linux-lvm] Inconsistent metadata

[Marius Gravdal]

Hi!
Just thought I'd share something with all of you. 
I had a problem with my LVM recently and I was quite frustrated when I
couldn't fix it. I've been searching around the net for quite some
time now before I managed to fix it thanks to a post to this list.

Here's the problem I had, exactly the same problem actually. Just a
little bit different hardware setup.
http://www.redhat.com/archives/linux-lvm/2003-October/msg00041.html

And here's the solution in the same thread:
http://www.redhat.com/archives/linux-lvm/2003-October/msg00055.html

I was just wondering if it would be possible to add this to the
official LVM FAQ, cause I was about to wipe my entire LVM and start
over from (the-not-so-complete) backups. If this info would be easier
to access we might save someone else out there from a lot of downtime
and from maybe even doing what I planned to do.

All that said, thanks for saving my LVM. It made my day :) (or
something more like a month probably)

-- marius