* [linux-lvm] Recovering from a failed pvmove
@ 2022-06-27 6:52 Roger James
2022-06-28 8:30 ` Roger James
2022-06-28 18:12 ` Roger Heflin
0 siblings, 2 replies; 4+ messages in thread
From: Roger James @ 2022-06-27 6:52 UTC (permalink / raw)
To: linux-lvm
[-- Attachment #1.1: Type: text/plain, Size: 4075 bytes --]
Hi,
I am struggling to recover from a failed pvmove. Unfortunately I only have
a limited knowledge of lvm. I setup my lvm configuration many years ago.
I was trying to move a lv to a SSD using pvmove. Unfortunately my brand new
SSD choose that moment to fail (never buy cheap SSDs, lesson learnt!").
This is the current status.
roger@dragon:~$ sudo pvs
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
PV VG Fmt Attr PSize PFree
/dev/sda1 wd lvm2 a-- <465.76g 0
/dev/sdb1 wd lvm2 a-- <465.76g <80.45g
/dev/sdc2 wd lvm2 a-- 778.74g 278.74g
/dev/sdd1 wd lvm2 a-- <465.76g 0
[unknown] wd lvm2 a-m <784.49g 685.66g
roger@dragon:~$ sudo lvs
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
home wd -wi------- 1.46t
root wd -wI-----p- <108.83g
swap wd -wi------- 8.00g
work wd -wi------- 200.00g
roger@dragon:~$ sudo vgs
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
VG #PV #LV #SN Attr VSize VFree
wd 5 4 0 wz-pn- 2.89t 1.02t
This is a recap of what I have tried so far.
roger@dragon:~$ sudo pvmove --abort
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
LVM command executed by lvmpolld failed.
For more information see lvmpolld messages in syslog or lvmpolld log file.
roger@dragon:~$ sudo vgreduce --removemissing wd
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: Partial LV root needs to be repaired or removed.
WARNING: Partial LV pvmove0 needs to be repaired or removed.
There are still partial LVs in VG wd.
To remove them unconditionally use: vgreduce --removemissing --force.
To remove them unconditionally from mirror LVs use: vgreduce
--removemissing --mirrorsonly --force.
WARNING: Proceeding to remove empty missing PVs.
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
roger@dragon:~$ sudo lvchange -an wd/root
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
roger@dragon:~$ sudo vgreduce --removemissing wd
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: Partial LV root needs to be repaired or removed.
WARNING: Partial LV pvmove0 needs to be repaired or removed.
There are still partial LVs in VG wd.
To remove them unconditionally use: vgreduce --removemissing --force.
To remove them unconditionally from mirror LVs use: vgreduce
--removemissing --mirrorsonly --force.
WARNING: Proceeding to remove empty missing PVs.
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
roger@dragon:~$ sudo lvremove wd/pvmove0
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
written to [unknown]).
WARNING: Couldn't find device with uuid
uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
Can't remove locked logical volume wd/pvmove0.
I am quite happy to loose the root lv, I just need the home and work lvs.
What am I missing?
Help!
Roger
[-- Attachment #1.2: Type: text/html, Size: 6799 bytes --]
[-- Attachment #2: Type: text/plain, Size: 202 bytes --]
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linux-lvm] Recovering from a failed pvmove
2022-06-27 6:52 [linux-lvm] Recovering from a failed pvmove Roger James
@ 2022-06-28 8:30 ` Roger James
2022-06-29 11:05 ` Roger Heflin
2022-06-28 18:12 ` Roger Heflin
1 sibling, 1 reply; 4+ messages in thread
From: Roger James @ 2022-06-28 8:30 UTC (permalink / raw)
To: LVM general discussion and development
[-- Attachment #1.1: Type: text/plain, Size: 4803 bytes --]
I have now managed to fix the problem. I ran vgcfgbackup, then made a copy
of the backup for safety purposes. I than hand edited the original backup
to remove the missing pv (pv4), the root lv and the pvmove0 lv. I then than
vgcfgrestore. Everything is working.
There must be a better way of doing this. Hand editing cfg files is not
safe or sensible. What have I missed?
Roger
On 28 June 2022 07:38:48 Roger James <roger@beardandsandals.co.uk> wrote:
> Hi,
>
> I am struggling to recover from a failed pvmove. Unfortunately I only have
> a limited knowledge of lvm. I setup my lvm configuration many years ago.
>
> I was trying to move a lv to a SSD using pvmove. Unfortunately my brand new
> SSD choose that moment to fail (never buy cheap SSDs, lesson learnt!").
>
> This is the current status.
>
> roger@dragon:~$ sudo pvs
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> PV VG Fmt Attr PSize PFree
> /dev/sda1 wd lvm2 a-- <465.76g 0
> /dev/sdb1 wd lvm2 a-- <465.76g <80.45g
> /dev/sdc2 wd lvm2 a-- 778.74g 278.74g
> /dev/sdd1 wd lvm2 a-- <465.76g 0
> [unknown] wd lvm2 a-m <784.49g 685.66g
> roger@dragon:~$ sudo lvs
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
> home wd -wi------- 1.46t
> root wd -wI-----p- <108.83g
> swap wd -wi------- 8.00g
> work wd -wi------- 200.00g
> roger@dragon:~$ sudo vgs
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> VG #PV #LV #SN Attr VSize VFree
> wd 5 4 0 wz-pn- 2.89t 1.02t
>
> This is a recap of what I have tried so far.
>
> roger@dragon:~$ sudo pvmove --abort
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> LVM command executed by lvmpolld failed.
> For more information see lvmpolld messages in syslog or lvmpolld log file.
> roger@dragon:~$ sudo vgreduce --removemissing wd
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: Partial LV root needs to be repaired or removed.
> WARNING: Partial LV pvmove0 needs to be repaired or removed.
> There are still partial LVs in VG wd.
> To remove them unconditionally use: vgreduce --removemissing --force.
> To remove them unconditionally from mirror LVs use: vgreduce
> --removemissing --mirrorsonly --force.
> WARNING: Proceeding to remove empty missing PVs.
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> roger@dragon:~$ sudo lvchange -an wd/root
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> roger@dragon:~$ sudo vgreduce --removemissing wd
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: Partial LV root needs to be repaired or removed.
> WARNING: Partial LV pvmove0 needs to be repaired or removed.
> There are still partial LVs in VG wd.
> To remove them unconditionally use: vgreduce --removemissing --force.
> To remove them unconditionally from mirror LVs use: vgreduce
> --removemissing --mirrorsonly --force.
> WARNING: Proceeding to remove empty missing PVs.
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> roger@dragon:~$ sudo lvremove wd/pvmove0
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last
> written to [unknown]).
> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> Can't remove locked logical volume wd/pvmove0.
>
> I am quite happy to loose the root lv, I just need the home and work lvs.
> What am I missing?
>
> Help!
>
> Roger
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
[-- Attachment #1.2: Type: text/html, Size: 8397 bytes --]
[-- Attachment #2: Type: text/plain, Size: 202 bytes --]
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linux-lvm] Recovering from a failed pvmove
2022-06-27 6:52 [linux-lvm] Recovering from a failed pvmove Roger James
2022-06-28 8:30 ` Roger James
@ 2022-06-28 18:12 ` Roger Heflin
1 sibling, 0 replies; 4+ messages in thread
From: Roger Heflin @ 2022-06-28 18:12 UTC (permalink / raw)
To: LVM general discussion and development
[-- Attachment #1.1: Type: text/plain, Size: 5646 bytes --]
For a case like this vgcfgrestore is probably the best option. man
vgcfgrestore.
You need to see if you have archived vg copies that you can revert to
before the "add" of the pv that went bad.
The archives are typically in /etc/lvm/archiive/<vgname>* on RedHat
derivative OSes, not sure if they are different(and/or configured to exist)
on other distributions.
grep -i before /etc/lvm/archive/<vgname>* and see which archive was made
before the initial pv addition. vgcfgrestore -f <goodconfig> should work
but I usually have to adjust command line options to get it work when I
have used it to revert configs. I think in that case it will find the vg
and pvid correctly. No cleanup should need to be done so long as the other
device is completely gone.
And you will probably need to answer some prompts and warnings, and then
reboot the machine, and/or do this all under a livecd rescue boot.
What kind of cheap ssd were you using? I have had really bad luck with
ones without RAM. I RMA'ed one that failed in under a week and the new one
also failed in a very similar way in under a week.
On Tue, Jun 28, 2022 at 1:38 AM Roger James <roger@beardandsandals.co.uk>
wrote:
> Hi,
>
> I am struggling to recover from a failed pvmove. Unfortunately I only have
> a limited knowledge of lvm. I setup my lvm configuration many years ago.
>
> I was trying to move a lv to a SSD using pvmove. Unfortunately my brand
> new SSD choose that moment to fail (never buy cheap SSDs, lesson learnt!").
>
> This is the current status.
>
> roger@dragon:~$ sudo pvs
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> PV VG Fmt Attr PSize PFree
> /dev/sda1 wd lvm2 a-- <465.76g 0
> /dev/sdb1 wd lvm2 a-- <465.76g <80.45g
> /dev/sdc2 wd lvm2 a-- 778.74g 278.74g
> /dev/sdd1 wd lvm2 a-- <465.76g 0
> [unknown] wd lvm2 a-m <784.49g 685.66g
> roger@dragon:~$ sudo lvs
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
> home wd -wi------- 1.46t
> root wd -wI-----p- <108.83g
>
> swap wd -wi------- 8.00g
> work wd -wi------- 200.00g
> roger@dragon:~$ sudo vgs
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> VG #PV #LV #SN Attr VSize VFree
> wd 5 4 0 wz-pn- 2.89t 1.02t
>
> This is a recap of what I have tried so far.
>
> roger@dragon:~$ sudo pvmove --abort
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> LVM command executed by lvmpolld failed.
> For more information see lvmpolld messages in syslog or lvmpolld log
> file.
> roger@dragon:~$ sudo vgreduce --removemissing wd
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: Partial LV root needs to be repaired or removed.
> WARNING: Partial LV pvmove0 needs to be repaired or removed.
> There are still partial LVs in VG wd.
> To remove them unconditionally use: vgreduce --removemissing --force.
> To remove them unconditionally from mirror LVs use: vgreduce
> --removemissing --mirrorsonly --force.
> WARNING: Proceeding to remove empty missing PVs.
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> roger@dragon:~$ sudo lvchange -an wd/root
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> roger@dragon:~$ sudo vgreduce --removemissing wd
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: Partial LV root needs to be repaired or removed.
> WARNING: Partial LV pvmove0 needs to be repaired or removed.
> There are still partial LVs in VG wd.
> To remove them unconditionally use: vgreduce --removemissing --force.
> To remove them unconditionally from mirror LVs use: vgreduce
> --removemissing --mirrorsonly --force.
> WARNING: Proceeding to remove empty missing PVs.
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> roger@dragon:~$ sudo lvremove wd/pvmove0
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ
> (last written to [unknown]).
> WARNING: Couldn't find device with uuid
> uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
> Can't remove locked logical volume wd/pvmove0.
>
> I am quite happy to loose the root lv, I just need the home and work lvs.
> What am I missing?
>
> Help!
>
> Roger
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>
[-- Attachment #1.2: Type: text/html, Size: 8295 bytes --]
[-- Attachment #2: Type: text/plain, Size: 202 bytes --]
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [linux-lvm] Recovering from a failed pvmove
2022-06-28 8:30 ` Roger James
@ 2022-06-29 11:05 ` Roger Heflin
0 siblings, 0 replies; 4+ messages in thread
From: Roger Heflin @ 2022-06-29 11:05 UTC (permalink / raw)
To: LVM general discussion and development
Did your operating system have backups/archives of the old cfgs?
Fedora/Redhat keeps around 20 old copies going back typically several
months and so typically one of those can be used.
My archive dir looks like this for the roovg (note I do not do a lot
of vg work, typically
ls -l /etc/lvm/archive/fedora*
-rw-------. 1 root root 3159 Oct 16 2021
/etc/lvm/archive/fedora_00031-1021957302.vg
-rw-------. 1 root root 2740 Oct 16 2021
/etc/lvm/archive/fedora_00032-1140767599.vg
-rw-------. 1 root root 2331 Oct 16 2021
/etc/lvm/archive/fedora_00033-104560841.vg
-rw-------. 1 root root 2084 Oct 16 2021
/etc/lvm/archive/fedora_00034-239698665.vg
-rw-------. 1 root root 2330 Oct 16 2021
/etc/lvm/archive/fedora_00035-1994061504.vg
-rw-------. 1 root root 1973 Oct 16 2021
/etc/lvm/archive/fedora_00036-974449793.vg
-rw-------. 1 root root 1997 Oct 16 2021
/etc/lvm/archive/fedora_00037-1503932417.vg
-rw-------. 1 root root 1997 Oct 16 2021
/etc/lvm/archive/fedora_00038-951442204.vg
-rw-------. 1 root root 1997 Oct 16 2021
/etc/lvm/archive/fedora_00039-989943813.vg
-rw-------. 1 root root 1968 Oct 16 2021
/etc/lvm/archive/fedora_00040-815563362.vg
-rw-------. 1 root root 1997 Oct 16 2021
/etc/lvm/archive/fedora_00041-1303737065.vg
-rw-------. 1 root root 1968 Oct 16 2021
/etc/lvm/archive/fedora_00042-714254626.vg
-rw-------. 1 root root 1976 Oct 16 2021
/etc/lvm/archive/fedora_00043-858775161.vg
-rw-------. 1 root root 2229 Oct 16 2021
/etc/lvm/archive/fedora_00044-1360584830.vg
-rw-------. 1 root root 2238 Oct 16 2021
/etc/lvm/archive/fedora_00045-1806472194.vg
-rw-------. 1 root root 2218 Oct 16 2021
/etc/lvm/archive/fedora_00046-519515389.vg
-rw-------. 1 root root 1992 Oct 16 2021
/etc/lvm/archive/fedora_00047-1997891375.vg
-rw-------. 1 root root 1988 Oct 16 2021
/etc/lvm/archive/fedora_00048-1666128451.vg
-rw-------. 1 root root 2396 Oct 18 2021
/etc/lvm/archive/fedora_00049-67607509.vg
-rw-------. 1 root root 2396 Oct 18 2021
/etc/lvm/archive/fedora_00050-1102265641.vg
-rw-------. 1 root root 2404 Oct 18 2021
/etc/lvm/archive/fedora_00051-243833122.vg
-rw-------. 1 root root 1738 Oct 18 2021
/etc/lvm/archive/fedora_00052-1619420890.vg
-rw-------. 1 root root 1744 Oct 18 2021
/etc/lvm/archive/fedora_00053-149551096.vg
-rw-------. 1 root root 1748 Nov 11 2021
/etc/lvm/archive/fedora_00054-1785934566.vg
-rw-------. 1 root root 1748 Nov 11 2021
/etc/lvm/archive/fedora_00055-1992369253.vg
-rw-------. 1 root root 1719 Nov 11 2021
/etc/lvm/archive/fedora_00056-26730175.vg
-rw-------. 1 root root 1748 Nov 11 2021
/etc/lvm/archive/fedora_00057-1989410182.vg
-rw-------. 1 root root 1719 Nov 11 2021
/etc/lvm/archive/fedora_00058-1590942582.vg
-rw-------. 1 root root 1748 Nov 11 2021
/etc/lvm/archive/fedora_00059-140743745.vg
I have only personally had to edit the lvm archive I needed to use a
few times, and that has always been to remove the MISSING tag from
PV's that are no longer missing and the missing tag blocks the archive
from being used.
Most of the time I am correcting VG's I revert to the archive copy
that was taked before the bad steps were done.
There are options that can be put in /etc/lvm/lvm.conf and
/etc/lvm/lvmlocal.conf that can cause the archive's to get collected
if they are not being automatically collected, I am pretty sure I have
seen a few distributions that do not seem to collect the archives that
do typically give you a restore point to revert to (without editing
the file directly).
On Wed, Jun 29, 2022 at 2:32 AM Roger James <roger@beardandsandals.co.uk> wrote:
>
> I have now managed to fix the problem. I ran vgcfgbackup, then made a copy of the backup for safety purposes. I than hand edited the original backup to remove the missing pv (pv4), the root lv and the pvmove0 lv. I then than vgcfgrestore. Everything is working.
>
> There must be a better way of doing this. Hand editing cfg files is not safe or sensible. What have I missed?
>
> Roger
>
> On 28 June 2022 07:38:48 Roger James <roger@beardandsandals.co.uk> wrote:
>>
>> Hi,
>>
>> I am struggling to recover from a failed pvmove. Unfortunately I only have a limited knowledge of lvm. I setup my lvm configuration many years ago.
>>
>> I was trying to move a lv to a SSD using pvmove. Unfortunately my brand new SSD choose that moment to fail (never buy cheap SSDs, lesson learnt!").
>>
>> This is the current status.
>>
>> roger@dragon:~$ sudo pvs
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> PV VG Fmt Attr PSize PFree
>> /dev/sda1 wd lvm2 a-- <465.76g 0
>> /dev/sdb1 wd lvm2 a-- <465.76g <80.45g
>> /dev/sdc2 wd lvm2 a-- 778.74g 278.74g
>> /dev/sdd1 wd lvm2 a-- <465.76g 0
>> [unknown] wd lvm2 a-m <784.49g 685.66g
>> roger@dragon:~$ sudo lvs
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
>> home wd -wi------- 1.46t
>> root wd -wI-----p- <108.83g
>> swap wd -wi------- 8.00g
>> work wd -wi------- 200.00g
>> roger@dragon:~$ sudo vgs
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> VG #PV #LV #SN Attr VSize VFree
>> wd 5 4 0 wz-pn- 2.89t 1.02t
>>
>> This is a recap of what I have tried so far.
>>
>> roger@dragon:~$ sudo pvmove --abort
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> LVM command executed by lvmpolld failed.
>> For more information see lvmpolld messages in syslog or lvmpolld log file.
>> roger@dragon:~$ sudo vgreduce --removemissing wd
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: Partial LV root needs to be repaired or removed.
>> WARNING: Partial LV pvmove0 needs to be repaired or removed.
>> There are still partial LVs in VG wd.
>> To remove them unconditionally use: vgreduce --removemissing --force.
>> To remove them unconditionally from mirror LVs use: vgreduce --removemissing --mirrorsonly --force.
>> WARNING: Proceeding to remove empty missing PVs.
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> roger@dragon:~$ sudo lvchange -an wd/root
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> roger@dragon:~$ sudo vgreduce --removemissing wd
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: Partial LV root needs to be repaired or removed.
>> WARNING: Partial LV pvmove0 needs to be repaired or removed.
>> There are still partial LVs in VG wd.
>> To remove them unconditionally use: vgreduce --removemissing --force.
>> To remove them unconditionally from mirror LVs use: vgreduce --removemissing --mirrorsonly --force.
>> WARNING: Proceeding to remove empty missing PVs.
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> roger@dragon:~$ sudo lvremove wd/pvmove0
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> WARNING: VG wd is missing PV uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ (last written to [unknown]).
>> WARNING: Couldn't find device with uuid uMtjop-PmMT-603f-GWWQ-fR4f-s4Sw-XSKNXZ.
>> Can't remove locked logical volume wd/pvmove0.
>>
>> I am quite happy to loose the root lv, I just need the home and work lvs. What am I missing?
>>
>> Help!
>>
>> Roger
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm@redhat.com
>> https://listman.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://listman.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-06-29 11:06 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-27 6:52 [linux-lvm] Recovering from a failed pvmove Roger James
2022-06-28 8:30 ` Roger James
2022-06-29 11:05 ` Roger Heflin
2022-06-28 18:12 ` Roger Heflin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).