* [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? @ 2021-12-19 16:43 Tomas Dalebjörk 2021-12-20 14:51 ` Phillip Susi 2021-12-21 13:46 ` Zdenek Kabelac 0 siblings, 2 replies; 11+ messages in thread From: Tomas Dalebjörk @ 2021-12-19 16:43 UTC (permalink / raw) To: LVM general discussion and development [-- Attachment #1.1: Type: text/plain, Size: 2318 bytes --] Hi, I am trying to understand how to convert a disk containing snapshot data. This is how I tested this: 1. locate the snapshot testlv.211218.232255 root@debian10:/dev/mapper# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert home debian10-vg -wi-ao---- 1.00g root debian10-vg -wi-ao---- <3.81g swap_1 debian10-vg -wi-ao---- 976.00m testlv debian10-vg owi-aos--- 100.00m testlv.211218.232255 debian10-vg swi-a-s--- 104.00m testlv 1.44 root@debian10:/dev/mapper# 2. copy the lv - cow data to a file # dd if=/dev/mapper/debian10--vg-testlv.211218.232255-cow of=/tmp/out bs=1024 3. Setup a loopback disk for the file # losetup -fP /tmp/out 4. Verify that disk exists root@debian10:/dev/mapper# losetup -a /dev/loop0: [65025]:39274 (/tmp/out) root@debian10:/dev/mapper# 5. Try converting the disk using lvconvert command # lvconvert -Zn -s debian10-vg/testlv /tmp/out "/tmp/out": Invalid path for Logical Volume. 6. Trying creating a softlink in /dev/mapper # ln -s /tmp/out debian10--vg-loopback 7. verify link root@debian10:/dev/mapper# ls -la total 0 drwxr-xr-x 2 root root 240 Dec 19 17:40 . drwxr-xr-x 18 root root 3820 Dec 19 17:22 .. crw------- 1 root root 10, 236 Dec 10 19:10 control lrwxrwxrwx 1 root root 7 Dec 10 19:10 debian10--vg-home -> ../dm-5 lrwxrwxrwx 1 root root 8 Dec 19 17:40 debian10--vg-loopback -> /tmp/out lrwxrwxrwx 1 root root 7 Dec 13 22:20 debian10--vg-root -> ../dm-1 lrwxrwxrwx 1 root root 7 Dec 10 19:10 debian10--vg-swap_1 -> ../dm-4 lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv -> ../dm-0 lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv.211218.232255 -> ../dm-6 lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv.211218.232255-cow -> ../dm-3 lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv-real -> ../dm-2 root@debian10:/dev/mapper# 8. retrying lvconvert command root@debian10:/dev/mapper# lvconvert -Zn -s debian10-vg/testlv /dev/mapper/debian10--vg-loopback Failed to find logical volume "debian10-vg/loopback" root@debian10:/dev/mapper# Are there more things to be considered, such as recreating pv data on disk? Regards Tomas [-- Attachment #1.2: Type: text/html, Size: 3814 bytes --] [-- Attachment #2: Type: text/plain, Size: 201 bytes --] _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-19 16:43 [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? Tomas Dalebjörk @ 2021-12-20 14:51 ` Phillip Susi 2021-12-21 13:46 ` Zdenek Kabelac 1 sibling, 0 replies; 11+ messages in thread From: Phillip Susi @ 2021-12-20 14:51 UTC (permalink / raw) To: LVM general discussion and development; +Cc: Tomas Dalebjörk I'm confused. What exactly are you trying to do here? I can't think of any reason why you would want to copy a snapshot lv to a file then try to loop mount it. Tomas Dalebjörk <tomas.dalebjork@gmail.com> writes: > Hi, > > I am trying to understand how to convert a disk containing snapshot data. > This is how I tested this: > 1. locate the snapshot testlv.211218.232255 > root@debian10:/dev/mapper# lvs > LV VG Attr LSize Pool Origin Data% > Meta% Move Log Cpy%Sync Convert > home debian10-vg -wi-ao---- 1.00g > root debian10-vg -wi-ao---- <3.81g > swap_1 debian10-vg -wi-ao---- 976.00m > testlv debian10-vg owi-aos--- 100.00m > testlv.211218.232255 debian10-vg swi-a-s--- 104.00m testlv 1.44 > root@debian10:/dev/mapper# > > 2. copy the lv - cow data to a file > # dd if=/dev/mapper/debian10--vg-testlv.211218.232255-cow of=/tmp/out > bs=1024 > > 3. Setup a loopback disk for the file > # losetup -fP /tmp/out > > 4. Verify that disk exists > root@debian10:/dev/mapper# losetup -a > /dev/loop0: [65025]:39274 (/tmp/out) > root@debian10:/dev/mapper# > > 5. Try converting the disk using lvconvert command > # lvconvert -Zn -s debian10-vg/testlv /tmp/out > "/tmp/out": Invalid path for Logical Volume. > > 6. Trying creating a softlink in /dev/mapper > # ln -s /tmp/out debian10--vg-loopback > > 7. verify link > root@debian10:/dev/mapper# ls -la > total 0 > drwxr-xr-x 2 root root 240 Dec 19 17:40 . > drwxr-xr-x 18 root root 3820 Dec 19 17:22 .. > crw------- 1 root root 10, 236 Dec 10 19:10 control > lrwxrwxrwx 1 root root 7 Dec 10 19:10 debian10--vg-home -> ../dm-5 > lrwxrwxrwx 1 root root 8 Dec 19 17:40 debian10--vg-loopback -> > /tmp/out > lrwxrwxrwx 1 root root 7 Dec 13 22:20 debian10--vg-root -> ../dm-1 > lrwxrwxrwx 1 root root 7 Dec 10 19:10 debian10--vg-swap_1 -> ../dm-4 > lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv -> ../dm-0 > lrwxrwxrwx 1 root root 7 Dec 18 23:23 > debian10--vg-testlv.211218.232255 -> ../dm-6 > lrwxrwxrwx 1 root root 7 Dec 18 23:23 > debian10--vg-testlv.211218.232255-cow -> ../dm-3 > lrwxrwxrwx 1 root root 7 Dec 18 23:23 debian10--vg-testlv-real -> > ../dm-2 > root@debian10:/dev/mapper# > > 8. retrying lvconvert command > root@debian10:/dev/mapper# lvconvert -Zn -s debian10-vg/testlv > /dev/mapper/debian10--vg-loopback > Failed to find logical volume "debian10-vg/loopback" > root@debian10:/dev/mapper# > > Are there more things to be considered, such as recreating pv data on disk? > > Regards Tomas > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-19 16:43 [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? Tomas Dalebjörk 2021-12-20 14:51 ` Phillip Susi @ 2021-12-21 13:46 ` Zdenek Kabelac 2021-12-21 14:44 ` Tomas Dalebjörk 1 sibling, 1 reply; 11+ messages in thread From: Zdenek Kabelac @ 2021-12-21 13:46 UTC (permalink / raw) To: LVM general discussion and development, Tomas Dalebjörk Dne 19. 12. 21 v 17:43 Tomas Dalebjörk napsal(a): > Hi, > > I am trying to understand how to convert a disk containing snapshot data. > This is how I tested this: > 1. locate the snapshot testlv.211218.232255 > root@debian10:/dev/mapper# lvs > LV VG Attr LSize Pool Origin Data% > Meta% Move Log Cpy%Sync Convert > home debian10-vg -wi-ao---- 1.00g > root debian10-vg -wi-ao---- <3.81g > swap_1 debian10-vg -wi-ao---- 976.00m > testlv debian10-vg owi-aos--- 100.00m > testlv.211218.232255 debian10-vg swi-a-s--- 104.00m testlv 1.44 > root@debian10:/dev/mapper# > Hi It looks like there is some sort of misunderstanding HOW these snapshots with lvm2 do work. The snapshot LV does NOT contain whole copy of the origin LV (the one you've take its snapshot) So while 'lvs' is presenting you the size of snapshot - it's not a 'regular' volume behind the scene. Instead it's a set of differentiating blocks from your origin stored in a way the came in use. So while your snapshot looks it has size 104Mib and origin 100MiB - it means there is just allocated max possible size of snapshot to store all different chunk + some extra metadata. But ATM only 1.44% of this space is actually used to hold all these block (AKA ~1.5MiB or real storage was used so far) There is basically NO WAY to use this 'COW' storage area without its original volume. If you want to use such 'snapshot' elsewhere you simply need to *dd* such LV - which is presented (via magic world of DM targets) to user-space to appear like your original LV in the moment of snapshot. But the actual 'raw' content stored in disk is in 'internal' form - not really usable outside (although the format is not really a complicated one) Note - these so called 'thick' snapshot or 'old' snapshot are now better handled via thin-provisioning - giving much better performance - especially if you plan to keep snapshot for long term or have it in bigger size. Hopefully it now makes it way more clear. Regards Zdenek _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 13:46 ` Zdenek Kabelac @ 2021-12-21 14:44 ` Tomas Dalebjörk 2021-12-21 15:30 ` Zdenek Kabelac 2021-12-21 19:23 ` Phillip Susi 0 siblings, 2 replies; 11+ messages in thread From: Tomas Dalebjörk @ 2021-12-21 14:44 UTC (permalink / raw) To: Zdenek Kabelac; +Cc: LVM general discussion and development hi I think I didn’t explain this clear enough Allthe lvm data is present in the snapshot that I provision from our backup system I can guarantee that! If I just mount that snapshot from our backup system, it works perfectly well so we don’t need the origin volumes in other way than copying back to it we just need to reanimate it as a cow volume mentioning that all data has been changed the cow is just referencing to the origin location, so no problem there All our data is in the cow volume, not just the changes just to compare if you change just 1 byte on every chunksize in the origin volume, than the snapshot will contain all data, plus some meta data etc. That is what I talk about here. So how do I retach this volume to a new server? as the only argument acceptable argument by the lvconvert is vg/s1 ? That assumes that vg/s1 is present so how to make it present? Regards Tomas Sent from my iPhone > On 21 Dec 2021, at 14:46, Zdenek Kabelac <zdenek.kabelac@gmail.com> wrote: > > Dne 19. 12. 21 v 17:43 Tomas Dalebjörk napsal(a): >> Hi, >> I am trying to understand how to convert a disk containing snapshot data. >> This is how I tested this: >> 1. locate the snapshot testlv.211218.232255 >> root@debian10:/dev/mapper# lvs >> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert >> home debian10-vg -wi-ao---- 1.00g >> root debian10-vg -wi-ao---- <3.81g >> swap_1 debian10-vg -wi-ao---- 976.00m >> testlv debian10-vg owi-aos--- 100.00m >> testlv.211218.232255 debian10-vg swi-a-s--- 104.00m testlv 1.44 >> root@debian10:/dev/mapper# > > Hi > > It looks like there is some sort of misunderstanding HOW these snapshots with lvm2 do work. > > The snapshot LV does NOT contain whole copy of the origin LV (the one you've take its snapshot) > > So while 'lvs' is presenting you the size of snapshot - it's not a 'regular' volume behind the scene. Instead it's a set of differentiating blocks from your origin stored in a way the came in use. > > So while your snapshot looks it has size 104Mib and origin 100MiB - it means > there is just allocated max possible size of snapshot to store all different chunk + some extra metadata. But ATM only 1.44% of this space is actually used to hold all these block (AKA ~1.5MiB or real storage was used so far) > > There is basically NO WAY to use this 'COW' storage area without its original volume. > > If you want to use such 'snapshot' elsewhere you simply need to *dd* such LV - which is presented (via magic world of DM targets) to user-space to appear like your original LV in the moment of snapshot. > > But the actual 'raw' content stored in disk is in 'internal' form - not really usable outside (although the format is not really a complicated one) > > Note - these so called 'thick' snapshot or 'old' snapshot are now better handled via thin-provisioning - giving much better performance - especially if you plan to keep snapshot for long term or have it in bigger size. > > Hopefully it now makes it way more clear. > > Regards > > Zdenek _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 14:44 ` Tomas Dalebjörk @ 2021-12-21 15:30 ` Zdenek Kabelac 2021-12-21 16:12 ` Tomas Dalebjörk 2021-12-21 19:23 ` Phillip Susi 1 sibling, 1 reply; 11+ messages in thread From: Zdenek Kabelac @ 2021-12-21 15:30 UTC (permalink / raw) To: Tomas Dalebjörk; +Cc: LVM general discussion and development Dne 21. 12. 21 v 15:44 Tomas Dalebjörk napsal(a): > hi > > I think I didn’t explain this clear enough > Allthe lvm data is present in the snapshot that I provision from our backup system > I can guarantee that! > > If I just mount that snapshot from our backup system, it works perfectly well > > so we don’t need the origin volumes in other way than copying back to it > we just need to reanimate it as a cow volume > mentioning that all data has been changed > the cow is just referencing to the origin location, so no problem there > All our data is in the cow volume, not just the changes > > just to compare > if you change just 1 byte on every chunksize in the origin volume, than the snapshot will contain all data, plus some meta data etc. > That is what I talk about here. > So how do I retach this volume to a new server? > > as the only argument acceptable argument by the lvconvert is vg/s1 ? > > That assumes that vg/s1 is present > so how to make it present? Hi As said in my previous post - the 'format' of data stored on COW storage (which is the 'real' meaning of snapshot LV) does NOT in any way resembles the 'normal' LV. So the COW LV could be really ONLY use together with 'snapshot' target. The easiest way how to 'copy' this snapshot to normal LV is like this: lvcreate -L size -n newLV vg dd if=/dev/vg/snapshotLV of=/dev/vg/newLV bs=512K (so with 'DD' you copy data in 'correct' format) You cannot convert snapshot LV to 'normal' LV in any other way then to merge this snapshot LV into your origin LV (so origin is gone) (lvconvert --merge....) You can also 'split' snapshot COW LV and 'reattach' such snapshot to other LV - but this requires rather good knowledge about whole functioning of this snapshotting - so you know what can you do and what can you expect. But I'd likely recommend 'dd'. You cannot use 'splitted' COW LV for i.e. filesystem - as it contains 'mixed' snapshot metadata and snapshot blocks. Old snapshot meaning was - to take 'time consistent' snapshot of LV which then you can use for i.e. taking backup.... Regards Zdenek _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 15:30 ` Zdenek Kabelac @ 2021-12-21 16:12 ` Tomas Dalebjörk 2021-12-22 10:40 ` Zdenek Kabelac 2021-12-28 5:04 ` Stuart D Gathman 0 siblings, 2 replies; 11+ messages in thread From: Tomas Dalebjörk @ 2021-12-21 16:12 UTC (permalink / raw) To: Zdenek Kabelac; +Cc: LVM general discussion and development [-- Attachment #1.1: Type: text/plain, Size: 4422 bytes --] Thanks for explaining all that details about how a snapshot is formatted on the COW device. I already know that part. I am more interested in how the disk containing the COW data can be merged back to an LV volume. The second part only mentioned that it is possible, but not which steps are involved. As documented in the manual. To split a snapshot from its origin (our words detach) one can use: lvconvert --splitsnapshot vg/s1 Right? To reverse that process, according to the manual; one can use: lvconvert -s vg/s1 Right? But as I mentioned before, this requires that the vg/s1 exists as an object in the LVM metadata. What if you are on a new server, that does not have vg/s1? How to create that object or whatever you like to call this on the server? The only way I got it is to use the vgextend lvcreate lvconvert --splitsnapshot And now reattach it, so that the actual merge can happen. The object should exist now, so that the command: lvconvert -s vg/s1 can work Or how can the object vg/s1 be created so that it can be referenced to by the lvconvert command? The disk is formated as a COW device, and contains all of the data. So how hard can it be to just reattach that volume to an empty or existing LV volume on the server? If it works on same server, why can't it work on any other new servers, as the COW device contains ALL the data needed (we make sure it contains all the data) If you want to give it a try, just create a snapshot on a specific device And change all the blocks on the origin, there you are, you now have a cow device containing all data needed. How to move this snapshot device to another server, reattach it to an empty lv volume as a snapshot. lvconvert -s, command requires an argument of an existing snapshot volume name. But there is no snapshot on the new server, so it can't re-attach the volume. So what procedures should be invoked to create just the detached references in LVM, so that the lvconver -s command can work? Regards Tomas Den tis 21 dec. 2021 kl 16:30 skrev Zdenek Kabelac <zdenek.kabelac@gmail.com >: > Dne 21. 12. 21 v 15:44 Tomas Dalebjörk napsal(a): > > hi > > > > I think I didn’t explain this clear enough > > Allthe lvm data is present in the snapshot that I provision from our > backup system > > I can guarantee that! > > > > If I just mount that snapshot from our backup system, it works perfectly > well > > > > so we don’t need the origin volumes in other way than copying back to it > > we just need to reanimate it as a cow volume > > mentioning that all data has been changed > > the cow is just referencing to the origin location, so no problem there > > All our data is in the cow volume, not just the changes > > > > just to compare > > if you change just 1 byte on every chunksize in the origin volume, than > the snapshot will contain all data, plus some meta data etc. > > That is what I talk about here. > > So how do I retach this volume to a new server? > > > > as the only argument acceptable argument by the lvconvert is vg/s1 ? > > > > That assumes that vg/s1 is present > > so how to make it present? > > Hi > > As said in my previous post - the 'format' of data stored on COW storage > (which is the 'real' meaning of snapshot LV) does NOT in any way resembles > the > 'normal' LV. > > So the COW LV could be really ONLY use together with 'snapshot' target. > > The easiest way how to 'copy' this snapshot to normal LV is like this: > > > lvcreate -L size -n newLV vg > > dd if=/dev/vg/snapshotLV of=/dev/vg/newLV bs=512K > > > (so with 'DD' you copy data in 'correct' format) > > You cannot convert snapshot LV to 'normal' LV in any other way then to > merge > this snapshot LV into your origin LV (so origin is gone) > (lvconvert --merge....) > > You can also 'split' snapshot COW LV and 'reattach' such snapshot to > other LV > - but this requires rather good knowledge about whole functioning of this > snapshotting - so you know what can you do and what can you expect. But > I'd > likely recommend 'dd'. > You cannot use 'splitted' COW LV for i.e. filesystem - as it contains > 'mixed' > snapshot metadata and snapshot blocks. > > Old snapshot meaning was - to take 'time consistent' snapshot of LV which > then > you can use for i.e. taking backup.... > > Regards > > Zdenek > [-- Attachment #1.2: Type: text/html, Size: 5399 bytes --] [-- Attachment #2: Type: text/plain, Size: 201 bytes --] _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 16:12 ` Tomas Dalebjörk @ 2021-12-22 10:40 ` Zdenek Kabelac 2021-12-28 5:04 ` Stuart D Gathman 1 sibling, 0 replies; 11+ messages in thread From: Zdenek Kabelac @ 2021-12-22 10:40 UTC (permalink / raw) To: Tomas Dalebjörk; +Cc: LVM general discussion and development Dne 21. 12. 21 v 17:12 Tomas Dalebjörk napsal(a): > Thanks for explaining all that details about how a snapshot is formatted on > the COW device. > I already know that part. > Well your messaging is then somewhat confusing. > I am more interested in how the disk containing the COW data can be merged > back to an LV volume. > The second part only mentioned that it is possible, but not which steps are > involved. > > As documented in the manual. > To split a snapshot from its origin (our words detach) one can use: > lvconvert --splitsnapshot vg/s1 > Right? > > To reverse that process, according to the manual; one can use: > lvconvert -s vg/s1 > Right? > > But as I mentioned before, this requires that the vg/s1 exists as an object in > the LVM metadata. > What if you are on a new server, that does not have vg/s1? > How to create that object or whatever you like to call this on the server? > The only way I got it is to use the > vgextend > lvcreate > lvconvert --splitsnapshot > > And now reattach it, so that the actual merge can happen. > The object should exist now, so that the command: lvconvert -s vg/s1 can work > There is no problem with reattaching existing COW to any other LV - it's plain 'lvconvert -s -Zn -c xxx vg/origin vg/snapcow' this will rejoin former origin with former COW volume and avoids 'zeroing' metadata stored on COW. But to have this usable - there must have been NO write access to the origin between the moment your snapshot was 'split' and 'reattached' and it's documented in 'man lvconvert' > Or how can the object vg/s1 be created so that it can be referenced to by the > lvconvert command? > The disk is formated as a COW device, and contains all of the data. > So how hard can it be to just reattach that volume to an empty or existing LV > volume on the server? There is no problem with this - however your provided case just shown you have very small amount of data stored in snapshot. So while snapshot has large size - it's been mostly unoccupied with data - so any merge into 'clean' origin is kind of meaningless in this particular case. > If it works on same server, why can't it work on any other new servers, as the > COW device contains ALL the data needed (we make sure it contains all the data) > > If you want to give it a try, just create a snapshot on a specific device > And change all the blocks on the origin, there you are, you now have a cow > device containing all data needed. > How to move this snapshot device to another server, reattach it to an empty lv > volume as a snapshot. I'm aware how this works - being lvm2 developer... I'm just trying to explain there is better and simpler way how to reach your goal. Simply take a 'time consistent' snapshot of your origin. (So it does not change while you take a 'dd' copy) Take a 'DD' copy of such stable snapshot to a new LV. And restore this 'copied' LV on your 2nd. server. I simply don't follow why would you want to complicate all of this with snapshot merging - makes no sense and just cause a massive slowdown.... Regards Zdenek _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 16:12 ` Tomas Dalebjörk 2021-12-22 10:40 ` Zdenek Kabelac @ 2021-12-28 5:04 ` Stuart D Gathman 2021-12-28 6:40 ` Tomas Dalebjörk 1 sibling, 1 reply; 11+ messages in thread From: Stuart D Gathman @ 2021-12-28 5:04 UTC (permalink / raw) To: LVM general discussion and development; +Cc: Zdenek Kabelac > If you want to give it a try, just create a snapshot on a specific device > And change all the blocks on the origin, there you are, you now have a cow > device containing all data needed. > How to move this snapshot device to another server, reattach it to an empty > lv volume as a snapshot. > lvconvert -s, command requires an argument of an existing snapshot volume > name. > But there is no snapshot on the new server, so it can't re-attach the > volume. > So what procedures should be invoked to create just the detached references > in LVM, so that the lvconver -s command can work? Just copy the snapshot to another server, by whatever method you would use to copy the COW and Data volumes (I prefer partclone for supported filesystems). No need for lvconvert. You are trying WAY WAY too hard. Are you by any chance trying to create an incremental backup system based on lvm snapshot COW? If so, say so. _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-28 5:04 ` Stuart D Gathman @ 2021-12-28 6:40 ` Tomas Dalebjörk 2021-12-29 13:35 ` Stuart D Gathman 0 siblings, 1 reply; 11+ messages in thread From: Tomas Dalebjörk @ 2021-12-28 6:40 UTC (permalink / raw) To: LVM general discussion and development; +Cc: Zdenek Kabelac Hi Yes, it is an incremental backup based of the cow device no worries about the backup, that works fine, and now by just creating a lv snapshot on it, works too sent you an example before eg # extend vg with new bu disk vgextend xxx newdisk # create lv structure on disk lvcreate -s -L wholedisksize -n s1 xxx/lv newdisk # merge has to be started offline? lvchange -a n xxx/lv # start merge lvconvert -merge -b xxx/s1 # online the lv lvchange -a y xxx/s1 The backup is provisioned through the disk, and makes the data available immediately But, I guess I can skip some steps? Restoring data in just a matter of minutes, regardless of size, or will say, making the data available anywhere in just a few minutes, regardless of size Also working with writes, without destroying the backup are there a way to get a signal when a merge has completed? or do I have to implement a monitor dmsetup xxx or lvs to check when merge has completed Reference Tomas Sent from my iPhone > On 28 Dec 2021, at 06:32, Stuart D Gathman <stuart@gathman.org> wrote: > > >> >> If you want to give it a try, just create a snapshot on a specific device >> And change all the blocks on the origin, there you are, you now have a cow >> device containing all data needed. >> How to move this snapshot device to another server, reattach it to an empty >> lv volume as a snapshot. >> lvconvert -s, command requires an argument of an existing snapshot volume >> name. >> But there is no snapshot on the new server, so it can't re-attach the >> volume. >> So what procedures should be invoked to create just the detached references >> in LVM, so that the lvconver -s command can work? > > Just copy the snapshot to another server, by whatever method you would > use to copy the COW and Data volumes (I prefer partclone for supported > filesystems). No need for lvconvert. You are trying WAY WAY too hard. > Are you by any chance trying to create an incremental backup system > based on lvm snapshot COW? If so, say so. > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-28 6:40 ` Tomas Dalebjörk @ 2021-12-29 13:35 ` Stuart D Gathman 0 siblings, 0 replies; 11+ messages in thread From: Stuart D Gathman @ 2021-12-29 13:35 UTC (permalink / raw) To: LVM general discussion and development; +Cc: Zdenek Kabelac [-- Attachment #1: Type: text/plain, Size: 886 bytes --] On Tue, 28 Dec 2021, Tomas Dalebjörk wrote: > Yes, it is an incremental backup based of the cow device I've used such a COW based backup (can't remember the name just now, currently using DRBD and rsync for incremental mirrors). The way it worked was to read and interpret the raw COW device itself and send blocks over the wire - writing directly to a volume on the remote end. It did not try to patch up metadata and use LVM to merge. You need an intimate knowledge of COW internals for either approach - BUT the read-only approach (with plain writes at the other end) is MUCH safer (not going to trash metadata at either end) and just as efficient on the wire. I've also used a block device rsync, that read every block on both sides and compared hashes - but that is obviously a lot more disk io that using the COW where LVM is already tracking changed blocks. [-- Attachment #2: Type: text/plain, Size: 201 bytes --] _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? 2021-12-21 14:44 ` Tomas Dalebjörk 2021-12-21 15:30 ` Zdenek Kabelac @ 2021-12-21 19:23 ` Phillip Susi 1 sibling, 0 replies; 11+ messages in thread From: Phillip Susi @ 2021-12-21 19:23 UTC (permalink / raw) To: LVM general discussion and development; +Cc: örk, Zdenek Kabelac Tomas Dalebjörk <tomas.dalebjork@gmail.com> writes: > hi > > I think I didn’t explain this clear enough > Allthe lvm data is present in the snapshot that I provision from our backup system > I can guarantee that! > > If I just mount that snapshot from our backup system, it works perfectly well > > so we don’t need the origin volumes in other way than copying back to it > we just need to reanimate it as a cow volume > mentioning that all data has been changed > the cow is just referencing to the origin location, so no problem there > All our data is in the cow volume, not just the changes Ok, so you have thrown out the snapshot relationship when you made your backup, and backed up both the origin and the snapshot as two separate backups? Thus your backup requires much more space than the original system did since all of the common data has now been duplicated in both backups. Now it seems you want to restore both backups, but NOT pay the storage penalty for duplicating the common parts. I don't think there is really a good way of doing that. I'd say that you can restore the origin volume, then create a snapshot of that, then run some sort of binary diff between the origin and the backup of the snapshot and write that diff to the new snapshot. _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2021-12-29 14:32 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-12-19 16:43 [linux-lvm] how to convert a disk containing a snapshot to a snapshot lv? Tomas Dalebjörk 2021-12-20 14:51 ` Phillip Susi 2021-12-21 13:46 ` Zdenek Kabelac 2021-12-21 14:44 ` Tomas Dalebjörk 2021-12-21 15:30 ` Zdenek Kabelac 2021-12-21 16:12 ` Tomas Dalebjörk 2021-12-22 10:40 ` Zdenek Kabelac 2021-12-28 5:04 ` Stuart D Gathman 2021-12-28 6:40 ` Tomas Dalebjörk 2021-12-29 13:35 ` Stuart D Gathman 2021-12-21 19:23 ` Phillip Susi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).