[linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Sreyan Chakravarty]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

I am looking into thin snapshots since the older COW snapshots delay my
boot enormously.

My normal root fs is on a normal "non-thin" volume.

So to make a snapshot I have boot into a live cd environment then set my
root fs volume to read-only and then set it to inactive.

Commands:

       sudo lvchange -pr vgfedora/fedora
       sudo lvchange -an vgfedora/fedora

Now I create the thin snapshot using lvcreate, after that I need to restore
my root LVS to its original state:

     sudo lvchange -ay vgfedora/fedora

      sudo lvchange -prw vgfedora/fedora

This fails with:

            Command on LV vgfedora/fedora uses options that are invalid
with LV parameters: lv_is_external_origin.


I mean what is the point of creating a snapshot if I can't change my
original volume ?

Is there some sort of resolution ?

-- 
Regards,
Sreyan Chakravarty

[-- Attachment #2: Type: text/html, Size: 1340 bytes --]

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Gionatan Danti]

Il 2020-11-21 04:10 Sreyan Chakravarty ha scritto:
> I mean what is the point of creating a snapshot if I can't change my
> original volume ?
> 
> Is there some sort of resolution ?

External thin snapshot are useful to share a common, read-only base (ie: 
a "gold-master" image) with different writable thin lvm overlay volumes. 
They can not be merged into external origin; otherwise, a single merge 
operation would invalidate *all* other thin snapshot relying on the same 
origin.

Classical LVM snapshots are good for temporary, short-lived snapshots - 
basically taken for backup purpose only (and immediately removed after 
the backup completed). You should not use them for long-lived snapshots 
(otherwise you can see much lower performance and delayed boot).

If you want to have long-lived snapshot you should use LVM thin 
snapshots. For root volumes, you have two choices:
- use a thin lvm volume for root, which is a supported use case (but it 
will probably require a system reinstallation);
- continue using your classical LVM as a immutable base and use a thin 
lvm with external snapshot to store all new writes.

Regards.


-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Bryn M. Reeves]

On Sat, Nov 21, 2020 at 08:40:44AM +0530, Sreyan Chakravarty wrote:
> I am looking into thin snapshots since the older COW snapshots delay my
> boot enormously.
> 
> My normal root fs is on a normal "non-thin" volume.
> 
> So to make a snapshot I have boot into a live cd environment then set my
> root fs volume to read-only and then set it to inactive.

OK - I understand what's going on in your environment now, thanks!

Unfortunately it's not possible to have a writable external origin when
using device-mapper thin provisioned snapshots. To be able to write to
the origin while snapshots exist the origin device must also be a thin
provisioned logical volume.

This is explained in more detail in the kernel documentation for the
thin provisioning targets.

Documentation/admin-guide/device-mapper/thin-provisioning.rst:

    External snapshots
    ------------------
    
    You can use an external **read only** device as an origin for a
    thinly-provisioned volume.  Any read to an unprovisioned area of the
    thin device will be passed through to the origin.  Writes trigger
    the allocation of new blocks as usual.
    
    One use case for this is VM hosts that want to run guests on
    thinly-provisioned volumes but have the base image on another device
    (possibly shared between many VMs).
    
    You must not write to the origin device if you use this technique!
    Of course, you may write to the thin device and take internal snapshots
    of the thin volume.

This allows a few niche use cases (like the VM example given), but it's
not the conventional way of using snapshots with thinp and it does
restrict what you can do.

This means that to use thinp snapshots most effectively you must set the
system up with a thin pool from the start (e.g. using the distro's
installer to set up the VG).

>             Command on LV vgfedora/fedora uses options that are invalid
> with LV parameters: lv_is_external_origin.

This is correct: currently you cannot make the origin writable since it
is an external snapshot.

There is some work going on at the moment that would make device-mapper
type features more flexible and available in other device types, but
with the features provided by current tools and the thinp kernel
support you need to use a thinp device for the snapshot origin too.
 
> Is there some sort of resolution ?

It means re-installing but if the system is set up to use a thin pool
and thin provisioned logical volumes from the start then you can use
snapshots without any of the limitations that you've bumped into with
external origin devices.

Regards,
Bryn.

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Sreyan Chakravarty]

[-- Attachment #1: Type: text/plain, Size: 2549 bytes --]

On Mon, Nov 23, 2020 at 6:34 PM Bryn M. Reeves <bmr@redhat.com> wrote:

> OK - I understand what's going on in your environment now, thanks!
>
> Unfortunately it's not possible to have a writable external origin when
> using device-mapper thin provisioned snapshots. To be able to write to
> the origin while snapshots exist the origin device must also be a thin
> provisioned logical volume.
>
> This is explained in more detail in the kernel documentation for the
> thin provisioning targets.
>
> Documentation/admin-guide/device-mapper/thin-provisioning.rst:
>
>     External snapshots
>     ------------------
>
>     You can use an external **read only** device as an origin for a
>     thinly-provisioned volume.  Any read to an unprovisioned area of the
>     thin device will be passed through to the origin.  Writes trigger
>     the allocation of new blocks as usual.
>
>     One use case for this is VM hosts that want to run guests on
>     thinly-provisioned volumes but have the base image on another device
>     (possibly shared between many VMs).
>
>     You must not write to the origin device if you use this technique!
>     Of course, you may write to the thin device and take internal snapshots
>     of the thin volume.
>
> This allows a few niche use cases (like the VM example given), but it's
> not the conventional way of using snapshots with thinp and it does
> restrict what you can do.
>
> This means that to use thinp snapshots most effectively you must set the
> system up with a thin pool from the start (e.g. using the distro's
> installer to set up the VG).
>
> >             Command on LV vgfedora/fedora uses options that are invalid
> > with LV parameters: lv_is_external_origin.
>
> This is correct: currently you cannot make the origin writable since it
> is an external snapshot.
>
> There is some work going on at the moment that would make device-mapper
> type features more flexible and available in other device types, but
> with the features provided by current tools and the thinp kernel
> support you need to use a thinp device for the snapshot origin too.
>
> > Is there some sort of resolution ?
>
> It means re-installing but if the system is set up to use a thin pool
> and thin provisioned logical volumes from the start then you can use
> snapshots without any of the limitations that you've bumped into with
> external origin devices.


Do I have to reinstall my system for thin snapshots ?

Can't I just clone my filesystem and then create a thin pool ?

>
>


-- 
Regards,
Sreyan Chakravarty

[-- Attachment #2: Type: text/html, Size: 3375 bytes --]

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Sreyan Chakravarty]

[-- Attachment #1: Type: text/plain, Size: 342 bytes --]

On Tue, Nov 24, 2020 at 5:29 PM Sreyan Chakravarty <sreyan32@gmail.com>
wrote:

> Do I have to reinstall my system for thin snapshots ?
>
> Can't I just clone my filesystem and then create a thin pool ?
>
>>
>
>
Used partclone to restore my backup.

Did not have re-install but did have to delete old volumes.
-- 
Regards,
Sreyan Chakravarty

[-- Attachment #2: Type: text/html, Size: 1005 bytes --]

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Bryn M. Reeves]

On Tue, Nov 24, 2020 at 05:29:36PM +0530, Sreyan Chakravarty wrote:
> On Mon, Nov 23, 2020 at 6:34 PM Bryn M. Reeves <bmr@redhat.com> wrote:
> > It means re-installing but if the system is set up to use a thin pool
> > and thin provisioned logical volumes from the start then you can use
> > snapshots without any of the limitations that you've bumped into with
> > external origin devices.
> 
> 
> Do I have to reinstall my system for thin snapshots ?
> 
> Can't I just clone my filesystem and then create a thin pool ?

You can, it's just typically easier for most users to re-install than to
execute the necessary steps to convert the system after the fact.

If you have sufficient space then you can just create a thin pool, copy
your existing file systems into it and then fix up the fstab, grub boot
entries, initramfs and any other configuration that may still reference
the old device names. Once you're finished you can remove the original
volumes and reclaim the space for use by the thin pool.

Regards,
Bryn.

Re: [linux-lvm] What is the use of thin snapshots if the external origin cannot be set to writable ?

[Sreyan Chakravarty]

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

On Wed, Nov 25, 2020 at 8:50 PM Bryn M. Reeves <bmr@redhat.com> wrote:

> You can, it's just typically easier for most users to re-install than to
> execute the necessary steps to convert the system after the fact.
>
> If you have sufficient space then you can just create a thin pool, copy
> your existing file systems into it and then fix up the fstab, grub boot
> entries, initramfs and any other configuration that may still reference
> the old device names. Once you're finished you can remove the original
> volumes and reclaim the space for use by the thin pool.
>

Yes, I have migrated to thin pools successfully. Thanks.


-- 
Regards,
Sreyan Chakravarty

[-- Attachment #2: Type: text/html, Size: 1127 bytes --]