[linux-lvm] The feasibility of implementing an alternative snapshot approach

[linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zhiyong Ye]

Dear all,

The current standard lv implementation of snapshots is COW 
(Copy-on-write), which creates snapshots very quickly. However, the 
first write performance of the original lv will be poor after creating a 
snapshot because of COW. Moreover, the more snapshots there are, the 
worse the performance of the original lv will be.

I tested the random read/write performance when the original lv was 
created with different number of snapshots. The data is shown below:
Number of snapshots  Randread(iops)  Randwrite(iops)
         0                21989           22034
         1                10048           10041
         2                6770            6773
         3                5375            5378

There are scenarios where the performance of the original lv is more 
demanding, and the speed of snapshot creation is not as strong a 
requirement. Because it is the original lv that will actually be used, 
and the snapshot is only a secondary function. Therefore snapshots using 
the COW approach will not meet the needs of this scenario.

Therefore, is it feasible to implement another way of taking snapshots? 
Let's say the first snapshot is created as a full snapshot, and all 
subsequent snapshots are based on incremental data from the previous 
snapshot.

Regards,

Zhiyong Ye

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zdenek Kabelac]

Dne 04. 01. 23 v 9:00 Zhiyong Ye napsal(a):
> Dear all,
> 
> The current standard lv implementation of snapshots is COW (Copy-on-write), 
> which creates snapshots very quickly. However, the first write performance of 
> the original lv will be poor after creating a snapshot because of COW. 
> Moreover, the more snapshots there are, the worse the performance of the 
> original lv will be.
> 
> I tested the random read/write performance when the original lv was created 
> with different number of snapshots. The data is shown below:
> Number of snapshots  Randread(iops)  Randwrite(iops)
>          0                21989           22034
>          1                10048           10041
>          2                6770            6773
>          3                5375            5378
> 
> There are scenarios where the performance of the original lv is more 
> demanding, and the speed of snapshot creation is not as strong a requirement. 
> Because it is the original lv that will actually be used, and the snapshot is 
> only a secondary function. Therefore snapshots using the COW approach will not 
> meet the needs of this scenario.
> 
> Therefore, is it feasible to implement another way of taking snapshots? Let's 
> say the first snapshot is created as a full snapshot, and all subsequent 
> snapshots are based on incremental data from the previous snapshot.

Hi

Have you played with thin provisioning - as that's the answer to the slow 
snapshots.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zhiyong Ye]

Hi Zdenek,

Thank you for your reply.

Snapshots of thinlv are indeed more efficient compared to standard lv, 
this is because data blocks can be shared between snapshot and original 
thinlv. But there is also a performance loss after thinlv creates a 
snapshot. This is because the first write to the snapshotted thinlv 
requires not only allocating a new chunk but also copying the old data.

Here are some performance data and a discussion of the thinlv snapshot:

https://listman.redhat.com/archives/linux-lvm/2022-June/026200.html

Regards,

Zhiyong

在 1/4/23 10:09 PM, Zdenek Kabelac 写道:
> Dne 04. 01. 23 v 9:00 Zhiyong Ye napsal(a):
>> Dear all,
>>
>> The current standard lv implementation of snapshots is COW 
>> (Copy-on-write), which creates snapshots very quickly. However, the 
>> first write performance of the original lv will be poor after creating 
>> a snapshot because of COW. Moreover, the more snapshots there are, the 
>> worse the performance of the original lv will be.
>>
>> I tested the random read/write performance when the original lv was 
>> created with different number of snapshots. The data is shown below:
>> Number of snapshots  Randread(iops)  Randwrite(iops)
>>          0                21989           22034
>>          1                10048           10041
>>          2                6770            6773
>>          3                5375            5378
>>
>> There are scenarios where the performance of the original lv is more 
>> demanding, and the speed of snapshot creation is not as strong a 
>> requirement. Because it is the original lv that will actually be used, 
>> and the snapshot is only a secondary function. Therefore snapshots 
>> using the COW approach will not meet the needs of this scenario.
>>
>> Therefore, is it feasible to implement another way of taking 
>> snapshots? Let's say the first snapshot is created as a full snapshot, 
>> and all subsequent snapshots are based on incremental data from the 
>> previous snapshot.
> 
> Hi
> 
> Have you played with thin provisioning - as that's the answer to the 
> slow snapshots.
> 
> Regards
> 
> Zdenek
> 

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zdenek Kabelac]

Dne 04. 01. 23 v 17:12 Zhiyong Ye napsal(a):
> Hi Zdenek,
> 
> Thank you for your reply.
> 
> Snapshots of thinlv are indeed more efficient compared to standard lv, this is 
> because data blocks can be shared between snapshot and original thinlv. But 
> there is also a performance loss after thinlv creates a snapshot. This is 
> because the first write to the snapshotted thinlv requires not only allocating 
> a new chunk but also copying the old data.
> 
> Here are some performance data and a discussion of the thinlv snapshot:
> 
> https://listman.redhat.com/archives/linux-lvm/2022-June/026200.html
>

Well that's our current  'state-of-the-art' solution.

Make sure you are using latest kernels for your performance testing - there 
have been several improvements around the locking (6+ kernels) - but if this 
still not good enough for your case you might need to seek for some other 
solutions (although would be nice to know who handles this task better).

Definitely the old 'thick-snapshot' is mostly in maintenance phase and it's 
usability (and its design) is limited for some short living temporary 
snapshoting (i.e. you are making backup and after completing your backup of 
the filesystem you remove your temporary snapshot - it's been never designed 
to be used for multi-level multi-GiB snapshots - this will not fly...

When you use thin snapshots - make sure your metadata LV is located on your 
fast device and you use best fitting chunksize.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zhiyong Ye]

Hi Zdenek,

Thank you for your detailed answer.

For the thin snapshot I will use the latest version of kernel and lvm 
for further testing. I want to use both snapshot methods (thin and 
thick) in the production environment. But if the thick snapshot is only 
still in the maintenance phase, then for thick lv I have to see if there 
is any other way to accomplish the snapshot function.

I use lvm mainly for virtualized environments. Each lv acts as a block 
device of the virtual machine. So I also consider using qemu's own 
snapshot feature. When qemu creates a snapshot, the original image used 
by the virtual machine becomes read-only, and all write changes are 
stored in the new snapshot. But currently qemu's snapshots only support 
files, not block devices.

Regards

Zhiyong

On 1/6/23 9:42 PM, Zdenek Kabelac wrote:
> Dne 04. 01. 23 v 17:12 Zhiyong Ye napsal(a):
>> Hi Zdenek,
>>
>> Thank you for your reply.
>>
>> Snapshots of thinlv are indeed more efficient compared to standard lv, 
>> this is because data blocks can be shared between snapshot and 
>> original thinlv. But there is also a performance loss after thinlv 
>> creates a snapshot. This is because the first write to the snapshotted 
>> thinlv requires not only allocating a new chunk but also copying the 
>> old data.
>>
>> Here are some performance data and a discussion of the thinlv snapshot:
>>
>> https://listman.redhat.com/archives/linux-lvm/2022-June/026200.html
>>
> 
> Well that's our current  'state-of-the-art' solution.
> 
> Make sure you are using latest kernels for your performance testing - 
> there have been several improvements around the locking (6+ kernels) - 
> but if this still not good enough for your case you might need to seek 
> for some other solutions (although would be nice to know who handles 
> this task better).
> 
> Definitely the old 'thick-snapshot' is mostly in maintenance phase and 
> it's usability (and its design) is limited for some short living 
> temporary snapshoting (i.e. you are making backup and after completing 
> your backup of the filesystem you remove your temporary snapshot - it's 
> been never designed to be used for multi-level multi-GiB snapshots - 
> this will not fly...
> 
> When you use thin snapshots - make sure your metadata LV is located on 
> your fast device and you use best fitting chunksize.
> 
> Regards
> 
> Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] The feasibility of implementing an alternative snapshot approach

[Zdenek Kabelac]

Dne 09. 01. 23 v 7:21 Zhiyong Ye napsal(a):
> Hi Zdenek,
> 
> Thank you for your detailed answer.
> 
> For the thin snapshot I will use the latest version of kernel and lvm for 
> further testing. I want to use both snapshot methods (thin and thick) in the 
> production environment. But if the thick snapshot is only still in the 
> maintenance phase, then for thick lv I have to see if there is any other way 
> to accomplish the snapshot function.

FYI - there are still some delays with up-streaming of the latest improvement 
patches - so stay tuned for further speedup gains & IO throughput with thin 
provisioning)

By the maintenance phase for old thick snapshot I mean - the development of 
the existing thick snapshot target is basically done - the format is very 
ancient and cannot be changed without major rewrite of the whole snapshot 
target as such - and that's what we've made with newly introduced 
thin-provisioning target which addressed many shortcomings of the old 
dm-snapshot target.

> I use lvm mainly for virtualized environments. Each lv acts as a block device 
> of the virtual machine. So I also consider using qemu's own snapshot feature. 
> When qemu creates a snapshot, the original image used by the virtual machine 
> becomes read-only, and all write changes are stored in the new snapshot. But 
> currently qemu's snapshots only support files, not block devices.

Depending on the use-case it might matter to pick the best fitting chunk-size.
i.e. if the changes are 'localized' in the filesystem areas to match thin-pool 
chunks (also selection of the filesystem itself might be part of equation 
here) - even if you use snapshots a lot, you may eventually get better result 
with bigger chunks like 128k or even 256k size instead of default 64K.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] [External] Re: The feasibility of implementing an alternative snapshot approach

[Zhiyong Ye]

Hi Zdenek,

Thank you for your patience and explanations. I learned a lot from our 
discussions and thank you again for your help.

Regards

Zhiyong

On 1/10/23 6:18 AM, Zdenek Kabelac wrote:
> Dne 09. 01. 23 v 7:21 Zhiyong Ye napsal(a):
>> Hi Zdenek,
>>
>> Thank you for your detailed answer.
>>
>> For the thin snapshot I will use the latest version of kernel and lvm 
>> for further testing. I want to use both snapshot methods (thin and 
>> thick) in the production environment. But if the thick snapshot is 
>> only still in the maintenance phase, then for thick lv I have to see 
>> if there is any other way to accomplish the snapshot function.
> 
> FYI - there are still some delays with up-streaming of the latest 
> improvement patches - so stay tuned for further speedup gains & IO 
> throughput with thin provisioning)
> 
> By the maintenance phase for old thick snapshot I mean - the development 
> of the existing thick snapshot target is basically done - the format is 
> very ancient and cannot be changed without major rewrite of the whole 
> snapshot target as such - and that's what we've made with newly 
> introduced thin-provisioning target which addressed many shortcomings of 
> the old dm-snapshot target.
> 
>> I use lvm mainly for virtualized environments. Each lv acts as a block 
>> device of the virtual machine. So I also consider using qemu's own 
>> snapshot feature. When qemu creates a snapshot, the original image 
>> used by the virtual machine becomes read-only, and all write changes 
>> are stored in the new snapshot. But currently qemu's snapshots only 
>> support files, not block devices.
> 
> Depending on the use-case it might matter to pick the best fitting 
> chunk-size.
> i.e. if the changes are 'localized' in the filesystem areas to match 
> thin-pool chunks (also selection of the filesystem itself might be part 
> of equation here) - even if you use snapshots a lot, you may eventually 
> get better result with bigger chunks like 128k or even 256k size instead 
> of default 64K.
> 
> Regards
> 
> Zdenek
> 

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/