From: Eric Toombs <ewtoombs@uwaterloo.ca>
To: Zdenek Kabelac <zkabelac@redhat.com>,
LVM general discussion and development <linux-lvm@redhat.com>
Subject: Re: [linux-lvm] faster snapshot creation?
Date: Tue, 25 Feb 2020 16:35:01 -0500 [thread overview]
Message-ID: <bfd3824a-f360-b4ab-9a08-6a51c8c37a94@uwaterloo.ca> (raw)
In-Reply-To: <bcd5bb9d-9ea5-69ba-f041-7b0422b48405@redhat.com>
I tried thin pools and got the same snapshot creation time, about .45s.
I also tried creating the entire vg in memory with losetup and that
didn't change the creation time either.
On 2020-02-25 04:38, Zdenek Kabelac wrote:
> Dne 22. 02. 20 v 12:58 Eric Toombs napsal(a):
>> Snapshot creation is already pretty fast:
>>
>>> $ time sudo lvcreate --size 512M --snapshot --name snap
>>> /dev/testdbs/template
>>> �� Logical volume "snap" created.
>>> 0.03user 0.05system 0:00.46elapsed 18%CPU (0avgtext+0avgdata
>>> 28916maxresident)k
>>> 768inputs+9828outputs (0major+6315minor)pagefaults 0swaps
>>
>> That's about half a second in real time. But I have a scenario that
>> would benefit from it being even faster. I'm doing many small unit tests
>> starting from a template filesystem. I do the snapshot, run the unit
>> test on the snapshot, then delete the snapshot afterwards using
>> lvremove. Each unit test, though, takes much less than a second to run
>> (often on the order of 10ms), so most of the time is being spent making
>> these snapshots.
>>
>> So, is there a sort of "dumber" way of making these snapshots, maybe by
>> changing the allocation algorithm or something?
>
> Hi
>
> IMHO - what takes most of the time are these couple things:
>
> Each command has 'non-trivial' time overhead on its startup (scanning
> you system with devices and validating everything)
>
> For old snapshots -� COW are needs to be 'created' & 'zeroed' as
> separate LV.
>
> Then you need to 'flush' all existing IO on origin device (so it's in
> the consistent states - i.e. the filesystem synchronizes all it's
> content in its metadata) - this all takes some measurable amount of time.
>
> You can 'prepare' empty zeroed LV ahead of time and then use
> 'lvconvert' to attach snapshot (with -Zn)� - this should speed-up
> attachment
> of snapshot. For the 'second' point you could likely issue 'sync' ahead
> of time so most buffers will be flushed (if there is no big IO traffic).
>
> Saying all this - why are you using old snapshot when you are targeting
> for performance ??
>
> You really should consider usage of thin-pool - where you could chain a
> long series of snapshot without having a dramatic performance
> degradation of the whole IO throughput� - old snapshot are really meant
> to be used only if you want to take i.e. backup of a filesystem and you
> need some 'consistent' point in time - for everything else you should be
> using thin-pools nowdays...
>
> Regards
>
> Zdenek
>
next prev parent reply other threads:[~2020-02-25 21:34 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-22 11:58 [linux-lvm] faster snapshot creation? Eric Toombs
2020-02-25 9:31 ` Gionatan Danti
2020-02-25 9:38 ` Zdenek Kabelac
2020-02-25 21:35 ` Eric Toombs [this message]
2020-02-26 10:01 ` Zdenek Kabelac
2020-02-26 14:03 ` Douglas Paul
2020-02-26 0:38 ` Stuart D. Gathman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bfd3824a-f360-b4ab-9a08-6a51c8c37a94@uwaterloo.ca \
--to=ewtoombs@uwaterloo.ca \
--cc=linux-lvm@redhat.com \
--cc=zkabelac@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).