linux-lvm.redhat.com archive mirror
 help / color / mirror / Atom feed
From: Eric Toombs <ewtoombs@uwaterloo.ca>
To: Zdenek Kabelac <zkabelac@redhat.com>,
	LVM general discussion and development <linux-lvm@redhat.com>
Subject: Re: [linux-lvm] faster snapshot creation?
Date: Tue, 25 Feb 2020 16:35:01 -0500	[thread overview]
Message-ID: <bfd3824a-f360-b4ab-9a08-6a51c8c37a94@uwaterloo.ca> (raw)
In-Reply-To: <bcd5bb9d-9ea5-69ba-f041-7b0422b48405@redhat.com>

I tried thin pools and got the same snapshot creation time, about .45s.

I also tried creating the entire vg in memory with losetup and that
didn't change the creation time either.

On 2020-02-25 04:38, Zdenek Kabelac wrote:
> Dne 22. 02. 20 v 12:58 Eric Toombs napsal(a):
>> Snapshot creation is already pretty fast:
>>
>>> $ time sudo lvcreate --size 512M --snapshot --name snap
>>> /dev/testdbs/template
>>> �� Logical volume "snap" created.
>>> 0.03user 0.05system 0:00.46elapsed 18%CPU (0avgtext+0avgdata
>>> 28916maxresident)k
>>> 768inputs+9828outputs (0major+6315minor)pagefaults 0swaps
>>
>> That's about half a second in real time. But I have a scenario that
>> would benefit from it being even faster. I'm doing many small unit tests
>> starting from a template filesystem. I do the snapshot, run the unit
>> test on the snapshot, then delete the snapshot afterwards using
>> lvremove. Each unit test, though, takes much less than a second to run
>> (often on the order of 10ms), so most of the time is being spent making
>> these snapshots.
>>
>> So, is there a sort of "dumber" way of making these snapshots, maybe by
>> changing the allocation algorithm or something?
> 
> Hi
> 
> IMHO - what takes most of the time are these couple things:
> 
> Each command has 'non-trivial' time overhead on its startup (scanning
> you system with devices and validating everything)
> 
> For old snapshots -� COW are needs to be 'created' & 'zeroed' as
> separate LV.
> 
> Then you need to 'flush' all existing IO on origin device (so it's in
> the consistent states - i.e. the filesystem synchronizes all it's
> content in its metadata) - this all takes some measurable amount of time.
> 
> You can 'prepare' empty zeroed LV ahead of time and then use
> 'lvconvert' to attach snapshot (with -Zn)� - this should speed-up
> attachment
> of snapshot. For the 'second' point you could likely issue 'sync' ahead
> of time so most buffers will be flushed (if there is no big IO traffic).
> 
> Saying all this - why are you using old snapshot when you are targeting
> for performance ??
> 
> You really should consider usage of thin-pool - where you could chain a
> long series of snapshot without having a dramatic performance
> degradation of the whole IO throughput� - old snapshot are really meant
> to be used only if you want to take i.e. backup of a filesystem and you
> need some 'consistent' point in time - for everything else you should be
> using thin-pools nowdays...
> 
> Regards
> 
> Zdenek
> 

  reply	other threads:[~2020-02-25 21:34 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-22 11:58 [linux-lvm] faster snapshot creation? Eric Toombs
2020-02-25  9:31 ` Gionatan Danti
2020-02-25  9:38 ` Zdenek Kabelac
2020-02-25 21:35   ` Eric Toombs [this message]
2020-02-26 10:01     ` Zdenek Kabelac
2020-02-26 14:03       ` Douglas Paul
2020-02-26  0:38 ` Stuart D. Gathman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bfd3824a-f360-b4ab-9a08-6a51c8c37a94@uwaterloo.ca \
    --to=ewtoombs@uwaterloo.ca \
    --cc=linux-lvm@redhat.com \
    --cc=zkabelac@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).