Re: [linux-lvm] Thin pool performance when allocating lots of blocks

From: Zdenek Kabelac <zdenek.kabelac@gmail.com>
To: Demi Marie Obenour <demi@invisiblethingslab.com>,
	LVM general discussion and development <linux-lvm@redhat.com>,
	device-mapper development <dm-devel@redhat.com>
Cc: "Frédéric Pierret" <frederic.pierret@qubes-os.org>,
	"Marek Marczykowski-Górecki" <marmarek@invisiblethingslab.com>
Subject: Re: [linux-lvm] Thin pool performance when allocating lots of blocks
Date: Tue, 8 Feb 2022 22:30:25 +0100	[thread overview]
Message-ID: <fb00c57d-cf11-8632-bcdd-6b52826f212f@gmail.com> (raw)
In-Reply-To: <733f87f3-5ed9-b266-b951-4526f872bad1@invisiblethingslab.com>

Dne 08. 02. 22 v 22:02 Demi Marie Obenour napsal(a):
> On 2/8/22 15:37, Zdenek Kabelac wrote:
>> Dne 08. 02. 22 v 20:00 Demi Marie Obenour napsal(a):
>>> Are thin volumes (which start as snapshots of a blank volume) efficient
>>> for building virtual machine images?  Given the nature of this workload
>>> (writing to lots of new, possibly-small files, then copying data from
>>> them to a huge disk image), I expect that this will cause sharing to be
>>> broken many, many times, and the kernel code that breaks sharing appears
>>> to be rather heavyweight.  Furthermore, since zeroing is enabled, this
>>> might cause substantial write amplification.  Turning zeroing off is not
>>> an option for security reasons.
>>>
>>> Is there a way to determine if breaking sharing is the cause of
>>> performance problems?  If it is, are there any better solutions?
>>
>> Hi
>>
>> Usually the smaller the thin chunks size is the smaller the problem gets.
>> With current released version of thin-provisioning minimal chunk size is
>> 64KiB. So you can't use smaller value to further reduce this impact.
>>
>> Note - even if you do a lot of tiny 4KiB writes  - only the 'first' such write
>> into 64K area breaks sharing all following writes to same location no longer
>> have this penalty (also zeroing with 64K is less impactful...)
>>
>> But it's clear thin-provisioning comes with some price - so if it's not good
>> enough from time constrains some other solutions might need to be explored.
>> (i.e. caching, better hw, splitting  FS into multiple partitions with
>> 'read-only sections,....)
> 
> Are the code paths that break sharing as heavyweight as I was worried
> about?  Would a hypothetical dm-thin2 that used dm-bio-prison-v2 be
> faster?
> 

Biggest problem is the size of chunks - the smaller chunk you could use,
the less amplification you get. On the other hand the amount of metadata 
handling is increasing. Then there is a lot about parallelization, locking and 
disk synchronization.

If you are more interested in this topic, dive into kernel code.
Also I'd suggest to make some good benchmarking.

Regards

Zdenek

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/