Re: [linux-lvm] Why is the performance of my lvmthin snapshot so poor

From: Gionatan Danti <g.danti@assyoma.it>
To: Demi Marie Obenour <demi@invisiblethingslab.com>
Cc: LVM general discussion and development <linux-lvm@redhat.com>
Subject: Re: [linux-lvm] Why is the performance of my lvmthin snapshot so poor
Date: Thu, 16 Jun 2022 15:22:09 +0200	[thread overview]
Message-ID: <7caac0c00c5c7cd93fdf50b62e2e7907@assyoma.it> (raw)
In-Reply-To: <Yqrhmwwbp5KfiEWb@itl-email>

Il 2022-06-16 09:53 Demi Marie Obenour ha scritto:
> That seems reasonable.  My conclusion is that dm-thin (which is what 
> LVM
> uses) is not a good fit for workloads with a lot of small random writes
> and frequent snapshots, due to the 64k minimum chunk size.  This also
> explains why dm-thin does not allow smaller blocks: not only would it
> only support very small thin pools, it would also have massive metadata
> write overhead.  Hopefully dm-thin v2 will improve the situation.

I think that, in this case, no free lunch really exists. I tried the 
following thin provisioning methods, each with its strong & weak points:

lvmthin: probably the more flexible of the mainline kernel options. You 
pay for r/m/w only when allocating a small block (say 4K) the first time 
after taking a snapshot. It is fast and well integrated with lvm command 
line. Con: bad behavior on out-of-space condition

xfs + reflink: a great, simple to use tool when applicable. It has a 
very small granularity (4K) with no r/m/w. Cons: requires fine tuning 
for good performance when reflinking big files; IO freezes during 
metadata copy for reflink; a very small granularity means sequential IO 
is going to suffer heavily (see here for more details: 
https://marc.info/?l=linux-xfs&m=157891132109888&w=2)

btrfs: very small granularity (4K) and many integrated features. Cons: 
bad performance overall, especially when using mechanical HDD

vdo: is provides small granularity (4K) thin provisioning, compression 
and deduplication. Cons: (still) out-of-tree; requires a powerloss 
protected writeback cache to maintain good performance; no snapshot 
capability

zfs: designed for the ground up for pervasive CoW, with many features 
and ARC/L2ARC. Cons: out-of-tree; using small granularity (4K) means bad 
overall performance; using big granularity (128K by default) is a 
necessary compromise for most HDD pools.

For what it is worth, I settled on ZFS when using out-of-tree modules is 
not an issue and lvmthin otherwise (but I plan to use xfs + reflink more 
in the future).

Do you have any information to share about dm-thin v2? I heard about it 
some years ago, but I found no recent info.

Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://listman.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/