* [linux-lvm] Fast thin volume preallocation?
@ 2019-05-31 13:13 Gionatan Danti
2019-06-03 13:23 ` Joe Thornber
0 siblings, 1 reply; 6+ messages in thread
From: Gionatan Danti @ 2019-05-31 13:13 UTC (permalink / raw)
To: LVM general discussion and development
Hi all,
doing some tests on a 4-bays, entry-level NAS/SAN system, I discovered
it is entirely based on lvm thin volumes.
On configuring what it calls "thick volumes" it create a new thin
logical volume and pre-allocates all space inside the new volume.
What surprised me is the speed at which this allocation happens: a 2 TB
volume was allocated (ie: all data chunks were touched) in about 2 min.
This immediately exclude any simple zeroing of the volume, which will
require much more time.
I tried on a regular CentOS box + lvmthin with lvm zeroing disabled and,
indeed, allocating all blocks inside a thin volume requires much more
time. I tried both a very simple "dd if=/dev/zero of=/dev/test/thinvol
bs=1M oflag=direct" and "blkdiscard -z /dev/test/thinvol".
Being curious, I found that the NAS use a binary [1] that seems to
create a pattern of null writes with extremely high queue depth [2].
So, my questions:
- for what you know, are commercial NAS using some patched/custom
lvmthin version which enables fast volume preallocation, or early zero
rejection?
- does standard lvmthin support something similar? If not, how do you
see a zero coalesce/compression/trim/whatever feature?
- can I obtain something similar by simply touching (maybe with a 512B
only write) once each thin chunk?
Thanks.
[1] I am not calling it because I don't know if discovering the binary,
and so the NAS vendor, is contrary to this mailing list policy.
[2] iostat -x 1 produces the following example output. Please see how
*no* writes are passed on backing devices:
extended device statistics
device mgr/s mgw/s r/s w/s kr/s kw/s size queue wait
svc_t %b
sda 0 0 105.2 0.0 841.3 0.0 8.0 0.0 0.2
0.2 2
sdb 0 0 62.9 0.0 503.2 0.0 8.0 0.0 0.2
0.2 1
sdd 0 0 124.8 0.0 998.5 0.0 8.0 0.0 0.0
0.0 0
sdc 0 0 119.9 0.0 959.2 0.0 8.0 0.0 0.2
0.2 2
mtdblock0 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock1 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock2 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock3 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock4 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock5 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
mtdblock6 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
md9 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
md13 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
md256 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
md322 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
md1 0 0 412.8 0.0 3302.2 0.0 8.0 0.0 0.0
0.0 0
dm-1 0 0 412.8 0.0 3302.2 0.0 8.0 0.0 0.1
0.1 5
dm-2 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-3 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-4 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-5 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-6 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-7 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
dm-8 0 0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
0.0 99
dm-0 0 0 0.0 0.0 0.0 0.0 0.0 1.0 0.0
0.0 99
dm-9 0 0 0.0 0.0 0.0 0.0 0.0 345966.5 0.0
0.0 99
dm-10 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [linux-lvm] Fast thin volume preallocation?
2019-05-31 13:13 [linux-lvm] Fast thin volume preallocation? Gionatan Danti
@ 2019-06-03 13:23 ` Joe Thornber
2019-06-03 19:23 ` Gionatan Danti
0 siblings, 1 reply; 6+ messages in thread
From: Joe Thornber @ 2019-06-03 13:23 UTC (permalink / raw)
To: LVM general discussion and development
On Fri, May 31, 2019 at 03:13:41PM +0200, Gionatan Danti wrote:
> - does standard lvmthin support something similar? If not, how do you see a
> zero coalesce/compression/trim/whatever feature?
There isn't such a feature as yet. With the next iteration of thin I'd like to
get away from the fixed allocation block sizes that we're using which should greatly
reduce the number of mappings that we need to create to provision a thick volume and
so speed it up a lot.
Presumably you want a thick volume but inside a thin pool so that you can used snapshots?
If so have you considered the 'external snapshot' feature?
> - can I obtain something similar by simply touching (maybe with a 512B only
> write) once each thin chunk?
Yes.
- Joe
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] Fast thin volume preallocation?
2019-06-03 13:23 ` Joe Thornber
@ 2019-06-03 19:23 ` Gionatan Danti
2019-06-03 21:12 ` Ilia Zykov
2019-06-04 5:23 ` Ilia Zykov
0 siblings, 2 replies; 6+ messages in thread
From: Gionatan Danti @ 2019-06-03 19:23 UTC (permalink / raw)
To: LVM general discussion and development; +Cc: Joe Thornber
Il 03-06-2019 15:23 Joe Thornber ha scritto:
> On Fri, May 31, 2019 at 03:13:41PM +0200, Gionatan Danti wrote:
>
>> - does standard lvmthin support something similar? If not, how do you
>> see a
>> zero coalesce/compression/trim/whatever feature?
>
> There isn't such a feature as yet.
Ok, so the NAS vendor did something custom/proprietary. This is a reason
why I do *not* like such devices: how much testing and verification are
done before and after such customization? Not a lot, I would say.
> With the next iteration of thin I'd like to
> get away from the fixed allocation block sizes that we're using which
> should greatly
> reduce the number of mappings that we need to create to provision a
> thick volume and
> so speed it up a lot.
Interesting. Do you plan to have an dynamic chunk size bound to some
minumum/maximum? If so, how do you plan to tackle situations where a
single big chunk should be split due to a small CoW write happening
inside the big chunk? Example: a 128MB chunk-sized thin LV which is
snapshotted and a small 4K writes happening at offset 32M from chunk
start.
> Presumably you want a thick volume but inside a thin pool so that you
> can used snapshots?
> If so have you considered the 'external snapshot' feature?
Yes, in some cases they are quite useful. Still, a fast volume
allocation can be an handy addition.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] Fast thin volume preallocation?
2019-06-03 19:23 ` Gionatan Danti
@ 2019-06-03 21:12 ` Ilia Zykov
2019-06-05 10:31 ` Zdenek Kabelac
2019-06-04 5:23 ` Ilia Zykov
1 sibling, 1 reply; 6+ messages in thread
From: Ilia Zykov @ 2019-06-03 21:12 UTC (permalink / raw)
To: linux-lvm
[-- Attachment #1: Type: text/plain, Size: 600 bytes --]
>
>> Presumably you want a thick volume but inside a thin pool so that you
>> can used snapshots?
>> If so have you considered the 'external snapshot' feature?
>
> Yes, in some cases they are quite useful. Still, a fast volume
> allocation can be an handy addition.
>
Hello.
Can I use external snapshot for fast zero allocation?
"thpool" - is lvmthin with lvm zeroing disabled
# lvcreate -n ext2T -V 2TiB --thinpool thpool VG
# lvchange --permission r VG/ext2T
# lvcreate -n zeroed_lve -s VG/ext2T --thinpool VG/thpool
Or it will be the same as zeroing enabled?
Thanks.
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3599 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] Fast thin volume preallocation?
2019-06-03 21:12 ` Ilia Zykov
@ 2019-06-05 10:31 ` Zdenek Kabelac
0 siblings, 0 replies; 6+ messages in thread
From: Zdenek Kabelac @ 2019-06-05 10:31 UTC (permalink / raw)
To: LVM general discussion and development, Ilia Zykov
Dne 03. 06. 19 v 23:12 Ilia Zykov napsal(a):
>>
>>> Presumably you want a thick volume but inside a thin pool so that you
>>> can used snapshots?
>>> If so have you considered the 'external snapshot' feature?
>>
>> Yes, in some cases they are quite useful. Still, a fast volume
>> allocation can be an handy addition.
>>
>
> Hello.
> Can I use external snapshot for fast zero allocation?
> "thpool" - is lvmthin with lvm zeroing disabled
>
> # lvcreate -n ext2T -V 2TiB --thinpool thpool VG
> # lvchange --permission r VG/ext2T
>
> # lvcreate -n zeroed_lve -s VG/ext2T --thinpool VG/thpool
>
> Or it will be the same as zeroing enabled?
Hi
It's not clear what do you want to achieve.
External origin for thin-volume is always a read-only LV.
External origin can be of any type - i.e. even thinLV from another thin-pool
and single origin can be used for number of thin volumes with external origin.
For now you CANNOT 'merge' such thinLV with external origin - so the only way
to get 'writable' origin back is to drop all thins that are using it as
external origin.
Now - how the 'zeroing' plays the role here ?
Zeroing is property of thin-pool that serves thinLV.
Unprovisioned blocks always do return 'zeroes'.
When block is provisioned - unwritten portions needs to be zeroed (depending
on zeroing setting) - the bigger the block the bigger the chance the zeroing
will take more time - lvm2 does not recommend using thin-pool with zeroing and
chunksize bigger then 512KB.
Most modern filesystem keep track of written pieces of a block device
themselves, so zeroing has almost no practical use.
Regards
Zdenek
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] Fast thin volume preallocation?
2019-06-03 19:23 ` Gionatan Danti
2019-06-03 21:12 ` Ilia Zykov
@ 2019-06-04 5:23 ` Ilia Zykov
1 sibling, 0 replies; 6+ messages in thread
From: Ilia Zykov @ 2019-06-04 5:23 UTC (permalink / raw)
To: linux-lvm
>> Presumably you want a thick volume but inside a thin pool so that you
>> can used snapshots?
>> If so have you considered the 'external snapshot' feature?
>
> Yes, in some cases they are quite useful. Still, a fast volume
> allocation can be an handy addition.
>
Hello.
Can I use external snapshot for fast zero allocation?
"thpool" - is lvmthin with lvm zeroing disabled
# lvcreate -n ext2T -V 2TiB --thinpool thpool VG
# lvchange --permission r VG/ext2T
# lvcreate -n zeroed_lve -s VG/ext2T --thinpool VG/thpool
Or it will be the same as zeroing enabled?
Thanks.
----
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-06-05 10:31 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-31 13:13 [linux-lvm] Fast thin volume preallocation? Gionatan Danti
2019-06-03 13:23 ` Joe Thornber
2019-06-03 19:23 ` Gionatan Danti
2019-06-03 21:12 ` Ilia Zykov
2019-06-05 10:31 ` Zdenek Kabelac
2019-06-04 5:23 ` Ilia Zykov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).