* Thinly-Provisioned Logical Volumes
@ 2014-09-20 14:19 lilofile
2014-09-22 7:02 ` Zdenek Kabelac
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: lilofile @ 2014-09-20 14:19 UTC (permalink / raw)
To: Mike Snitzer, Ondrej Kozina, Alasdair G. Kergon; +Cc: dm-devel
when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of thin volume. how can i quickly understand dm-bufio.c?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Thinly-Provisioned Logical Volumes
2014-09-20 14:19 Thinly-Provisioned Logical Volumes lilofile
@ 2014-09-22 7:02 ` Zdenek Kabelac
2014-09-22 12:38 ` Mike Snitzer
2014-09-23 13:51 ` reply: " lilofile
2 siblings, 0 replies; 8+ messages in thread
From: Zdenek Kabelac @ 2014-09-22 7:02 UTC (permalink / raw)
To: lilofile, device-mapper development, Mike Snitzer, Ondrej Kozina,
Alasdair G. Kergon
Dne 20.9.2014 v 16:19 lilofile napsal(a):
> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of thin volume. how can i quickly understand dm-bufio.c?
>
In general 2 major factors -
1. - size of chunk/(provisioned block)
the bigger the block is - the better usually filesystem performs - since the
all the aggregation logic and optimization of fs works.
on the other - the bigger the block is - the less efficient snapshot you get
(since much bigger chunk needs to be copied for i.e. single byte write)
2. - zeroing on provisionsing.
when the block provisioned for the first time - it's (by default config) fully
zeroes - so you don't get random disk content from unwritten areas.
If you don't care about this - you could safely disable zeroing - number of
filesystem usually already manages unwritten space well so if you use block
device for ext4 so shouldn't normally be able to read unwritten areas.
So please try i.e. 256KB chunk size with non-zeroing (-c 256 -Zn) when
creating thin-pool - do you still fell you get poor performance ?
Zdenek
NOTE: you could change zeroing with existing pool, but not your thin-pool
chunk size for now (though future version might be less restrictive here).
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Thinly-Provisioned Logical Volumes
2014-09-20 14:19 Thinly-Provisioned Logical Volumes lilofile
2014-09-22 7:02 ` Zdenek Kabelac
@ 2014-09-22 12:38 ` Mike Snitzer
2014-09-23 13:51 ` reply: " lilofile
2 siblings, 0 replies; 8+ messages in thread
From: Mike Snitzer @ 2014-09-22 12:38 UTC (permalink / raw)
To: lilofile; +Cc: Ondrej Kozina, device-mapper development, Alasdair G. Kergon
On Sat, Sep 20 2014 at 10:19am -0400,
lilofile <lilofile@aliyun.com> wrote:
> when I test Thinly-Provisioned Logical Volumes(use device thinprovison
> target),I found the random write performance is very low. What factors
> will influence the random write performance of thin volume.
It is worth reading this blog post from Joe:
http://device-mapper.org/blog/2014/01/21/random-io-and-copy-on-write-schemes/
> how can i quickly understand dm-bufio.c?
What is it you'd like to know? dm-bufio.c isn't the smoking gun for
random write performance being slow though. It is more to do with the
natural impact random writes have on thinp (as discussed in the above
article).
^ permalink raw reply [flat|nested] 8+ messages in thread
* reply: Thinly-Provisioned Logical Volumes
2014-09-20 14:19 Thinly-Provisioned Logical Volumes lilofile
2014-09-22 7:02 ` Zdenek Kabelac
2014-09-22 12:38 ` Mike Snitzer
@ 2014-09-23 13:51 ` lilofile
2014-09-23 14:07 ` Zdenek Kabelac
2014-09-24 2:11 ` lilofile
2 siblings, 2 replies; 8+ messages in thread
From: lilofile @ 2014-09-23 13:51 UTC (permalink / raw)
To: device-mapper development, Mike Snitzer, Ondrej Kozina,
Alasdair G. Kergon, Zdenek Kabelac
when I creating thin-pool with non-zeroing(-Zn),the ramdom performance can be greatly improved. what is the difference in code with non-zeroing and zeroing?
------------------------------------------------------------------
发件人:Zdenek Kabelac <zkabelac@redhat.com>
发送时间:2014年9月22日(星期一) 15:02
收件人:lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
主 题:Re: [dm-devel] Thinly-Provisioned Logical Volumes
Dne 20.9.2014 v 16:19 lilofile napsal(a):
> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of thin volume. how can i quickly understand dm-bufio.c?
>
In general 2 major factors -
1. - size of chunk/(provisioned block)
the bigger the block is - the better usually filesystem performs - since the
all the aggregation logic and optimization of fs works.
on the other - the bigger the block is - the less efficient snapshot you get
(since much bigger chunk needs to be copied for i.e. single byte write)
2. - zeroing on provisionsing.
when the block provisioned for the first time - it's (by default config) fully
zeroes - so you don't get random disk content from unwritten areas.
If you don't care about this - you could safely disable zeroing - number of
filesystem usually already manages unwritten space well so if you use block
device for ext4 so shouldn't normally be able to read unwritten areas.
So please try i.e. 256KB chunk size with non-zeroing (-c 256 -Zn) when
creating thin-pool - do you still fell you get poor performance ?
Zdenek
NOTE: you could change zeroing with existing pool, but not your thin-pool
chunk size for now (though future version might be less restrictive here).
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: reply: Thinly-Provisioned Logical Volumes
2014-09-23 13:51 ` reply: " lilofile
@ 2014-09-23 14:07 ` Zdenek Kabelac
2014-09-24 2:11 ` lilofile
1 sibling, 0 replies; 8+ messages in thread
From: Zdenek Kabelac @ 2014-09-23 14:07 UTC (permalink / raw)
To: lilofile, device-mapper development, Mike Snitzer, Ondrej Kozina,
Alasdair G. Kergon
Dne 23.9.2014 v 15:51 lilofile napsal(a):
> when I creating thin-pool with non-zeroing(-Zn),the ramdom performance can be greatly improved. what is the difference in code with non-zeroing and zeroing?
>
>
*Zeroing is set*
When you write to an unprovisioned chunk - certain amount of bytes in this
chunk have known content (from your write BIO) and everything else gets zeroes
(that's zeroing on provisioning)
So when you read your thin device - you always get same content.
(since unprovisioned areas returns zero - and provisioned returns
what you have written - or zeros from unwritten part)
Of course zeroing i.e. 1MB chunks is quite slow compared with write of few
single bytes on chunk provisioning.
*Zeroing is unset*
Unprovisioned data are still 'zeroes'
However when you first write to 'unprovisioned' chunk - then your data
are again in the place you wrote then - BUT remaining content is - what has
been found on your disk in the provisioned chunk - so now when you read from
'unwritten' areas in provisioned chunk - you read some random garbage left on
your disk.
This may have i.e. security impact.
So you need to pick what suits your needs best.
For filesystem that keep map of written space - go with -Zn
for improved security stay with -Zy and be prepared to pay some price with
initial provisioning - since repeated write has then regular performance.
Zdenek
> ------------------------------------------------------------------
> 发件人:Zdenek Kabelac <zkabelac@redhat.com>
> 发送时间:2014年9月22日(星期一) 15:02
> 收件人:lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
> 主 题:Re: [dm-devel] Thinly-Provisioned Logical Volumes
>
> Dne 20.9.2014 v 16:19 lilofile napsal(a):
>> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of thin volume. how can i quickly understand dm-bufio.c?
>>
>
>
> In general 2 major factors -
>
> 1. - size of chunk/(provisioned block)
>
> the bigger the block is - the better usually filesystem performs - since the
> all the aggregation logic and optimization of fs works.
>
> on the other - the bigger the block is - the less efficient snapshot you get
> (since much bigger chunk needs to be copied for i.e. single byte write)
>
>
> 2. - zeroing on provisionsing.
>
> when the block provisioned for the first time - it's (by default config) fully
> zeroes - so you don't get random disk content from unwritten areas.
>
> If you don't care about this - you could safely disable zeroing - number of
> filesystem usually already manages unwritten space well so if you use block
> device for ext4 so shouldn't normally be able to read unwritten areas.
>
>
> So please try i.e. 256KB chunk size with non-zeroing (-c 256 -Zn) when
> creating thin-pool - do you still fell you get poor performance ?
>
> Zdenek
>
>
> NOTE: you could change zeroing with existing pool, but not your thin-pool
> chunk size for now (though future version might be less restrictive here).
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* reply: Thinly-Provisioned Logical Volumes
2014-09-23 13:51 ` reply: " lilofile
2014-09-23 14:07 ` Zdenek Kabelac
@ 2014-09-24 2:11 ` lilofile
2014-09-24 7:02 ` Zdenek Kabelac
2014-09-24 8:37 ` Joe Thornber
1 sibling, 2 replies; 8+ messages in thread
From: lilofile @ 2014-09-24 2:11 UTC (permalink / raw)
To: device-mapper development, Mike Snitzer, Ondrej Kozina,
Alasdair G. Kergon
When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
I only write 4k data in mythinvolume.but In thin_map functions, I found there are really 4K data written,but I found more read IO,probably dozens of read io. who issue these IO?
------------------------------------------------------------------
发件人:Zdenek Kabelac <zkabelac@redhat.com>
发送时间:2014年9月23日(星期二) 22:07
收件人:lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
主 题:Re: [dm-devel] reply: Thinly-Provisioned Logical Volumes
Dne 23.9.2014 v 15:51 lilofile napsal(a):
> when I creating thin-pool with non-zeroing(-Zn),the ramdom performance can be greatly improved. what is the difference in code with non-zeroing and zeroing?
>
>
*Zeroing is set*
When you write to an unprovisioned chunk - certain amount of bytes in this
chunk have known content (from your write BIO) and everything else gets zeroes
(that's zeroing on provisioning)
So when you read your thin device - you always get same content.
(since unprovisioned areas returns zero - and provisioned returns
what you have written - or zeros from unwritten part)
Of course zeroing i.e. 1MB chunks is quite slow compared with write of few
single bytes on chunk provisioning.
*Zeroing is unset*
Unprovisioned data are still 'zeroes'
However when you first write to 'unprovisioned' chunk - then your data
are again in the place you wrote then - BUT remaining content is - what has
been found on your disk in the provisioned chunk - so now when you read from
'unwritten' areas in provisioned chunk - you read some random garbage left on
your disk.
This may have i.e. security impact.
So you need to pick what suits your needs best.
For filesystem that keep map of written space - go with -Zn
for improved security stay with -Zy and be prepared to pay some price with
initial provisioning - since repeated write has then regular performance.
Zdenek
> ------------------------------------------------------------------
> 发件人:Zdenek Kabelac <zkabelac@redhat.com>
> 发送时间:2014年9月22日(星期一) 15:02
> 收件人:lilofile <lilofile@aliyun.com>; device-mapper development <dm-devel@redhat.com>; Mike Snitzer <msnitzer@redhat.com>; Ondrej Kozina <okozina@redhat.com>; Alasdair G. Kergon <agk@redhat.com>
> 主 题:Re: [dm-devel] Thinly-Provisioned Logical Volumes
>
> Dne 20.9.2014 v 16:19 lilofile napsal(a):
>> when I test Thinly-Provisioned Logical Volumes(use device thinprovison target),I found the random write performance is very low. What factors will influence the random write performance of thin volume. how can i quickly understand dm-bufio.c?
>>
>
>
> In general 2 major factors -
>
> 1. - size of chunk/(provisioned block)
>
> the bigger the block is - the better usually filesystem performs - since the
> all the aggregation logic and optimization of fs works.
>
> on the other - the bigger the block is - the less efficient snapshot you get
> (since much bigger chunk needs to be copied for i.e. single byte write)
>
>
> 2. - zeroing on provisionsing.
>
> when the block provisioned for the first time - it's (by default config) fully
> zeroes - so you don't get random disk content from unwritten areas.
>
> If you don't care about this - you could safely disable zeroing - number of
> filesystem usually already manages unwritten space well so if you use block
> device for ext4 so shouldn't normally be able to read unwritten areas.
>
>
> So please try i.e. 256KB chunk size with non-zeroing (-c 256 -Zn) when
> creating thin-pool - do you still fell you get poor performance ?
>
> Zdenek
>
>
> NOTE: you could change zeroing with existing pool, but not your thin-pool
> chunk size for now (though future version might be less restrictive here).
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: reply: Thinly-Provisioned Logical Volumes
2014-09-24 2:11 ` lilofile
@ 2014-09-24 7:02 ` Zdenek Kabelac
2014-09-24 8:37 ` Joe Thornber
1 sibling, 0 replies; 8+ messages in thread
From: Zdenek Kabelac @ 2014-09-24 7:02 UTC (permalink / raw)
To: lilofile, device-mapper development, Mike Snitzer, Ondrej Kozina,
Alasdair G. Kergon
Dne 24.9.2014 v 04:11 lilofile napsal(a):
> When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
> I only write 4k data in mythinvolume.but In thin_map functions, I found there are really 4K data written,but I found more read IO,probably dozens of read io. who issue these IO?
>
Hi
I'd have guessed those I/O are likely metadata update related - at most every
second metadata are synchronized to your metadata storage volume.
(Thin pool consists of _tdata data volume where provisioned chunks are located
and _tmeta metadata volume).
And btw you could use tools like thin_dump to explore how is your pool used,
which chunks belong to which volume, which are share and so on....
Regards
Zdenek
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: reply: Thinly-Provisioned Logical Volumes
2014-09-24 2:11 ` lilofile
2014-09-24 7:02 ` Zdenek Kabelac
@ 2014-09-24 8:37 ` Joe Thornber
1 sibling, 0 replies; 8+ messages in thread
From: Joe Thornber @ 2014-09-24 8:37 UTC (permalink / raw)
To: lilofile, device-mapper development
Cc: Ondrej Kozina, Mike Snitzer, Alasdair G. Kergon
On Wed, Sep 24, 2014 at 10:11:00AM +0800, lilofile wrote:
> When I test thinly-provisioned logical volumes,I use dd if=/dev/zero of=/dev/mapper/mythinvolume count=1 bs=4k
> I only write 4k data in mythinvolume.but In thin_map functions, I found there are really 4K data written,but I found more read IO,probably dozens of read io. who issue these IO?
Assuming the IO is going to the data device, and not the metadata
device, then the likely candidates would be read-ahead or udev.
- Joe
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-09-24 8:37 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-20 14:19 Thinly-Provisioned Logical Volumes lilofile
2014-09-22 7:02 ` Zdenek Kabelac
2014-09-22 12:38 ` Mike Snitzer
2014-09-23 13:51 ` reply: " lilofile
2014-09-23 14:07 ` Zdenek Kabelac
2014-09-24 2:11 ` lilofile
2014-09-24 7:02 ` Zdenek Kabelac
2014-09-24 8:37 ` Joe Thornber
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.