Re: Proposition - latency histogram

["Bartłomiej Święcki" <bartlomiej.swiecki@corp.ovh.com>]

[-- Attachment #1: Type: text/plain, Size: 2710 bytes --]

Attached is a sample output from test python script (included in PR)
to display live results.


On 01/31/2017 04:22 PM, Bartłomiej Święcki wrote:
> Hi,
>
> Bringing back performance histograms: 
> https://github.com/ceph/ceph/pull/12829
> I've updated the PR, rebased on master and made internal changes less 
> aggressive.
>
> All ctest tests passing and I haven't seen any issues with performance
> (and I can actually see much better what the performance 
> characteristics are
>
> Waiting for your comments,
> Bartek
>
>
>
> Looking
>
> On 01/09/2017 12:27 PM, Bartłomiej Święcki wrote:
>> Hi,
>>
>> I've made a simple implementation of performance histograms. 
>> Implementation is not very sophisticated
>> but I think it could be a good start for more detailed discussion.
>>
>> Here's the PR: https://github.com/ceph/ceph/pull/12829
>>
>>
>> Regards,
>> Bartek
>>
>>
>> On 11/28/2016 05:22 PM, Bartłomiej Święcki wrote:
>>> Hi,
>>>
>>>
>>> Currently we can query OSD for op latency but it's given as an 
>>> average. Average may not give
>>> the bets information in this case - i.e. spikes can easily get 
>>> hidden there.
>>>
>>> Instead of an average we could easily do a simple histogram - 
>>> quantize the latency into
>>> predefined set of time intervals, for each of them have a simple 
>>> performance counter,
>>> at each op increase one of them. Since those are per OSD, we could 
>>> have pretty high resolution
>>> with fractional memory usage, performance impact should be 
>>> negligible since only one (two if split
>>> into read and write) of those counters would be incremented per one 
>>> osd op.
>>>
>>> In addition we could also do this in 2D - each counter matching 
>>> given latency range and op size range.
>>> having such 2D table would show both latency histogram, request size 
>>> histogram and combinations of those
>>> (i.e. latency histogram of ~4k ops only).
>>>
>>> What do you think about this idea? I can prepare some code - a 
>>> simple proof of concept looks really
>>> straightforward to implement.
>>>
>>>
>>> Bartek
>>>
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: Screenshot - 01312017 - 04:29:15 PM.png --]
[-- Type: image/png, Size: 70819 bytes --]