From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: Max Reitz <mreitz@redhat.com>, qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, eblake@redhat.com, armbru@redhat.com,
fam@euphon.net, stefanha@redhat.com, kwolf@redhat.com,
den@virtuozzo.com
Subject: Re: [PATCH v7 18/21] simplebench/results_to_text: improve view of the table
Date: Fri, 13 Nov 2020 19:24:05 +0300 [thread overview]
Message-ID: <fc4bed94-feb3-822a-4218-0e504ed068fe@virtuozzo.com> (raw)
In-Reply-To: <1ad91d41-1f35-f49a-93b9-ef7f16de78ad@redhat.com>
13.11.2020 18:59, Max Reitz wrote:
> On 21.10.20 16:58, Vladimir Sementsov-Ogievskiy wrote:
>> Move to generic format for floats and percentage for error.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>> scripts/simplebench/results_to_text.py | 13 ++++++++++++-
>> 1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/simplebench/results_to_text.py b/scripts/simplebench/results_to_text.py
>> index 58d909ffd9..479f7ac1d4 100644
>> --- a/scripts/simplebench/results_to_text.py
>> +++ b/scripts/simplebench/results_to_text.py
>> @@ -16,11 +16,22 @@
>> # along with this program. If not, see <http://www.gnu.org/licenses/>.
>> #
>> +import math
>> +
>> +
>> +def format_value(x, stdev):
>> + stdev_pr = stdev / x * 100
>> + if stdev_pr < 1.5:
>> + # don't care too much
>> + return f'{x:.2g}'
>> + else:
>> + return f'{x:.2g} ± {math.ceil(stdev_pr)}%'
>
> OK, so no magnitude-based precision this time (except for the %f -> %g change). Works for me.
>
> Other than that, I personally don’t like the relative standard deviation much, because the absolute SD immediately shows the 68 % boundaries (and an idea on the 95 % boundaries with 2σ), whereas the RSD just gives an impression on how spread out the data is. (Which I find the absolute SD also does, when given together with the average, which is the case here.)
I realized that for myself, the only thing I need from +-deviation is a kind of "reliability" of the result. And it's more obvious for me in percentage. Looking at the table with a lot of numbers, where most of values have deviation less than +-5%, some values with deviation +-30 would catch the eye immediately. And with absolute SD I have to look through the table more carefully.
>
> To be completely honest, though, I didn’t even know the term “relative SD” existed until a couple of minutes ago, and I didn’t know it was something that was used at all.
> And if I haven’t seen the RSD used in practice, I can’t confidently say that I have good reasons not to like it.
>
> But, well, I can’t have any confidence in liking it either, and because the change from ASD to RSD is basically the most important change of this patch (which I can’t really agree is an improvement), I can’t really give an R-b.
>
> Perhaps this is OK:
>
> Acked-by: Max Reitz <mreitz@redhat.com>
>
Thanks!
I don't have strict opinion about how this should look. For now these scripts don't have wide usage... We can always add an option to adjust table view, and I will not mind making absolute SD the default view.
--
Best regards,
Vladimir
next prev parent reply other threads:[~2020-11-13 16:25 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-21 14:58 [PATCH v7 00/21] preallocate filter Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 01/21] block: simplify comment to BDRV_REQ_SERIALISING Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 02/21] block/io.c: drop assertion on double waiting for request serialisation Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 03/21] block/io: split out bdrv_find_conflicting_request Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 04/21] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 05/21] block: bdrv_mark_request_serialising: split non-waiting function Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 06/21] block: introduce BDRV_REQ_NO_WAIT flag Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 07/21] block: bdrv_check_perm(): process children anyway Vladimir Sementsov-Ogievskiy
2020-11-13 13:59 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 08/21] block: introduce preallocate filter Vladimir Sementsov-Ogievskiy
2020-11-13 14:32 ` Max Reitz
2020-11-13 15:46 ` Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 09/21] qemu-io: add preallocate mode parameter for truncate command Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 10/21] iotests: qemu_io_silent: support --image-opts Vladimir Sementsov-Ogievskiy
2020-10-21 14:58 ` [PATCH v7 11/21] iotests.py: execute_setup_common(): add required_fmts argument Vladimir Sementsov-Ogievskiy
2020-11-13 14:42 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 12/21] iotests: add 298 to test new preallocate filter driver Vladimir Sementsov-Ogievskiy
2020-11-13 14:43 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 13/21] scripts/simplebench: fix grammar: s/successed/succeeded/ Vladimir Sementsov-Ogievskiy
2020-11-13 14:44 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 14/21] scripts/simplebench: support iops Vladimir Sementsov-Ogievskiy
2020-11-13 15:02 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 15/21] scripts/simplebench: use standard deviation for +- error Vladimir Sementsov-Ogievskiy
2020-11-13 15:30 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 16/21] simplebench: rename ascii() to results_to_text() Vladimir Sementsov-Ogievskiy
2020-11-13 15:31 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 17/21] simplebench: move results_to_text() into separate file Vladimir Sementsov-Ogievskiy
2020-11-13 15:39 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 18/21] simplebench/results_to_text: improve view of the table Vladimir Sementsov-Ogievskiy
2020-11-13 15:59 ` Max Reitz
2020-11-13 16:24 ` Vladimir Sementsov-Ogievskiy [this message]
2020-10-21 14:58 ` [PATCH v7 19/21] simplebench/results_to_text: add difference line to " Vladimir Sementsov-Ogievskiy
2020-10-24 10:24 ` Vladimir Sementsov-Ogievskiy
2020-11-13 16:21 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 20/21] simplebench/results_to_text: make executable Vladimir Sementsov-Ogievskiy
2020-11-13 16:22 ` Max Reitz
2020-10-21 14:58 ` [PATCH v7 21/21] scripts/simplebench: add bench_prealloc.py Vladimir Sementsov-Ogievskiy
2020-11-13 16:24 ` Max Reitz
2020-11-13 18:00 ` Vladimir Sementsov-Ogievskiy
2020-11-13 19:32 ` Max Reitz
2020-11-13 19:33 ` [PATCH v7 00/21] preallocate filter Max Reitz
2020-11-16 9:32 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fc4bed94-feb3-822a-4218-0e504ed068fe@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=armbru@redhat.com \
--cc=den@virtuozzo.com \
--cc=eblake@redhat.com \
--cc=fam@euphon.net \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).