From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, den@openvz.org, vsementsov@virtuozzo.com,
mreitz@redhat.com
Subject: [PATCH v2 1/8] simplebench: bench_one(): add slow_limit argument
Date: Thu, 4 Mar 2021 13:17:31 +0300 [thread overview]
Message-ID: <20210304101738.20248-2-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20210304101738.20248-1-vsementsov@virtuozzo.com>
Sometimes one of cells in a testing table runs too slow. And we really
don't want to wait so long. Limit number of runs in this case.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
scripts/simplebench/simplebench.py | 29 +++++++++++++++++++++++++----
1 file changed, 25 insertions(+), 4 deletions(-)
diff --git a/scripts/simplebench/simplebench.py b/scripts/simplebench/simplebench.py
index f61513af90..b153cae274 100644
--- a/scripts/simplebench/simplebench.py
+++ b/scripts/simplebench/simplebench.py
@@ -19,9 +19,11 @@
#
import statistics
+import time
-def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
+def bench_one(test_func, test_env, test_case, count=5, initial_run=True,
+ slow_limit=100):
"""Benchmark one test-case
test_func -- benchmarking function with prototype
@@ -36,6 +38,8 @@ def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
test_case -- test case - opaque second argument for test_func
count -- how many times to call test_func, to calculate average
initial_run -- do initial run of test_func, which don't get into result
+ slow_limit -- reduce test runs to 2, if current run exceedes the limit
+ (in seconds)
Returns dict with the following fields:
'runs': list of test_func results
@@ -47,17 +51,34 @@ def bench_one(test_func, test_env, test_case, count=5, initial_run=True):
'n-failed': number of failed runs (exists only if at least one run
failed)
"""
+ runs = []
+ i = 0
if initial_run:
+ t = time.time()
+
print(' #initial run:')
- print(' ', test_func(test_env, test_case))
+ res = test_func(test_env, test_case)
+ print(' ', res)
+
+ if time.time() - t > slow_limit:
+ print(' - initial run is too slow, so it counts')
+ runs.append(res)
+ i = 1
+
+ for i in range(i, count):
+ t = time.time()
- runs = []
- for i in range(count):
print(' #run {}'.format(i+1))
res = test_func(test_env, test_case)
print(' ', res)
runs.append(res)
+ if time.time() - t > slow_limit and len(runs) >= 2:
+ print(' - run is too slow, and we have enough runs, stop here')
+ break
+
+ count = len(runs)
+
result = {'runs': runs}
succeeded = [r for r in runs if ('seconds' in r or 'iops' in r)]
--
2.29.2
next prev parent reply other threads:[~2021-03-04 10:21 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-04 10:17 [PATCH v2 0/8] simplebench improvements Vladimir Sementsov-Ogievskiy
2021-03-04 10:17 ` Vladimir Sementsov-Ogievskiy [this message]
2021-03-05 1:22 ` [PATCH v2 1/8] simplebench: bench_one(): add slow_limit argument John Snow
2021-03-05 9:03 ` Vladimir Sementsov-Ogievskiy
2021-03-05 16:25 ` John Snow
2021-03-04 10:17 ` [PATCH v2 2/8] simplebench: bench_one(): support count=1 Vladimir Sementsov-Ogievskiy
2021-03-05 1:23 ` John Snow
2021-03-04 10:17 ` [PATCH v2 3/8] simplebench/bench-backup: add --compressed option Vladimir Sementsov-Ogievskiy
2021-03-04 10:17 ` [PATCH v2 4/8] simplebench/bench-backup: add target-cache argument Vladimir Sementsov-Ogievskiy
2021-03-05 1:50 ` John Snow
2021-03-04 10:17 ` [PATCH v2 5/8] simplebench/bench_block_job: handle error in BLOCK_JOB_COMPLETED Vladimir Sementsov-Ogievskiy
2021-03-05 1:47 ` John Snow
2021-03-04 10:17 ` [PATCH v2 6/8] simplebench/bench-backup: support qcow2 source files Vladimir Sementsov-Ogievskiy
2021-03-05 1:43 ` John Snow
2021-03-04 10:17 ` [PATCH v2 7/8] simplebench/bench-backup: add --count and --no-initial-run Vladimir Sementsov-Ogievskiy
2021-03-05 1:37 ` John Snow
2021-03-05 9:09 ` Vladimir Sementsov-Ogievskiy
2021-03-04 10:17 ` [PATCH v2 8/8] simplebench/bench_block_job: drop caches before test run Vladimir Sementsov-Ogievskiy
2021-03-05 1:30 ` John Snow
2021-03-05 9:11 ` Vladimir Sementsov-Ogievskiy
2021-03-05 16:30 ` John Snow
2021-03-05 16:50 ` Vladimir Sementsov-Ogievskiy
2021-03-05 16:54 ` John Snow
2021-03-04 10:44 ` [PATCH v2 9/8] MAINTAINERS: update Benchmark util: add git tree Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210304101738.20248-2-vsementsov@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=den@openvz.org \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.