From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=3.0 tests=DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C72F8C3F2CD for ; Mon, 2 Mar 2020 21:05:59 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F9A321D56 for ; Mon, 2 Mar 2020 21:05:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="s0bAys15" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F9A321D56 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38672 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sGE-0005mX-HS for qemu-devel@archiver.kernel.org; Mon, 02 Mar 2020 16:05:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42987) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sFW-0004zx-Qs for qemu-devel@nongnu.org; Mon, 02 Mar 2020 16:05:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j8sFV-0001Wk-6w for qemu-devel@nongnu.org; Mon, 02 Mar 2020 16:05:14 -0500 Received: from mail-oi1-x232.google.com ([2607:f8b0:4864:20::232]:45406) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j8sFU-0001Vu-Vx; Mon, 02 Mar 2020 16:05:13 -0500 Received: by mail-oi1-x232.google.com with SMTP id v19so664489oic.12; Mon, 02 Mar 2020 13:05:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=uOsTYYmk3JNHpnDxPJSm4+e5Ig7FQrCHgTPHOERUesw=; b=s0bAys15cvAz2uCgowmu3kSMkgMb+Hbe7UUPiLZ9SZdae9or8zf3UGjQtC95SqI1x1 2dAqwkVezPnInaeb9Ols7CRXr4pd0mH3ekhiWGjRE77NOdE3YZx6VugJGm1FWBhOO2TB zRK/p4EaXBrmr5Xhgv+XvZJNh8TAhzAZGARdcbp8kfWCKKCKRz+fnoaFwwEdqX1yphFO U2+LRK6QB+T8tDO/teH1W+JGlehkcPqdL0hFxw6Nn/TtPtJAXdlgIASmYH93g400107W UB1ItLn3eISod/rOZckeKEVhbx+sdyNb+TRFG2CMMpiZ4Y7GHjsYL73thVyeFrijwD2F poEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uOsTYYmk3JNHpnDxPJSm4+e5Ig7FQrCHgTPHOERUesw=; b=H2c+ffvhT8YuEM0RI6c8uyYieCKPt22dIFRn5CigOABOGBVB3NbibQl/d/z+6gtvuf sUzkVgQAX1jvsOdVQwmz0ratqf2+UZrsRvNeWOWjA8+Zb+Yo1gUM497OH1mbF2Md7KLa DptzASrP2hTM0L8iofLp7exI2RJwpDPTvXVqjudIHtvZRRMVLBSLazb6I4ZLgd/i7jzo gmdLM2TLplsEyN7tBY4jdbuL6KZVsznSH7+JA9oxh4hhYOm+RkUDL3WSsrki0qFIx9xX yl31rFvwHn0pEgolKNlFzhPBZN+6ZXLetbuHgHhRVxWVFmRKuo6pQ/HTqATszqtVsZxN wsSw== X-Gm-Message-State: ANhLgQ2vWYBeuv6eoCxveH6PpMSheL/8Y2/h0jaWS6RyTLbv1MtCH2di AYszZwfYjLFM59WO6ucpBp7GTeQ5hf5s6U362Nc= X-Google-Smtp-Source: ADFU+vtbi3tPjuOeO/cMf4B0iVj+kH2U0dt7Bbf80jZLYgqMAjYbTxJ1LgRmfcNg3jsiPPGuMoCHoLwVNUhM2lgKZv0= X-Received: by 2002:aca:c3c5:: with SMTP id t188mr241853oif.53.1583183111562; Mon, 02 Mar 2020 13:05:11 -0800 (PST) MIME-Version: 1.0 References: <20200228071914.11746-1-vsementsov@virtuozzo.com> <20200228071914.11746-2-vsementsov@virtuozzo.com> <75b4d5ad-f95f-dab2-1b53-5e654c29015e@virtuozzo.com> In-Reply-To: <75b4d5ad-f95f-dab2-1b53-5e654c29015e@virtuozzo.com> From: Aleksandar Markovic Date: Mon, 2 Mar 2020 22:05:00 +0100 Message-ID: Subject: Re: [PATCH v3 1/4] scripts/simplebench: add simplebench.py To: Vladimir Sementsov-Ogievskiy Content-Type: multipart/alternative; boundary="000000000000faa7bf059fe58a6b" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::232 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , "open list:bochs" , Stefan Hajnoczi , QEMU Developers , Max Reitz , Cleber Rosa , "Denis V. Lunev" , John Snow Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --000000000000faa7bf059fe58a6b Content-Type: text/plain; charset="UTF-8" > >> + > >> + > >> +def ascii_one(result): > >> + """Return ASCII representation of bench_one() returned dict.""" > >> + if 'average' in result: > >> + s = '{:.2f} +- {:.2f}'.format(result['average'], result['delta']) > >> + if 'n-failed' in result: > >> + s += '\n({} failed)'.format(result['n-failed']) > >> + return s > >> + else: > >> + return 'FAILED' > > > > I think it would be visually clearer if "+-" was printed without any > > space between it and the following number, using something > > like this: > > > > s = ' {:.2f} +-{:.2f}'.format(result['average'], result['delta']) > > > > The resulting table would look like: > > > > ---------- ------------- ------------- ------------- > > backup-1 backup-2 mirror > > ssd -> ssd 0.43 +-0.00 4.48 +-0.06 4.38 +-0.02 > > ssd -> hdd 10.60 +-0.08 10.69 +-0.18 10.57 +-0.05 > > ssd -> nbd 33.81 +-0.37 10.67 +-0.17 10.07 +-0.07 > > ---------- ------------- ------------- ------------- > > > > But, this is just cosmetics. > > > > With or without the suggestion above: > > > > Reviewed-by: Aleksandar Markovic > > Thanks for reviewing! > > Agree with this change, but I don't think it worth to resend the series for this one space) > Hope it may be applied with pull request.. > I am an occasional Python programmer, and I felt comfortable reviewing your series, but I am not a maintainer of this directory, and I believe Eduardo or Cleber or other more active Python contributors would be better choice for selecting this series in their pull request. So, I can't send this series to Peter - Cleber, Eduardo, please see to it. Yours, Aleksandar > > > >> + > >> + > >> +def bench(test_func, test_envs, test_cases, *args, **vargs): > >> + """Fill benchmark table > >> + > >> + test_func -- benchmarking function, see bench_one for description > >> + test_envs -- list of test environments, see bench_one > >> + test_cases -- list of test cases, see bench_one > >> + args, vargs -- additional arguments for bench_one > >> + > >> + Returns dict with the following fields: > >> + 'envs': test_envs > >> + 'cases': test_cases > >> + 'tab': filled 2D array, where cell [i][j] is bench_one result for > >> + test_cases[i] for test_envs[j] (i.e., rows are test cases and > >> + columns are test environments) > >> + """ > >> + tab = {} > >> + results = { > >> + 'envs': test_envs, > >> + 'cases': test_cases, > >> + 'tab': tab > >> + } > >> + n = 1 > >> + n_tests = len(test_envs) * len(test_cases) > >> + for env in test_envs: > >> + for case in test_cases: > >> + print('Testing {}/{}: {} :: {}'.format(n, n_tests, > >> + env['id'], case['id'])) > >> + if case['id'] not in tab: > >> + tab[case['id']] = {} > >> + tab[case['id']][env['id']] = bench_one(test_func, env, case, > >> + *args, **vargs) > >> + n += 1 > >> + > >> + print('Done') > >> + return results > >> + > >> + > >> +def ascii(results): > >> + """Return ASCII representation of bench() returned dict.""" > >> + from tabulate import tabulate > >> + > >> + tab = [[""] + [c['id'] for c in results['envs']]] > >> + for case in results['cases']: > >> + row = [case['id']] > >> + for env in results['envs']: > >> + row.append(ascii_one(results['tab'][case['id']][env['id']])) > >> + tab.append(row) > >> + > >> + return tabulate(tab) > >> -- > >> 2.21.0 > >> > >> > > > -- > Best regards, > Vladimir --000000000000faa7bf059fe58a6b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable



> >> +
> >> +
> >= ;> +def ascii_one(result):
> >> + =C2=A0 =C2=A0""&= quot;Return ASCII representation of bench_one() returned dict.""&= quot;
> >> + =C2=A0 =C2=A0if 'average' in result:
&g= t; >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0s =3D '{:.2f} +- {:.2f}'.f= ormat(result['average'], result['delta'])
> >> = + =C2=A0 =C2=A0 =C2=A0 =C2=A0if 'n-failed' in result:
> >&= gt; + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0s +=3D '\n({} failed)= 9;.format(result['n-failed'])
> >> + =C2=A0 =C2=A0 =C2= =A0 =C2=A0return s
> >> + =C2=A0 =C2=A0else:
> >> += =C2=A0 =C2=A0 =C2=A0 =C2=A0return 'FAILED'
> >
> &g= t; I think it would be visually clearer if "+-" was printed witho= ut any
> > space between it and the following number, using someth= ing
> > like this:
> >
> > s =3D ' {:.2f} +-= {:.2f}'.format(result['average'], result['delta'])
&= gt; >
> > The resulting table would look like:
> >
= > > ---------- =C2=A0------------- =C2=A0------------- =C2=A0--------= -----
> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0backup-1= =C2=A0 =C2=A0 =C2=A0 backup-2 =C2=A0 =C2=A0 =C2=A0 mirror
> > ssd= -> ssd =C2=A0 0.43 +-0.00 =C2=A0 =C2=A04.48 +-0.06 =C2=A0 =C2=A04.38 +-= 0.02
> > ssd -> hdd =C2=A0 10.60 +-0.08 =C2=A0 10.69 +-0.18 =C2= =A0 10.57 +-0.05
> > ssd -> nbd =C2=A0 33.81 +-0.37 =C2=A0 10.6= 7 +-0.17 =C2=A0 10.07 +-0.07
> > ---------- =C2=A0------------- = =C2=A0------------- =C2=A0-------------
> >
> > But, this= is just cosmetics.
> >
> > With or without the suggestio= n above:
> >
> > Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
&g= t;
> Thanks for reviewing!
>
> Agree with this change, bu= t I don't think it worth to resend the series for this one space)
&g= t; Hope it may be applied with pull request..
>

I am an occasional Python programmer, and I felt comfortable
=
reviewing your series, but I am not a maintainer of this directory,
and I believe Eduardo or Cleber or other more active Python
contributors would be better choice for selecting this series in
their pull request.

So, I can't send this s= eries to Peter - Cleber, Eduardo, please
see to it.
Yours,
Aleksandar

> >> >> +
> >> +
> >> +def bench(test_func,= test_envs, test_cases, *args, **vargs):
> >> + =C2=A0 =C2=A0&q= uot;""Fill benchmark table
> >> +
> >> + = =C2=A0 =C2=A0test_func -- benchmarking function, see bench_one for descript= ion
> >> + =C2=A0 =C2=A0test_envs -- list of test environments,= see bench_one
> >> + =C2=A0 =C2=A0test_cases -- list of test c= ases, see bench_one
> >> + =C2=A0 =C2=A0args, vargs -- addition= al arguments for bench_one
> >> +
> >> + =C2=A0 =C2= =A0Returns dict with the following fields:
> >> + =C2=A0 =C2=A0= =C2=A0 =C2=A0'envs': =C2=A0test_envs
> >> + =C2=A0 =C2= =A0 =C2=A0 =C2=A0'cases': test_cases
> >> + =C2=A0 =C2= =A0 =C2=A0 =C2=A0'tab': =C2=A0 filled 2D array, where cell [i][j] i= s bench_one result for
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 test_cases[i] for test_envs[j] (i.e., rows are tes= t cases and
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 columns are test environments)
> >> + =C2=A0 =C2= =A0"""
> >> + =C2=A0 =C2=A0tab =3D {}
> &g= t;> + =C2=A0 =C2=A0results =3D {
> >> + =C2=A0 =C2=A0 =C2=A0= =C2=A0'envs': test_envs,
> >> + =C2=A0 =C2=A0 =C2=A0 = =C2=A0'cases': test_cases,
> >> + =C2=A0 =C2=A0 =C2=A0 = =C2=A0'tab': tab
> >> + =C2=A0 =C2=A0}
> >>= + =C2=A0 =C2=A0n =3D 1
> >> + =C2=A0 =C2=A0n_tests =3D len(tes= t_envs) * len(test_cases)
> >> + =C2=A0 =C2=A0for env in test_e= nvs:
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0for case in test_cases:<= br>> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0print('Test= ing {}/{}: {} :: {}'.format(n, n_tests,
> >> + =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 env['id'], case['id']))
> >> = + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if case['id'] not in tab= :
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0tab[case['id']] =3D {}
> >> + =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0tab[case['id']][env['id']] =3D bench_on= e(test_func, env, case,
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 *args= , **vargs)
> >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0n += =3D 1
> >> +
> >> + =C2=A0 =C2=A0print('Done= 9;)
> >> + =C2=A0 =C2=A0return results
> >> +
&g= t; >> +
> >> +def ascii(results):
> >> + =C2= =A0 =C2=A0"""Return ASCII representation of bench() returned= dict."""
> >> + =C2=A0 =C2=A0from tabulate impo= rt tabulate
> >> +
> >> + =C2=A0 =C2=A0tab =3D [[&q= uot;"] + [c['id'] for c in results['envs']]]
> &= gt;> + =C2=A0 =C2=A0for case in results['cases']:
> >&g= t; + =C2=A0 =C2=A0 =C2=A0 =C2=A0row =3D [case['id']]
> >&g= t; + =C2=A0 =C2=A0 =C2=A0 =C2=A0for env in results['envs']:
>= >> + =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0row.append(ascii_one(r= esults['tab'][case['id']][env['id']]))
> >= > + =C2=A0 =C2=A0 =C2=A0 =C2=A0tab.append(row)
> >> +
>= ; >> + =C2=A0 =C2=A0return tabulate(tab)
> >> --
> = >> 2.21.0
> >>
> >>
>
>
> -= -
> Best regards,
> Vladimir
--000000000000faa7bf059fe58a6b--