All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ahmed Karaman <ahmedkhaledkaraman@gmail.com>
To: "Lukáš Doktor" <ldoktor@redhat.com>
Cc: "Aleksandar Markovic" <aleksandar.qemu.devel@gmail.com>,
	"Alex Bennée" <alex.bennee@linaro.org>,
	"QEMU Developers" <qemu-devel@nongnu.org>,
	"Richard Henderson" <rth@twiddle.net>
Subject: Re: [REPORT] [GSoC - TCG Continuous Benchmarking] [#2] Dissecting QEMU Into Three Main Parts
Date: Tue, 30 Jun 2020 09:18:34 +0200	[thread overview]
Message-ID: <CALTWKrUOjwi9xG7vXCQ9sXStFaPh782ytwwPbztAp_PNuHFHYA@mail.gmail.com> (raw)
In-Reply-To: <5ff8e75e-787e-a34d-2b3c-da63e2e46e1f@redhat.com>

On Tue, Jun 30, 2020 at 6:34 AM Lukáš Doktor <ldoktor@redhat.com> wrote:
>
> Dne 29. 06. 20 v 12:25 Ahmed Karaman napsal(a):
> > Hi,
> >
> > The second report of the TCG Continuous Benchmarking series builds
> > upon the QEMU performance metrics calculated in the previous report.
> > This report presents a method to dissect the number of instructions
> > executed by a QEMU invocation into three main phases:
> > - Code Generation
> > - JIT Execution
> > - Helpers Execution
> > It devises a Python script that automates this process.
> >
> > After that, the report presents an experiment for comparing the
> > output of running the script on 17 different targets. Many conclusions
> > can be drawn from the results and two of them are discussed in the
> > analysis section.
> >
> > Report link:
> > https://ahmedkrmn.github.io/TCG-Continuous-Benchmarking/Dissecting-QEMU-Into-Three-Main-Parts/
> >
> > Previous reports:
> > Report 1 - Measuring Basic Performance Metrics of QEMU:
> > https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06692.html
> >
> > Best regards,
> > Ahmed Karaman
>
> Hello Ahmed,
>
> very nice reading, both reports so far. One thing that could be better displayed is the system you used this to generate. This would come handy especially later when you move from examples to actual reports. I think it'd make sense to add a section with a clear definition of the machine as well as the operation system, qemu version and eventually other deps (like compiler, flags, ...). For this report something like:
>
> architecture: x86_64
> cpu_codename: Kaby Lake
> cpu: i7-8650U
> ram: 32GB DDR4
> os: Fedora 32
> qemu: 470dd165d152ff7ceac61c7b71c2b89220b3aad7
> compiler: gcc-10.1.1-1.fc32.x86_64
> flags: --target-list="x86_64-softmmu,ppc64-softmmu,aarch64-softmmu,s390x-softmmu,riscv64-softmmu" --disable-werror --disable-sparse --enable-sdl --enable-kvm  --enable-vhost-net --enable-vhost-net --enable-attr  --enable-kvm  --enable-fdt   --enable-vnc --enable-seccomp --block-drv-rw-whitelist="vmdk,null-aio,quorum,null-co,blkverify,file,nbd,raw,blkdebug,host_device,qed,nbd,iscsi,gluster,rbd,qcow2,throttle,copy-on-read" --python=/usr/bin/python3 --enable-linux-io-uring
>
> would do. Maybe it'd be even a good idea to create a script to report this basic set of information and add it after each of the perf scripts so people don't forget to double-check the conditions, but others might disagree so take this only as a suggestion.
>
> Regards,
> Lukáš
>
> PS: Automated cpu codenames, hosts OSes and such could be tricky, but one can use other libraries or just best-effort-approach with fallback to "unknown" to let people filling it manually or adding their branch to your script.
>
> Regards,
> Lukáš
>
Thanks Mr. Lukáš, I'm really glad you found both reports interesting.

Both reports are based on QEMU version 5.0.0, this wasn't mentioned in
the reports so thanks for the reminder. I'll add a short note about
that.

The used QEMU build is a very basic GCC build (created by just running
../configure in the build directory without any flags).

Regarding the detailed machine information (CPU, RAM ... etc), The two
reports introduce some concepts and methodologies that will produce
consistent results on whichever machine they are executed on. So I
think it's unnecessary to mention the detailed system information used
in the reports for now.

Best regards,
Ahmed Karaman


  reply	other threads:[~2020-06-30  7:20 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-29 10:25 [REPORT] [GSoC - TCG Continuous Benchmarking] [#2] Dissecting QEMU Into Three Main Parts Ahmed Karaman
2020-06-29 10:40 ` Aleksandar Markovic
2020-06-29 14:26   ` Ahmed Karaman
2020-06-29 16:03 ` Alex Bennée
2020-06-29 18:21   ` Aleksandar Markovic
2020-06-29 21:16   ` Ahmed Karaman
2020-07-01 13:44   ` Ahmed Karaman
2020-07-01 15:42     ` Alex Bennée
2020-07-01 17:47       ` Ahmed Karaman
2020-07-03 22:46       ` Aleksandar Markovic
2020-07-04  8:45         ` Alex Bennée
2020-07-04  9:19           ` Aleksandar Markovic
2020-07-04  9:55           ` Aleksandar Markovic
2020-07-04 17:10           ` Ahmed Karaman
2020-06-30  4:33 ` Lukáš Doktor
2020-06-30  7:18   ` Ahmed Karaman [this message]
2020-06-30  8:58     ` Aleksandar Markovic
2020-06-30 12:46       ` Lukáš Doktor
2020-06-30 19:14         ` Ahmed Karaman
2020-06-30  9:41   ` Aleksandar Markovic
2020-06-30 12:58     ` Lukáš Doktor
2020-06-30  5:59 ` 罗勇刚(Yonggang Luo)
2020-06-30  7:29   ` Ahmed Karaman
2020-06-30  8:21     ` Aleksandar Markovic
2020-06-30  9:52       ` Aleksandar Markovic
2020-06-30 19:02         ` Ahmed Karaman
2020-07-01 14:47 ` Ahmed Karaman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALTWKrUOjwi9xG7vXCQ9sXStFaPh782ytwwPbztAp_PNuHFHYA@mail.gmail.com \
    --to=ahmedkhaledkaraman@gmail.com \
    --cc=aleksandar.qemu.devel@gmail.com \
    --cc=alex.bennee@linaro.org \
    --cc=ldoktor@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=rth@twiddle.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.