All of lore.kernel.org
 help / color / mirror / Atom feed
From: Peter Maydell <peter.maydell@linaro.org>
To: "Alex Bennée" <alex.bennee@linaro.org>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>,
	"QEMU Developers" <qemu-devel@nongnu.org>
Subject: Re: [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test
Date: Thu, 7 May 2020 19:45:01 +0100	[thread overview]
Message-ID: <CAFEAcA800Lhgqa6gOWmA5M2D3+Xj0hYcJw9Tkget4yvW5fFs+A@mail.gmail.com> (raw)
In-Reply-To: <874ksr615d.fsf@linaro.org>

On Thu, 7 May 2020 at 19:38, Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
>
> > The BootLinuxAarch64.test_virt_tcg is reported to take >7min to run.
> > Add a possibility to users to skip this particular test, by setting
> > the AVOCADO_SKIP_SLOW_TESTS environment variable:
> >
> >   $ AVOCADO_SKIP_SLOW_TESTS=please make check-acceptance
> >   ...
> >     (05/88) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg: SKIP: Test takes >7min
> >   ...
>
> 7m = 420s - I'm seeing:
>
>   hyperfine "./tests/venv/bin/avocado run ./tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg"
>   Benchmark #1: ./tests/venv/bin/avocado run ./tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg
>     Time (mean ± σ):     162.179 s ±  3.138 s    [User: 204.726 s, System: 9.663 s]
>     Range (min … max):   158.651 s … 170.036 s    10 runs
>
> Is this on very slow hardware?

Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz 16-core
clang sanitizer build, which probably slows it down a bit.

But even 200 seconds is an order of magnitude slower than any
of the other tests that check-acceptance runs. I think we
should be aiming for tests here to be ~30 seconds at most,
or the whole thing will take forever by the time we have
decent coverage of most machines.

thanks
-- PMM


  reply	other threads:[~2020-05-07 18:46 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-07 16:22 [PATCH] tests/acceptance/boot_linux: Skip slow Aarch64 'virt' machine TCG test Philippe Mathieu-Daudé
2020-05-07 18:38 ` Alex Bennée
2020-05-07 18:45   ` Peter Maydell [this message]
2020-05-07 20:32     ` Alex Bennée
2020-05-08 13:07       ` Philippe Mathieu-Daudé
2020-05-09 13:18 ` Lukas Straub
2020-05-11  9:11   ` Peter Maydell
2020-05-11 16:51     ` Lukas Straub
2020-05-26  9:14       ` Philippe Mathieu-Daudé

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAFEAcA800Lhgqa6gOWmA5M2D3+Xj0hYcJw9Tkget4yvW5fFs+A@mail.gmail.com \
    --to=peter.maydell@linaro.org \
    --cc=alex.bennee@linaro.org \
    --cc=philmd@redhat.com \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.