All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Christopherson, Sean J" <sean.j.christopherson@intel.com>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Po-Hsu Lin <po-hsu.lin@canonical.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>
Subject: RE: [kvm-unit-tests PATCHv2] unittests.cfg: Increase timeout for apic test
Date: Mon, 19 Oct 2020 16:37:47 +0000	[thread overview]
Message-ID: <a6e33cd7d0084d6389a02786225db0e8@intel.com> (raw)
In-Reply-To: <87o8ky4fkf.fsf@vitty.brq.redhat.com>

On Mon, Oct 19, 2020 at 01:32:00PM +0200, Vitaly Kuznetsov wrote:
> Sean Christopherson <sean.j.christopherson@intel.com> writes:
> >> > diff --git a/x86/unittests.cfg b/x86/unittests.cfg
> >> > index 872d679..c72a659 100644
> >> > --- a/x86/unittests.cfg
> >> > +++ b/x86/unittests.cfg
> >> > @@ -41,7 +41,7 @@ file = apic.flat
> >> >  smp = 2
> >> >  extra_params = -cpu qemu64,+x2apic,+tsc-deadline
> >> >  arch = x86_64
> >> > -timeout = 30
> >> > +timeout = 240
> >> >
> >> >  [ioapic]
> >> >  file = ioapic.flat
> >>
> >> AFAIR the default timeout for tests where timeout it not set explicitly
> >> is 90s so don't you need to also modify it for other tests like
> >> 'apic-split', 'ioapic', 'ioapic-split', ... ?
> >>
> >> I was thinking about introducing a 'timeout multiplier' or something to
> >> run_tests.sh for running in slow (read: nested) environments, doing that
> >> would allow us to keep reasonably small timeouts by default. This is
> >> somewhat important as tests tend to hang and waiting for 4 minutes every
> >> time is not great.
> >
> > I would much prefer to go in the other direction and make tests like APIC not
> > do so many loops (in a nested environment?). The port80 test in particular is
> > an absolute waste of time.
>
> I don't think these two suggestions are opposite. Yes, making tests run fast
> is good, however, some of the tests are doomed to be slow. E.g. running
> VMX testsuite while nested (leaving aside the question about who needs
> three level nesting) is always going to be much slower than on bare metal.

Ya, I was specifically referring to tests that arbitrarily choose a high loop
count, without any real/documented justification for running millions of loops.

> > E.g. does running 1M loops in test_multiple_nmi() really add value versus
> > say 10k or 100k loops?
>
> Oddly enough, I vaguely remember this particular test hanging
> *sometimes* after a few thousand loops but I don't remember any
> details.

Thousands still ain't millions :-D.

IMO, the unit tests should sit between a smoke test and a long running,
intensive stress test, i.e. the default config shouldn't be trying to find
literal one-in-a-million bugs on every run.

  reply	other threads:[~2020-10-19 16:37 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-13  9:12 [kvm-unit-tests PATCHv2] unittests.cfg: Increase timeout for apic test Po-Hsu Lin
2020-10-15 15:59 ` Vitaly Kuznetsov
2020-10-15 16:35   ` Sean Christopherson
2020-10-16 17:02     ` Paolo Bonzini
2020-10-16 17:40       ` Andrew Jones
2020-10-20  5:53         ` Thomas Huth
2020-10-20  8:49           ` Andrew Jones
2020-10-19 11:32     ` Vitaly Kuznetsov
2020-10-19 16:37       ` Christopherson, Sean J [this message]
2020-10-19 16:52         ` Nadav Amit
2020-10-20  8:48           ` Paolo Bonzini
2020-10-20 15:05             ` Christopherson, Sean J
2020-10-30  8:15   ` Po-Hsu Lin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a6e33cd7d0084d6389a02786225db0e8@intel.com \
    --to=sean.j.christopherson@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=pbonzini@redhat.com \
    --cc=po-hsu.lin@canonical.com \
    --cc=vkuznets@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.