Re: [kvm-unit-tests PATCHv2] unittests.cfg: Increase timeout for apic test

From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Sean Christopherson <sean.j.christopherson@intel.com>
Cc: Po-Hsu Lin <po-hsu.lin@canonical.com>,
	kvm@vger.kernel.org, pbonzini@redhat.com
Subject: Re: [kvm-unit-tests PATCHv2] unittests.cfg: Increase timeout for apic test
Date: Mon, 19 Oct 2020 13:32:00 +0200	[thread overview]
Message-ID: <87o8ky4fkf.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20201015163539.GA27813@linux.intel.com>

Sean Christopherson <sean.j.christopherson@intel.com> writes:

> On Thu, Oct 15, 2020 at 05:59:52PM +0200, Vitaly Kuznetsov wrote:
>> Po-Hsu Lin <po-hsu.lin@canonical.com> writes:
>> 
>> > We found that on Azure cloud hyperv instance Standard_D48_v3, it will
>> > take about 45 seconds to run this apic test.
>> >
>> > It takes even longer (about 150 seconds) to run inside a KVM instance
>> > VM.Standard2.1 on Oracle cloud.
>> >
>> > Bump the timeout threshold to give it a chance to finish.
>> >
>> > Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com>
>> > ---
>> >  x86/unittests.cfg | 2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/x86/unittests.cfg b/x86/unittests.cfg
>> > index 872d679..c72a659 100644
>> > --- a/x86/unittests.cfg
>> > +++ b/x86/unittests.cfg
>> > @@ -41,7 +41,7 @@ file = apic.flat
>> >  smp = 2
>> >  extra_params = -cpu qemu64,+x2apic,+tsc-deadline
>> >  arch = x86_64
>> > -timeout = 30
>> > +timeout = 240
>> >  
>> >  [ioapic]
>> >  file = ioapic.flat
>> 
>> AFAIR the default timeout for tests where timeout it not set explicitly
>> is 90s so don't you need to also modify it for other tests like
>> 'apic-split', 'ioapic', 'ioapic-split', ... ?
>> 
>> I was thinking about introducing a 'timeout multiplier' or something to
>> run_tests.sh for running in slow (read: nested) environments, doing that
>> would allow us to keep reasonably small timeouts by default. This is
>> somewhat important as tests tend to hang and waiting for 4 minutes every
>> time is not great.
>
> I would much prefer to go in the other direction and make tests like APIC not
> do so many loops (in a nested environment?). The port80 test in particular is
> an absolute waste of time.

I don't think these two suggestions are opposite. Yes, making tests run fast
is good, however, some of the tests are doomed to be slow. E.g. running
VMX testsuite while nested (leaving aside the question about who needs
three level nesting) is always going to be much slower than on bare metal.

>
> E.g. does running 1M loops in test_multiple_nmi() really add value versus
> say 10k or 100k loops?

Oddly enough, I vaguely remember this particular test hanging
*sometimes* after a few thousand loops but I don't remember any
details.

-- 
Vitaly