On Wed, Sep 18, 2019 at 01:13:29PM +0200, Philippe Mathieu-Daudé wrote:
> On 9/18/19 9:16 AM, David Gibson wrote:
> > Hi,
> > 
> > I'm finding make check-acceptance is currently useless for me as a
> > pre-pull test, because a bunch of the tests are not at all reliable.
> > There are a bunch which I'm still investigating, but for now I'm
> > looking at the MIPS Malta SSH tests.
> > 
> > There seem to be at least two problems here.  First, the test includes
> > a download of a pretty big guest disk image.  This can easily exhaust
> > the 2m30 timeout on its own.
> 
> Gerd raised this issue few months ago:
> 
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg615619.html

Ah, yes indeed.

> > Even without the timeout, it makes the test really slow, even on
> > repeated runs.  Is there some way we can make the image download part
> > of "building" the tests rather than actually running the testsuite, so
> > that a) the test themselves go faster and b) we don't include the
> > download in the test timeout - obviously the download speed is hugely
> > dependent on factors that aren't really related to what we're testing
> > here.
> > 
> > In the meantime, I tried hacking it by just increasing the timeout to
> > 10m.  That got several of the tests working for me, but one still
> > failed.  Specifically 'LinuxSSH.test_mips_malta32eb_kernel3_2_0' still
> > timed out for me, but now after booting the guest, rather than during
> > the image download.  Looking at the avocado log file I'm seeing a
> > bunch of soft lockup messages from the guest console, AFAICT.  So it
> > looks like we have a real bug here, which I suspect has been
> > overlooked precisely because the download problems mean this test
> > isn't reliable.
> > 
> > Any thoughts on how to improve the situation?
> 
> Maybe we should disable this test and run it manually...

Until we can fix it better, I really think we should.  A test this
unreliable verges on worse than useless.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson