Re: [Qemu-devel] [PATCH 06/12] docker: Add basic test

From: Fam Zheng <famz@redhat.com>
To: "Peter Maydell" <peter.maydell@linaro.org>,
	"Alex Bennée" <alex.bennee@linaro.org>
Cc: Kevin Wolf <kwolf@redhat.com>, Stefan Weil <sw@weilnetz.de>,
	QEMU Developers <qemu-devel@nongnu.org>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>, John Snow <jsnow@redhat.com>,
	David Gibson <david@gibson.dropbear.id.au>
Subject: Re: [Qemu-devel] [PATCH 06/12] docker: Add basic test
Date: Tue, 16 Feb 2016 09:15:06 +0800	[thread overview]
Message-ID: <20160216011506.GA18664@ad.usersys.redhat.com> (raw)
In-Reply-To: <87si0um3fc.fsf@linaro.org>

On Mon, 02/15 14:52, Alex Bennée wrote:
> 
> Peter Maydell <peter.maydell@linaro.org> writes:
> 
> > On 15 February 2016 at 14:34, Alex Bennée <alex.bennee@linaro.org> wrote:
> >>
> >> Fam Zheng <famz@redhat.com> writes:
> >>> +cd $(mktemp -d)
> >>> +mkdir build
> >>> +mkdir install
> >>> +cd build
> >>> +build_qemu --target-list=x86_64-softmmu --prefix="${pwd}/install"
> >
> > Why restrict the target list ?

There is no particular reason, just I wanted to keep the "basic test" basic.
I'll rename it to "quick".

That said, this is only one of many possible tests, multiplied by the number of
images we ship. We need to be careful with the matrix, because the point of
having these tests is to run them frequently (either manually by developers or
automatically by CI on every patch series or even every patch). 10+ minutes for
a single test/image combination just makes that hard.

> >
> >>> +make check $MAKEFLAGS
> >>> +make install
> >>
> >> From my excursions last week into the tests directory I discovered the
> >> unit tests are built for a number of different qtest binaries. For
> >> completeness we should probably include the whole list:
> >>
> >> #+name: qtest-targets
> >> #+begin_src sh :dir ~/lsrc/qemu/qemu.git :results scalar
> >> grep -E "check-qtest-[[:alnum:]]+-y " tests/Makefile | cut -d " " -f 1 | sort -u
> >> #+end_src
> >>
> >> #+RESULTS: qtest-targets
> >> #+begin_example
> >> check-qtest-arm-y
> >> check-qtest-generic-y
> >> check-qtest-i386-y
> >> check-qtest-ipack-y
> >> check-qtest-microblazeel-y
> >> check-qtest-mips64el-y
> >> check-qtest-mips64-y
> >> check-qtest-mips-y
> >> check-qtest-pci-y
> >> check-qtest-ppc64-y
> >> check-qtest-ppc-y
> >> check-qtest-sh4eb-y
> >> check-qtest-sh4-y
> >> #check-qtest-sparc64-y
> >> check-qtest-sparc64-y
> >> #check-qtest-sparc-y
> >> check-qtest-virtioserial-y
> >> check-qtest-virtio-y
> >> check-qtest-x86_64-y
> >> check-qtest-xtensaeb-y
> >> check-qtest-y
> >> #+end_example
> >
> > I'm having difficulty figuring out what you're proposing here,
> > but it looks like you're suggesting listing all the check-qtest-*
> > test names again here, which seems worth avoiding. We should
> > just do a build and make check and let that take care of
> > running all the tests.
> 
> Well there is a halfway house between building one target and building
> all possible targets. Not all the softmmu targets include additional
> tests. Having said that it is probably simpler as you say to just build
> everything.

Sure, but I'd create a separate "full" test for that.

Fam