Re: [RFC 1/1] acceptance tests: rename acceptance to system

From: Willian Rampazzo <wrampazz@redhat.com>
To: Peter Maydell <peter.maydell@linaro.org>
Cc: "Thomas Huth" <thuth@redhat.com>,
	"Philippe Mathieu-Daudé" <f4bug@amsat.org>,
	"Wainer dos Santos Moschetta" <wainersm@redhat.com>,
	qemu-devel <qemu-devel@nongnu.org>,
	"Niek Linnenbank" <nieklinnenbank@gmail.com>,
	qemu-arm <qemu-arm@nongnu.org>,
	"Michael Rolnik" <mrolnik@gmail.com>,
	"Cleber Rosa" <crosa@redhat.com>,
	"Alex Bennée" <alex.bennee@linaro.org>
Subject: Re: [RFC 1/1] acceptance tests: rename acceptance to system
Date: Fri, 21 May 2021 12:34:25 -0300	[thread overview]
Message-ID: <CAKJDGDZckVS1ZgBPpJptSqFGhYT6FQsid+cTFcxoJCiz7q=yxg@mail.gmail.com> (raw)
In-Reply-To: <CAFEAcA8kc=x8KTtW+Qg1wgdPNSvN-_13FEbmF2TPK8xC5q-GfQ@mail.gmail.com>

On Fri, May 21, 2021 at 12:23 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Fri, 21 May 2021 at 16:13, Willian Rampazzo <wrampazz@redhat.com> wrote:
> > On Fri, May 21, 2021 at 11:29 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > > This does raise the question of what we're actually trying
> > > to distinguish. It seems to me somewhat that what tests/acceptance/
> > > actually contains that makes it interestingly different from other
> > > tests/ stuff is that it's specifically "tests using the Avocado
> > > framework". On that theory we might name it tests/avocado/.
> >
> > I think the updated README.rst from this RFC, inside the system
> > (originally acceptance) folder, is a good description of what these
> > tests should be: "This directory contains system tests. They're
> > usually higher level, and may interact with external resources and
> > with various guest operating systems." I can improve it, if needed.
> >
> > We are using Avocado Framework as a tool to accomplish the above
> > description, but I don't think we should strictly use it if there is
> > another way to accomplish what those tests are supposed to be. Calling
> > them "avocado" tests may restrict the intent of them, in my opinion.
>
> But the main reason IMHO that we have them in a separate directory is
> because that's where we have all the avocado machinery. I think the
> sharing of the machinery is what dictates whether a test winds up in
> tests/acceptance, tests/qtest, tests/tcg or tests/qemu-iotests
> much more than whether it is "usually higher level" or more of a
> unit test or whatever. If we ever added some other test framework for
> doing system tests, we'd probably want to put it in its own
> directory rather than lumping all its support machinery and
> build files in together with the avocado based tests.

Okay, I understand your point. With the organization of tests under
the test folder that QEMU has today, I agree with you.

>
> thanks
> -- PMM
>