Re: [RFC 1/1] acceptance tests: rename acceptance to system

[Peter Maydell <peter.maydell@linaro.org>]

On Fri, 21 May 2021 at 16:13, Willian Rampazzo <wrampazz@redhat.com> wrote:
> On Fri, May 21, 2021 at 11:29 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > This does raise the question of what we're actually trying
> > to distinguish. It seems to me somewhat that what tests/acceptance/
> > actually contains that makes it interestingly different from other
> > tests/ stuff is that it's specifically "tests using the Avocado
> > framework". On that theory we might name it tests/avocado/.
>
> I think the updated README.rst from this RFC, inside the system
> (originally acceptance) folder, is a good description of what these
> tests should be: "This directory contains system tests. They're
> usually higher level, and may interact with external resources and
> with various guest operating systems." I can improve it, if needed.
>
> We are using Avocado Framework as a tool to accomplish the above
> description, but I don't think we should strictly use it if there is
> another way to accomplish what those tests are supposed to be. Calling
> them "avocado" tests may restrict the intent of them, in my opinion.

But the main reason IMHO that we have them in a separate directory is
because that's where we have all the avocado machinery. I think the
sharing of the machinery is what dictates whether a test winds up in
tests/acceptance, tests/qtest, tests/tcg or tests/qemu-iotests
much more than whether it is "usually higher level" or more of a
unit test or whatever. If we ever added some other test framework for
doing system tests, we'd probably want to put it in its own
directory rather than lumping all its support machinery and
build files in together with the avocado based tests.

thanks
-- PMM