Re: "make check-acceptance" takes way too long

[Peter Maydell <peter.maydell@linaro.org>]

On Sat, 31 Jul 2021 at 19:43, Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Peter Maydell <peter.maydell@linaro.org> writes:
>
> > "make check-acceptance" takes way way too long. I just did a run
> > on an arm-and-aarch64-targets-only debug build and it took over
> > half an hour, and this despite it skipping or cancelling 26 out
> > of 58 tests!
> >
> > I think that ~10 minutes runtime is reasonable. 30 is not;
> > ideally no individual test would take more than a minute or so.
> >
> > Output saying where the time went. The first two tests take
> > more than 10 minutes *each*. I think a good start would be to find
> > a way of testing what they're testing that is less heavyweight.
> >
> >  (01/58) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv2:
> > PASS (629.74 s)
> >  (02/58) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg_gicv3:
> > PASS (628.75 s)
> >  (03/58) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm:
> > CANCEL: kvm accelerator does not seem to be available (1.18 s)
>
> For these tests which purport to exercise the various GIC configurations
> I think we would be much better served by running kvm-unit-tests which
> at least try and exercise all the features rather than rely on the side
> effect of booting an entire OS.

I think "can we boot Linux via UEFI?" is worth testing, as is
"can we boot Linux and do at least some stuff in userspace?"
(there's a lot of TCG that doesn't get exercised by pure kernel boot).
We just need to find a guest OS that isn't so overweight it takes 10
minutes...

-- PMM