Re: [PATCH 16/16] tests/acceptance: Test the MIPSsim machine

["Daniel P. Berrangé" <berrange@redhat.com>]

On Tue, Sep 29, 2020 at 10:38:30AM +0100, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> 
> > +Alex/Daniel
> >
> > On 9/28/20 10:33 PM, Willian Rampazzo wrote:
> >> On Mon, Sep 28, 2020 at 2:31 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> >>>
> >>> Add a test for the mipssim machine, based on the recommended
> >>> test setup from Thomas Huth:
> >>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg606846.html
> >>>
> >>> The test is quick and can be run as:
> >>>
> >>>   $ avocado --show=console run -t machine:mipssim tests/acceptance/
> >>>    (1/1) tests/acceptance/machine_mips_mipssim.py:MipsSimMachine.test_mipssim_linux_console:
> >>>   console: Linux version 3.6.11 (root@711bb8ba16a7) (gcc version 4.8.3 (Buildroot 2014.11) ) #2 Sun Sep 27 13:39:35 UTC 2020
> >>>   console: Setting default memory size 0x02000000
> >>>   console: bootconsole [early0] enabled
> >>>   console: CPU revision is: 00019300 (MIPS 24Kc)
> >>>   console: FPU revision is: 00739300
> >>>   ...
> >>>   console: CPU frequency 12.00 MHz
> >>>   console: Calibrating delay loop... 950.27 BogoMIPS (lpj=4751360)
> >>>   ...
> >>>   console: MIPSNet Ethernet driver. Version: 2007-11-17. (c)2005 MIPS Technologies, Inc.
> >>>   ...
> >>>   console: Welcome to Buildroot
> >>>   console: buildroot login: root
> >>>   console: # root
> >>>   console: -sh: root: not found
> >>>   console: # ping -c 3 10.0.2.2
> >>>   console: PING 10.0.2.2 (10.0.2.2): 56 data bytes
> >>>   console: 64 bytes from 10.0.2.2: seq=0 ttl=255 time=48.231 ms
> >>>   console: 64 bytes from 10.0.2.2: seq=1 ttl=255 time=9.407 ms
> >>>   console: 64 bytes from 10.0.2.2: seq=2 ttl=255 time=2.298 ms
> >>>   console: --- 10.0.2.2 ping statistics ---
> >>>   console: 3 packets transmitted, 3 packets received, 0% packet loss
> >>>   PASS (7.99 s)
> >>>
> >>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>> ---
> >>> Cc: Thomas Huth <huth@tuxfamily.org>
> >>> ---
> >>>  MAINTAINERS                              |  1 +
> >>>  tests/acceptance/machine_mips_mipssim.py | 56 ++++++++++++++++++++++++
> >>>  2 files changed, 57 insertions(+)
> >>>  create mode 100644 tests/acceptance/machine_mips_mipssim.py
> >>>
> >>> diff --git a/MAINTAINERS b/MAINTAINERS
> >>> index 5eed1e692b4..17d8a012b0e 100644
> >>> --- a/MAINTAINERS
> >>> +++ b/MAINTAINERS
> >>> @@ -240,6 +240,7 @@ F: include/hw/misc/mips_*
> >>>  F: include/hw/timer/mips_gictimer.h
> >>>  F: tests/acceptance/linux_ssh_mips_malta.py
> >>>  F: tests/acceptance/machine_mips_malta.py
> >>> +F: tests/acceptance/machine_mips_mipssim.py
> >>>  F: tests/tcg/mips/
> >>>  K: ^Subject:.*(?i)mips
> >>>
> >>> diff --git a/tests/acceptance/machine_mips_mipssim.py b/tests/acceptance/machine_mips_mipssim.py
> >>> new file mode 100644
> >>> index 00000000000..b2749917b08
> >>> --- /dev/null
> >>> +++ b/tests/acceptance/machine_mips_mipssim.py
> >>> @@ -0,0 +1,56 @@
> >>> +# Functional tests for the MIPS simulator (MIPSsim machine)
> >>> +#
> >>> +# Copyright (c) 2020 Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>> +#
> >>> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> >>> +# See the COPYING file in the top-level directory.
> >>> +#
> >>> +# SPDX-License-Identifier: GPL-2.0-or-later
> >>> +
> >>> +import os
> >>> +import logging
> >>> +import time
> >>> +
> >>> +from avocado import skipUnless
> >>> +from avocado_qemu import Test
> >>> +from avocado_qemu import exec_command_and_wait_for_pattern
> >>> +from avocado_qemu import interrupt_interactive_console_until_pattern
> >>> +from avocado_qemu import wait_for_console_pattern
> >>> +
> >>> +class MipsSimMachine(Test):
> >>> +
> >>> +    timeout = 30
> >>> +    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
> >>> +
> >>> +    @skipUnless(os.getenv('AVOCADO_ALLOW_UNTRUSTED_CODE'), 'untrusted code')
> >>> +    def test_mipssim_linux_console(self):
> >>> +        """
> >>> +        Boots the Linux kernel and checks that the console is operational
> >>> +        :avocado: tags=arch:mipsel
> >>> +        :avocado: tags=machine:mipssim
> >>> +        :avocado: tags=device:mipsnet
> >>> +        """
> >>> +        kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
> >>> +                      '32ea5764e1de8fffa0d59366c44822cd06d7c8e0/'
> >>> +                      'mips/mipssim/mipsel/vmlinux')
> >> 
> >> So, are you willing to maintain some images on your GitHub to avoid
> >> the image changes when they are not found?
> >
> > No, I am not willing to do that.
> >
> > But I see it pointless to have everyone (including CI) to spend 1h
> > building this image, when the sources and build recipe is available,
> > making the built image reproducible.
> 
> I agree we don't want to build from scratch each time. However as we
> move from relying on third parties (which outside of the major distros
> has proved very flaky) we are getting to the point of managing our GPL
> liabilities.

I'm wondering what the 1 hour time to build the images is
spent on, and what does the image need to actually do ?

Is it anything more than boot the kernel successfully ?

I've not tried a timed build, but surely a minimal / tiny
Linx kernel build is much less than an hour ? Combine with
a tiny C program that simply prints "OK" as /bin/init and
powers off the VM. Or we need more advanced then a minimal
busybox build.

I think it'd be great if we could self-build a minimal
kernel+init for all platforms we target, using distro
provided cross-compilers. 


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|