[PATCH v4 0/7] hw/arm/raspi: Run U-Boot on the raspi machines

[PATCH v4 0/7] hw/arm/raspi: Run U-Boot on the raspi machines

[Philippe Mathieu-Daudé]

Following Laurent report:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg639950.html

The SYS_timer is already merged, see:
https://git.qemu.org/?p=qemu.git;a=commit;h=d05be883fc
"hw/timer/bcm2835: Add the BCM2835 SYS_timer"

The first patch should fix Laurent other issue.
Then few python patches are require to break into U-Boot console,
and the last patches add U-Boot tests for Raspi2 and Raspi3.

Laurent, if you successfully test U-Boot with this patchset again,
do you mind replying with a "Tested-by:" tag?

Regards,

Phil.

Since v3:
- rewrote '-smp' fix.
- tests use Debian 'trustable' u-boot.elf

previous feedbacks from Peter on v3:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg655415.html

v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg653807.html
Supersedes: <20191019234715.25750-1-f4bug@amsat.org>

Philippe Mathieu-Daudé (7):
  hw/arm/raspi: Remove obsolete use of -smp to set the soc
    'enabled-cpus'
  Acceptance tests: Extract _console_interaction()
  Acceptance tests: Add interrupt_interactive_console_until_pattern()
  python/qemu/machine: Allow to use other serial consoles than default
  tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2
  tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 3
  tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'

 hw/arm/raspi.c                            |  2 -
 python/qemu/machine.py                    |  9 +++-
 tests/acceptance/avocado_qemu/__init__.py | 59 +++++++++++++++++------
 tests/acceptance/boot_linux_console.py    | 54 +++++++++++++++++++++
 4 files changed, 107 insertions(+), 17 deletions(-)

-- 
2.21.1

[PATCH v4 1/7] hw/arm/raspi: Remove obsolete use of -smp to set the soc 'enabled-cpus'

[Philippe Mathieu-Daudé]

Since we enabled parallel TCG code generation for softmmu (see
commit 3468b59 "tcg: enable multiple TCG contexts in softmmu")
and its subsequent fix (commit 72649619 "add .min_cpus and
.default_cpus fields to machine_class"), the raspi machines are
restricted to always use their 4 cores:

See in hw/arm/raspi2 (with BCM283X_NCPUS set to 4):

  222 static void raspi2_machine_init(MachineClass *mc)
  223 {
  224     mc->desc = "Raspberry Pi 2";
  230     mc->max_cpus = BCM283X_NCPUS;
  231     mc->min_cpus = BCM283X_NCPUS;
  232     mc->default_cpus = BCM283X_NCPUS;
  235 };
  236 DEFINE_MACHINE("raspi2", raspi2_machine_init)

We can no longer use the -smp option, as we get:

  $ qemu-system-arm -M raspi2 -smp 1
  qemu-system-arm: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi2' is 4

Since we can not set the TYPE_BCM283x SOC "enabled-cpus" with -smp,
remove the unuseful code.

We can achieve the same by using the '-global bcm2836.enabled-cpus=1'
option.

Reported-by: Laurent Bonnans <laurent.bonnans@here.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
Cc: Emilio G. Cota <cota@braap.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Andrew Baumann <Andrew.Baumann@microsoft.com>
Cc: Eduardo Habkost <ehabkost@redhat.com>
---
 hw/arm/raspi.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c
index 6a510aafc1..3996f6c63a 100644
--- a/hw/arm/raspi.c
+++ b/hw/arm/raspi.c
@@ -192,8 +192,6 @@ static void raspi_init(MachineState *machine, int version)
     /* Setup the SOC */
     object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram),
                                    &error_abort);
-    object_property_set_int(OBJECT(&s->soc), machine->smp.cpus, "enabled-cpus",
-                            &error_abort);
     int board_rev = version == 3 ? 0xa02082 : 0xa21041;
     object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev",
                             &error_abort);
-- 
2.21.1

[PATCH v4 2/7] Acceptance tests: Extract _console_interaction()

[Philippe Mathieu-Daudé]

Since we are going to re-use the code shared between
wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
extract the common part into a local function.

Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/acceptance/avocado_qemu/__init__.py | 31 +++++++++++++----------
 1 file changed, 17 insertions(+), 14 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 6618ea67c1..0a50fcf2be 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -55,19 +55,14 @@ def pick_default_qemu_bin(arch=None):
         return qemu_bin_from_src_dir_path
 
 
-def wait_for_console_pattern(test, success_message, failure_message=None):
-    """
-    Waits for messages to appear on the console, while logging the content
-
-    :param test: an Avocado test containing a VM that will have its console
-                 read and probed for a success or failure message
-    :type test: :class:`avocado_qemu.Test`
-    :param success_message: if this message appears, test succeeds
-    :param failure_message: if this message appears, test fails
-    """
+def _console_interaction(test, success_message, failure_message,
+                         send_string):
     console = test.vm.console_socket.makefile()
     console_logger = logging.getLogger('console')
     while True:
+        if send_string:
+            test.vm.console_socket.sendall(send_string.encode())
+            send_string = None # send only once
         msg = console.readline().strip()
         if not msg:
             continue
@@ -79,6 +74,17 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
             fail = 'Failure message found in console: %s' % failure_message
             test.fail(fail)
 
+def wait_for_console_pattern(test, success_message, failure_message=None):
+    """
+    Waits for messages to appear on the console, while logging the content
+
+    :param test: an Avocado test containing a VM that will have its console
+                 read and probed for a success or failure message
+    :type test: :class:`avocado_qemu.Test`
+    :param success_message: if this message appears, test succeeds
+    :param failure_message: if this message appears, test fails
+    """
+    _console_interaction(test, success_message, failure_message, None)
 
 def exec_command_and_wait_for_pattern(test, command,
                                       success_message, failure_message=None):
@@ -94,10 +100,7 @@ def exec_command_and_wait_for_pattern(test, command,
     :param success_message: if this message appears, test succeeds
     :param failure_message: if this message appears, test fails
     """
-    command += '\r'
-    test.vm.console_socket.sendall(command.encode())
-    wait_for_console_pattern(test, success_message, failure_message)
-
+    _console_interaction(test, success_message, failure_message, command + '\r')
 
 class Test(avocado.Test):
     def _get_unique_tag_val(self, tag_name):
-- 
2.21.1

[PATCH v4 3/7] Acceptance tests: Add interrupt_interactive_console_until_pattern()

[Philippe Mathieu-Daudé]

We need a function to interrupt interactive consoles.

Example: Interrupt U-Boot to set different environment values.

Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/acceptance/avocado_qemu/__init__.py | 32 +++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
index 0a50fcf2be..d4358eb431 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -56,13 +56,15 @@ def pick_default_qemu_bin(arch=None):
 
 
 def _console_interaction(test, success_message, failure_message,
-                         send_string):
+                         send_string, keep_sending=False):
+    assert not keep_sending or send_string
     console = test.vm.console_socket.makefile()
     console_logger = logging.getLogger('console')
     while True:
         if send_string:
             test.vm.console_socket.sendall(send_string.encode())
-            send_string = None # send only once
+            if not keep_sending:
+                send_string = None # send only once
         msg = console.readline().strip()
         if not msg:
             continue
@@ -74,6 +76,32 @@ def _console_interaction(test, success_message, failure_message,
             fail = 'Failure message found in console: %s' % failure_message
             test.fail(fail)
 
+def interrupt_interactive_console_until_pattern(test, success_message,
+                                                failure_message=None,
+                                                interrupt_string='\r'):
+    """
+    Keep sending a string to interrupt a console prompt, while logging the
+    console output. Typical use case is to break a boot loader prompt, such:
+
+        Press a key within 5 seconds to interrupt boot process.
+        5
+        4
+        3
+        2
+        1
+        Booting default image...
+
+    :param test: an Avocado test containing a VM that will have its console
+                 read and probed for a success or failure message
+    :type test: :class:`avocado_qemu.Test`
+    :param success_message: if this message appears, test succeeds
+    :param failure_message: if this message appears, test fails
+    :param interrupt_string: a string to send to the console before trying
+                             to read a new line
+    """
+    _console_interaction(test, success_message, failure_message,
+                         interrupt_string, True)
+
 def wait_for_console_pattern(test, success_message, failure_message=None):
     """
     Waits for messages to appear on the console, while logging the content
-- 
2.21.1

[PATCH v4 4/7] python/qemu/machine: Allow to use other serial consoles than default

[Philippe Mathieu-Daudé]

Currently the QEMU Python module limits the QEMUMachine class to
use the first serial console.

Some machines/guest might use another console than the first one as
the 'boot console'. For example the Raspberry Pi uses the second
(AUX) console.

To be able to use the Nth console as default, we simply need to
connect all the N - 1 consoles to the null chardev.

Add an index argument, so we can use a specific serial console as
default.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
v2:
- renamed 'console_index', added docstring (Cleber)
- reworded description (pm215)
---
 python/qemu/machine.py | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 734efd8536..ef9f5b213f 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -241,6 +241,8 @@ class QEMUMachine(object):
                          'chardev=mon,mode=control'])
         if self._machine is not None:
             args.extend(['-machine', self._machine])
+        for i in range(self._console_index):
+            args.extend(['-serial', 'null'])
         if self._console_set:
             self._console_address = os.path.join(self._sock_dir,
                                                  self._name + "-console.sock")
@@ -527,7 +529,7 @@ class QEMUMachine(object):
         """
         self._machine = machine_type
 
-    def set_console(self, device_type=None):
+    def set_console(self, device_type=None, console_index=0):
         """
         Sets the device type for a console device
 
@@ -548,9 +550,14 @@ class QEMUMachine(object):
                             chardev:console" command line argument will
                             be used instead, resorting to the machine's
                             default device type.
+        @param console_index: the index of the console device to use.
+                              If not zero, the command line will create
+                              'index - 1' consoles and connect them to
+                              the 'null' backing character device.
         """
         self._console_set = True
         self._console_device_type = device_type
+        self._console_index = console_index
 
     @property
     def console_socket(self):
-- 
2.21.1

[PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Philippe Mathieu-Daudé]

This test runs U-Boot on the Raspberry Pi 2.
It is very simple and fast:

  $ avocado --show=app,console run -t raspi2 -t u-boot tests/acceptance/
  JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot:
  console: MMC:   sdhci@7e300000: 0
  console: Loading Environment from FAT... Card did not respond to voltage select!
  console: In:    serial
  console: Out:   vidconsole
  console: Err:   vidconsole
  console: Net:   No ethernet found.
  console: starting USB...
  console: USB0:   Port not available.
  console: Hit any key to stop autoboot:  0
  console: U-Boot>
  console: U-Boot> bdinfo
  console: arch_number = 0x00000000
  console: boot_params = 0x00000100
  console: DRAM bank   = 0x00000000
  console: -> start    = 0x00000000
  console: -> size     = 0x3c000000
  console: baudrate    = 115200 bps
  console: TLB addr    = 0x3bff0000
  console: relocaddr   = 0x3bf64000
  console: reloc off   = 0x3bf5c000
  console: irq_sp      = 0x3bb5fec0
  console: sp start    = 0x3bb5feb0
  console: Early malloc usage: 2a4 / 400
  console: fdt_blob    = 0x3bfbdfb0
  console: U-Boot> version
  console: U-Boot 2019.01+dfsg-7 (May 14 2019 - 02:07:44 +0000)
  console: gcc (Debian 8.3.0-7) 8.3.0
  console: GNU ld (GNU Binutils for Debian) 2.31.1
  console: U-Boot> reset
  console: resetting ...
  PASS (0.46 s)

U-Boot is built by the Debian project, see:
https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/acceptance/boot_linux_console.py | 28 ++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index e40b84651b..682b801b4f 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -16,6 +16,7 @@ import shutil
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
 from avocado_qemu import wait_for_console_pattern
 from avocado.utils import process
 from avocado.utils import archive
@@ -485,6 +486,33 @@ class BootLinuxConsole(Test):
         exec_command_and_wait_for_pattern(self, 'reboot',
                                                 'reboot: Restarting system')
 
+    def test_arm_raspi2_uboot(self):
+        """
+        :avocado: tags=arch:arm
+        :avocado: tags=machine:raspi2
+        :avocado: tags=u-boot
+        """
+        deb_url = ('https://snapshot.debian.org/archive/debian/'
+                   '20190514T084354Z/pool/main/u/u-boot/'
+                   'u-boot-rpi_2019.01%2Bdfsg-7_armhf.deb')
+        deb_hash = 'ad858cf3afe623b6c3fa2e20dcdd1768fcb9ae83'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+        uboot_path = '/usr/lib/u-boot/rpi_2/uboot.elf'
+        uboot_path = self.extract_from_deb(deb_path, uboot_path)
+
+        self.vm.set_console()
+        self.vm.add_args('-kernel', uboot_path,
+                         # VideoCore starts CPU with only 1 core enabled
+                         '-global', 'bcm2836.enabled-cpus=1',
+                         '-no-reboot')
+        self.vm.launch()
+        interrupt_interactive_console_until_pattern(self,
+                                       'Hit any key to stop autoboot:',
+                                       'Config file not found')
+        exec_command_and_wait_for_pattern(self, 'bdinfo', 'U-Boot')
+        exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
+        exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
+
     def test_s390x_s390_ccw_virtio(self):
         """
         :avocado: tags=arch:s390x
-- 
2.21.1

[PATCH v4 6/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 3

[Philippe Mathieu-Daudé]

This test runs U-Boot on the Raspberry Pi 3.
It is very simple and fast:

  $ avocado --show=app,console run -t raspi3 -t u-boot tests/acceptance/
  JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot:
  console: MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
  console: Loading Environment from FAT... WARNING at drivers/mmc/bcm2835_sdhost.c:410/bcm2835_send_command()!
  console: WARNING at drivers/mmc/bcm2835_sdhost.c:410/bcm2835_send_command()!
  console: Card did not respond to voltage select!
  console: In:    serial
  console: Out:   vidconsole
  console: Err:   vidconsole
  console: Net:   No ethernet found.
  console: starting USB...
  console: Bus usb@7e980000: Port not available.
  console: Hit any key to stop autoboot:  0
  console: U-Boot>
  console: U-Boot>
  console: U-Boot> bdinfo
  console: arch_number = 0x0000000000000000
  console: boot_params = 0x0000000000000100
  console: DRAM bank   = 0x0000000000000000
  console: -> start    = 0x0000000000000000
  console: -> size     = 0x000000003c000000
  console: baudrate    = 115200 bps
  console: TLB addr    = 0x000000003bff0000
  console: relocaddr   = 0x000000003bf57000
  console: reloc off   = 0x000000003bed7000
  console: irq_sp      = 0x000000003bb52dd0
  console: sp start    = 0x000000003bb52dd0
  console: FB base     = 0x0000000000000000
  console: Early malloc usage: 7b0 / 2000
  console: fdt_blob    = 0x000000003bfbf200
  console: U-Boot> version
  console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000)
  console: gcc (Debian 9.2.1-22) 9.2.1 20200104
  console: GNU ld (GNU Binutils for Debian) 2.33.1
  console: U-Boot> reset
  console: resetting ...
  PASS (1.79 s)

U-Boot is built by the Debian project, see:
https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/acceptance/boot_linux_console.py | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 682b801b4f..22b360118d 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -513,6 +513,31 @@ class BootLinuxConsole(Test):
         exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
         exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
 
+    def test_aarch64_raspi3_uboot(self):
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:raspi3
+        :avocado: tags=u-boot
+        """
+        deb_url = ('https://snapshot.debian.org/archive/debian/'
+                   '20200108T145233Z/pool/main/u/u-boot/'
+                   'u-boot-rpi_2020.01%2Bdfsg-1_arm64.deb')
+        deb_hash = 'f394386e02469d52f2eb3c07a2325b1c95aeb00b'
+        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
+        uboot_path = '/usr/lib/u-boot/rpi_3/u-boot.bin'
+        uboot_path = self.extract_from_deb(deb_path, uboot_path)
+
+        self.vm.set_console(console_index=1)
+        self.vm.add_args('-kernel', uboot_path,
+                         '-no-reboot')
+        self.vm.launch()
+        interrupt_interactive_console_until_pattern(self,
+                                       'Hit any key to stop autoboot:',
+                                       'Config file not found')
+        exec_command_and_wait_for_pattern(self, 'bdinfo', 'U-Boot')
+        exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
+        exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
+
     def test_s390x_s390_ccw_virtio(self):
         """
         :avocado: tags=arch:s390x
-- 
2.21.1

[PATCH v4 7/7] tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'

[Philippe Mathieu-Daudé]

Avocado tags are handy to automatically select tests matching
the tags. Since this test also runs U-Boot, tag it.

We can run all the tests using U-Boot as once with:

  $ avocado --show=app run -t u-boot tests/acceptance/
  JOB LOG    : avocado/job-results/job-2020-01-21T00.16-ee9344e/job.log
   (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2: PASS (16.59 s)
   (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot: PASS (0.47 s)
   (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot: PASS (2.43 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 19.78 s

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 tests/acceptance/boot_linux_console.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 22b360118d..4a4cf9d0ea 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -305,6 +305,7 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:arm
         :avocado: tags=machine:emcraft-sf2
         :avocado: tags=endian:little
+        :avocado: tags=u-boot
         """
         uboot_url = ('https://raw.githubusercontent.com/'
                      'Subbaraya-Sundeep/qemu-test-binaries/'
-- 
2.21.1

Re: [PATCH v4 1/7] hw/arm/raspi: Remove obsolete use of -smp to set the soc 'enabled-cpus'

[Alistair Francis]

On Tue, Jan 21, 2020 at 9:53 AM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
>
> Since we enabled parallel TCG code generation for softmmu (see
> commit 3468b59 "tcg: enable multiple TCG contexts in softmmu")
> and its subsequent fix (commit 72649619 "add .min_cpus and
> .default_cpus fields to machine_class"), the raspi machines are
> restricted to always use their 4 cores:
>
> See in hw/arm/raspi2 (with BCM283X_NCPUS set to 4):
>
>   222 static void raspi2_machine_init(MachineClass *mc)
>   223 {
>   224     mc->desc = "Raspberry Pi 2";
>   230     mc->max_cpus = BCM283X_NCPUS;
>   231     mc->min_cpus = BCM283X_NCPUS;
>   232     mc->default_cpus = BCM283X_NCPUS;
>   235 };
>   236 DEFINE_MACHINE("raspi2", raspi2_machine_init)
>
> We can no longer use the -smp option, as we get:
>
>   $ qemu-system-arm -M raspi2 -smp 1
>   qemu-system-arm: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi2' is 4
>
> Since we can not set the TYPE_BCM283x SOC "enabled-cpus" with -smp,
> remove the unuseful code.
>
> We can achieve the same by using the '-global bcm2836.enabled-cpus=1'
> option.
>
> Reported-by: Laurent Bonnans <laurent.bonnans@here.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Alistair Francis <alistair.francis@wdc.com>

Alistair

> ---
> Cc: Emilio G. Cota <cota@braap.org>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: Andrew Baumann <Andrew.Baumann@microsoft.com>
> Cc: Eduardo Habkost <ehabkost@redhat.com>
> ---
>  hw/arm/raspi.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c
> index 6a510aafc1..3996f6c63a 100644
> --- a/hw/arm/raspi.c
> +++ b/hw/arm/raspi.c
> @@ -192,8 +192,6 @@ static void raspi_init(MachineState *machine, int version)
>      /* Setup the SOC */
>      object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram),
>                                     &error_abort);
> -    object_property_set_int(OBJECT(&s->soc), machine->smp.cpus, "enabled-cpus",
> -                            &error_abort);
>      int board_rev = version == 3 ? 0xa02082 : 0xa21041;
>      object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev",
>                              &error_abort);
> --
> 2.21.1
>
>

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Gerd Hoffmann]

On Tue, Jan 21, 2020 at 12:51:57AM +0100, Philippe Mathieu-Daudé wrote:
> This test runs U-Boot on the Raspberry Pi 2.

> U-Boot is built by the Debian project, see:
> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot

We already have a u-boot submodule in roms/

I guess it makes sense to just build & ship our own binaries
instead of downloading them from Debian?

cheers,
  Gerd

Re: [PATCH v4 4/7] python/qemu/machine: Allow to use other serial consoles than default

[Philippe Mathieu-Daudé]

On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> Currently the QEMU Python module limits the QEMUMachine class to
> use the first serial console.
> 
> Some machines/guest might use another console than the first one as
> the 'boot console'. For example the Raspberry Pi uses the second
> (AUX) console.
> 
> To be able to use the Nth console as default, we simply need to
> connect all the N - 1 consoles to the null chardev.
> 
> Add an index argument, so we can use a specific serial console as
> default.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> v2:
> - renamed 'console_index', added docstring (Cleber)
> - reworded description (pm215)
> ---
>   python/qemu/machine.py | 9 ++++++++-
>   1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index 734efd8536..ef9f5b213f 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -241,6 +241,8 @@ class QEMUMachine(object):
>                            'chardev=mon,mode=control'])
>           if self._machine is not None:
>               args.extend(['-machine', self._machine])
> +        for i in range(self._console_index):
> +            args.extend(['-serial', 'null'])

This patch is not sufficient, we have to initialize _console_index in 
__init__():

-- >8 --
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index ef9f5b213f..183d8f3d38 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -112,6 +112,7 @@ class QEMUMachine(object):
          self._sock_dir = sock_dir
          self._launched = False
          self._machine = None
+        self._console_index = 0
          self._console_set = False
          self._console_device_type = None
          self._console_address = None
---

Else for tests not calling self.set_console() we get:

  'QEMUMachine' object has no attribute '_console_index'

>           if self._console_set:
>               self._console_address = os.path.join(self._sock_dir,
>                                                    self._name + "-console.sock")
> @@ -527,7 +529,7 @@ class QEMUMachine(object):
>           """
>           self._machine = machine_type
>   
> -    def set_console(self, device_type=None):
> +    def set_console(self, device_type=None, console_index=0):
>           """
>           Sets the device type for a console device
>   
> @@ -548,9 +550,14 @@ class QEMUMachine(object):
>                               chardev:console" command line argument will
>                               be used instead, resorting to the machine's
>                               default device type.
> +        @param console_index: the index of the console device to use.
> +                              If not zero, the command line will create
> +                              'index - 1' consoles and connect them to
> +                              the 'null' backing character device.
>           """
>           self._console_set = True
>           self._console_device_type = device_type
> +        self._console_index = console_index
>   
>       @property
>       def console_socket(self):
> 

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Philippe Mathieu-Daudé]

On 1/21/20 7:57 AM, Gerd Hoffmann wrote:
> On Tue, Jan 21, 2020 at 12:51:57AM +0100, Philippe Mathieu-Daudé wrote:
>> This test runs U-Boot on the Raspberry Pi 2.
> 
>> U-Boot is built by the Debian project, see:
>> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
> 
> We already have a u-boot submodule in roms/
> 
> I guess it makes sense to just build & ship our own binaries
> instead of downloading them from Debian?

Your comment made me realize I pasted the wrong link, I meant this one:
https://wiki.debian.org/U-boot/Status

Maybe the commit description is not clear enough. I don't want to test 
U-Boot, I want to test all future QEMU releases with this particular 
pre-built binary, which has been:
1/ built with the proper configuration for the board
2/ tested by someone from the Debian project on physical hardware.

(Goal: catch regressions in QEMU).

The source is available, with build scripts, and reproducible builds:
https://salsa.debian.org/debian/u-boot/blob/master/.gitlab-ci.yml

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Thomas Huth]

On 21/01/2020 00.51, Philippe Mathieu-Daudé wrote:
> This test runs U-Boot on the Raspberry Pi 2.
> It is very simple and fast:
> 
>   $ avocado --show=app,console run -t raspi2 -t u-boot tests/acceptance/
>   JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
>    (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot:
>   console: MMC:   sdhci@7e300000: 0
>   console: Loading Environment from FAT... Card did not respond to voltage select!
>   console: In:    serial
>   console: Out:   vidconsole
>   console: Err:   vidconsole
>   console: Net:   No ethernet found.
>   console: starting USB...
>   console: USB0:   Port not available.
>   console: Hit any key to stop autoboot:  0
>   console: U-Boot>
>   console: U-Boot> bdinfo
>   console: arch_number = 0x00000000
>   console: boot_params = 0x00000100
>   console: DRAM bank   = 0x00000000
>   console: -> start    = 0x00000000
>   console: -> size     = 0x3c000000
>   console: baudrate    = 115200 bps
>   console: TLB addr    = 0x3bff0000
>   console: relocaddr   = 0x3bf64000
>   console: reloc off   = 0x3bf5c000
>   console: irq_sp      = 0x3bb5fec0
>   console: sp start    = 0x3bb5feb0
>   console: Early malloc usage: 2a4 / 400
>   console: fdt_blob    = 0x3bfbdfb0
>   console: U-Boot> version
>   console: U-Boot 2019.01+dfsg-7 (May 14 2019 - 02:07:44 +0000)
>   console: gcc (Debian 8.3.0-7) 8.3.0
>   console: GNU ld (GNU Binutils for Debian) 2.31.1
>   console: U-Boot> reset
>   console: resetting ...
>   PASS (0.46 s)
> 
> U-Boot is built by the Debian project, see:
> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/acceptance/boot_linux_console.py | 28 ++++++++++++++++++++++++++
>  1 file changed, 28 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index e40b84651b..682b801b4f 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py

The file is called boot_*linux*_console.py ... so shouldn't tests that
don't use Linux rather go into another file instead?
Either machine_arm_raspi.py or maybe a boot_firmware_console.py file?

 Thomas

Re: [PATCH v4 7/7] tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'

[Thomas Huth]

On 21/01/2020 00.51, Philippe Mathieu-Daudé wrote:
> Avocado tags are handy to automatically select tests matching
> the tags. Since this test also runs U-Boot, tag it.
> 
> We can run all the tests using U-Boot as once with:
> 
>   $ avocado --show=app run -t u-boot tests/acceptance/
>   JOB LOG    : avocado/job-results/job-2020-01-21T00.16-ee9344e/job.log
>    (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2: PASS (16.59 s)
>    (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot: PASS (0.47 s)
>    (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot: PASS (2.43 s)
>   RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>   JOB TIME   : 19.78 s
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  tests/acceptance/boot_linux_console.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 22b360118d..4a4cf9d0ea 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -305,6 +305,7 @@ class BootLinuxConsole(Test):
>          :avocado: tags=arch:arm
>          :avocado: tags=machine:emcraft-sf2
>          :avocado: tags=endian:little
> +        :avocado: tags=u-boot
>          """
>          uboot_url = ('https://raw.githubusercontent.com/'
>                       'Subbaraya-Sundeep/qemu-test-binaries/'
> 

We should maybe move that test to another file, too, since it is not
using Linux. Or should we maybe rename boot_linux_console.py to simply
boot_console.py ?

 Thomas

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Philippe Mathieu-Daudé]

On 1/27/20 5:02 PM, Thomas Huth wrote:
> On 21/01/2020 00.51, Philippe Mathieu-Daudé wrote:
>> This test runs U-Boot on the Raspberry Pi 2.
>> It is very simple and fast:
>>
>>    $ avocado --show=app,console run -t raspi2 -t u-boot tests/acceptance/
>>    JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
>>     (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot:
>>    console: MMC:   sdhci@7e300000: 0
>>    console: Loading Environment from FAT... Card did not respond to voltage select!
>>    console: In:    serial
>>    console: Out:   vidconsole
>>    console: Err:   vidconsole
>>    console: Net:   No ethernet found.
>>    console: starting USB...
>>    console: USB0:   Port not available.
>>    console: Hit any key to stop autoboot:  0
>>    console: U-Boot>
>>    console: U-Boot> bdinfo
>>    console: arch_number = 0x00000000
>>    console: boot_params = 0x00000100
>>    console: DRAM bank   = 0x00000000
>>    console: -> start    = 0x00000000
>>    console: -> size     = 0x3c000000
>>    console: baudrate    = 115200 bps
>>    console: TLB addr    = 0x3bff0000
>>    console: relocaddr   = 0x3bf64000
>>    console: reloc off   = 0x3bf5c000
>>    console: irq_sp      = 0x3bb5fec0
>>    console: sp start    = 0x3bb5feb0
>>    console: Early malloc usage: 2a4 / 400
>>    console: fdt_blob    = 0x3bfbdfb0
>>    console: U-Boot> version
>>    console: U-Boot 2019.01+dfsg-7 (May 14 2019 - 02:07:44 +0000)
>>    console: gcc (Debian 8.3.0-7) 8.3.0
>>    console: GNU ld (GNU Binutils for Debian) 2.31.1
>>    console: U-Boot> reset
>>    console: resetting ...
>>    PASS (0.46 s)
>>
>> U-Boot is built by the Debian project, see:
>> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   tests/acceptance/boot_linux_console.py | 28 ++++++++++++++++++++++++++
>>   1 file changed, 28 insertions(+)
>>
>> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
>> index e40b84651b..682b801b4f 100644
>> --- a/tests/acceptance/boot_linux_console.py
>> +++ b/tests/acceptance/boot_linux_console.py
> 
> The file is called boot_*linux*_console.py ... so shouldn't tests that
> don't use Linux rather go into another file instead?
> Either machine_arm_raspi.py or maybe a boot_firmware_console.py file?

Yes I have a series ready splitting boot_linux_console.py per machines 
(as suggested machine_arm_raspi.py) not for consistency with fw/linux 
but because I'm tired of fixing conflicts while rebasing since months.

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Thomas Huth]

On 27/01/2020 17.09, Philippe Mathieu-Daudé wrote:
> On 1/27/20 5:02 PM, Thomas Huth wrote:
>> On 21/01/2020 00.51, Philippe Mathieu-Daudé wrote:
>>> This test runs U-Boot on the Raspberry Pi 2.
>>> It is very simple and fast:
>>>
>>>    $ avocado --show=app,console run -t raspi2 -t u-boot
>>> tests/acceptance/
>>>    JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
>>>     (1/1)
>>> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot:
>>>
>>>    console: MMC:   sdhci@7e300000: 0
>>>    console: Loading Environment from FAT... Card did not respond to
>>> voltage select!
>>>    console: In:    serial
>>>    console: Out:   vidconsole
>>>    console: Err:   vidconsole
>>>    console: Net:   No ethernet found.
>>>    console: starting USB...
>>>    console: USB0:   Port not available.
>>>    console: Hit any key to stop autoboot:  0
>>>    console: U-Boot>
>>>    console: U-Boot> bdinfo
>>>    console: arch_number = 0x00000000
>>>    console: boot_params = 0x00000100
>>>    console: DRAM bank   = 0x00000000
>>>    console: -> start    = 0x00000000
>>>    console: -> size     = 0x3c000000
>>>    console: baudrate    = 115200 bps
>>>    console: TLB addr    = 0x3bff0000
>>>    console: relocaddr   = 0x3bf64000
>>>    console: reloc off   = 0x3bf5c000
>>>    console: irq_sp      = 0x3bb5fec0
>>>    console: sp start    = 0x3bb5feb0
>>>    console: Early malloc usage: 2a4 / 400
>>>    console: fdt_blob    = 0x3bfbdfb0
>>>    console: U-Boot> version
>>>    console: U-Boot 2019.01+dfsg-7 (May 14 2019 - 02:07:44 +0000)
>>>    console: gcc (Debian 8.3.0-7) 8.3.0
>>>    console: GNU ld (GNU Binutils for Debian) 2.31.1
>>>    console: U-Boot> reset
>>>    console: resetting ...
>>>    PASS (0.46 s)
>>>
>>> U-Boot is built by the Debian project, see:
>>> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
>>>
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>>   tests/acceptance/boot_linux_console.py | 28 ++++++++++++++++++++++++++
>>>   1 file changed, 28 insertions(+)
>>>
>>> diff --git a/tests/acceptance/boot_linux_console.py
>>> b/tests/acceptance/boot_linux_console.py
>>> index e40b84651b..682b801b4f 100644
>>> --- a/tests/acceptance/boot_linux_console.py
>>> +++ b/tests/acceptance/boot_linux_console.py
>>
>> The file is called boot_*linux*_console.py ... so shouldn't tests that
>> don't use Linux rather go into another file instead?
>> Either machine_arm_raspi.py or maybe a boot_firmware_console.py file?
> 
> Yes I have a series ready splitting boot_linux_console.py per machines
> (as suggested machine_arm_raspi.py) not for consistency with fw/linux
> but because I'm tired of fixing conflicts while rebasing since months.

Ah, great, good idea. That will also make it easier to assign the files
to entries in MAINTAINERS, I guess.

 Thanks,
  Thomas

Re: [PATCH v4 0/7] hw/arm/raspi: Run U-Boot on the raspi machines

[Philippe Mathieu-Daudé]

Hi Peter,

(Cc'ed Wainer from the Python part).

On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> Following Laurent report:
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg639950.html
> 
> The SYS_timer is already merged, see:
> https://git.qemu.org/?p=qemu.git;a=commit;h=d05be883fc
> "hw/timer/bcm2835: Add the BCM2835 SYS_timer"
> 
> The first patch should fix Laurent other issue.
> Then few python patches are require to break into U-Boot console,
> and the last patches add U-Boot tests for Raspi2 and Raspi3.
> 
> Laurent, if you successfully test U-Boot with this patchset again,
> do you mind replying with a "Tested-by:" tag?
> 
> Regards,
> 
> Phil.
> 
> Since v3:
> - rewrote '-smp' fix.
> - tests use Debian 'trustable' u-boot.elf
> 
> previous feedbacks from Peter on v3:
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg655415.html
> 
> v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg653807.html
> Supersedes: <20191019234715.25750-1-f4bug@amsat.org>
> 
> Philippe Mathieu-Daudé (7):
>    hw/arm/raspi: Remove obsolete use of -smp to set the soc 'enabled-cpus'

While the first patch is reviewed by Alistair, the rest (acceptance 
tests) still requires an eye from Cleber/Eduardo.

Can you queue the first patch via your qemu-arm tree?

Thanks,

Phil.

>    Acceptance tests: Extract _console_interaction()
>    Acceptance tests: Add interrupt_interactive_console_until_pattern()
>    python/qemu/machine: Allow to use other serial consoles than default
>    tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2
>    tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 3
>    tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'
> 
>   hw/arm/raspi.c                            |  2 -
>   python/qemu/machine.py                    |  9 +++-
>   tests/acceptance/avocado_qemu/__init__.py | 59 +++++++++++++++++------
>   tests/acceptance/boot_linux_console.py    | 54 +++++++++++++++++++++
>   4 files changed, 107 insertions(+), 17 deletions(-)
> 

Re: [PATCH v4 2/7] Acceptance tests: Extract _console_interaction()

[Liam Merwick]

On 20/01/2020 23:51, Philippe Mathieu-Daudé wrote:
> Since we are going to re-use the code shared between
> wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
> extract the common part into a local function.
> 
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>

> ---
>   tests/acceptance/avocado_qemu/__init__.py | 31 +++++++++++++----------
>   1 file changed, 17 insertions(+), 14 deletions(-)

Re: [PATCH v4 3/7] Acceptance tests: Add interrupt_interactive_console_until_pattern()

[Liam Merwick]

On 20/01/2020 23:51, Philippe Mathieu-Daudé wrote:
> We need a function to interrupt interactive consoles.
> 
> Example: Interrupt U-Boot to set different environment values.
> 
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>


> ---
>   tests/acceptance/avocado_qemu/__init__.py | 32 +++++++++++++++++++++--
>   1 file changed, 30 insertions(+), 2 deletions(-)
> 

Re: [PATCH v4 4/7] python/qemu/machine: Allow to use other serial consoles than default

[Liam Merwick]

On 21/01/2020 08:20, Philippe Mathieu-Daudé wrote:
> On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
>> Currently the QEMU Python module limits the QEMUMachine class to
>> use the first serial console.
>>
>> Some machines/guest might use another console than the first one as
>> the 'boot console'. For example the Raspberry Pi uses the second
>> (AUX) console.
>>
>> To be able to use the Nth console as default, we simply need to
>> connect all the N - 1 consoles to the null chardev.
>>
>> Add an index argument, so we can use a specific serial console as
>> default.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>> v2:
>> - renamed 'console_index', added docstring (Cleber)
>> - reworded description (pm215)
>> ---
>>   python/qemu/machine.py | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
>> index 734efd8536..ef9f5b213f 100644
>> --- a/python/qemu/machine.py
>> +++ b/python/qemu/machine.py
>> @@ -241,6 +241,8 @@ class QEMUMachine(object):
>>                            'chardev=mon,mode=control'])
>>           if self._machine is not None:
>>               args.extend(['-machine', self._machine])
>> +        for i in range(self._console_index):
>> +            args.extend(['-serial', 'null'])
> 
> This patch is not sufficient, we have to initialize _console_index in 
> __init__():
> 

with that fix applied

Reviewed-by: Liam Merwick <liam.merwick@oracle.com>
Tested-by: Liam Merwick <liam.merwick@oracle.com>




> -- >8 --
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index ef9f5b213f..183d8f3d38 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -112,6 +112,7 @@ class QEMUMachine(object):
>           self._sock_dir = sock_dir
>           self._launched = False
>           self._machine = None
> +        self._console_index = 0
>           self._console_set = False
>           self._console_device_type = None
>           self._console_address = None
> ---
> 
> Else for tests not calling self.set_console() we get:
> 
>   'QEMUMachine' object has no attribute '_console_index'
> 
>>           if self._console_set:
>>               self._console_address = os.path.join(self._sock_dir,
>>                                                    self._name + 
>> "-console.sock")
>> @@ -527,7 +529,7 @@ class QEMUMachine(object):
>>           """
>>           self._machine = machine_type
>> -    def set_console(self, device_type=None):
>> +    def set_console(self, device_type=None, console_index=0):
>>           """
>>           Sets the device type for a console device
>> @@ -548,9 +550,14 @@ class QEMUMachine(object):
>>                               chardev:console" command line argument will
>>                               be used instead, resorting to the machine's
>>                               default device type.
>> +        @param console_index: the index of the console device to use.
>> +                              If not zero, the command line will create
>> +                              'index - 1' consoles and connect them to
>> +                              the 'null' backing character device.
>>           """
>>           self._console_set = True
>>           self._console_device_type = device_type
>> +        self._console_index = console_index
>>       @property
>>       def console_socket(self):
>>
> 
> 

Re: [PATCH v4 4/7] python/qemu/machine: Allow to use other serial consoles than default

[Wainer dos Santos Moschetta]

On 1/21/20 6:20 AM, Philippe Mathieu-Daudé wrote:
> On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
>> Currently the QEMU Python module limits the QEMUMachine class to
>> use the first serial console.
>>
>> Some machines/guest might use another console than the first one as
>> the 'boot console'. For example the Raspberry Pi uses the second
>> (AUX) console.
>>
>> To be able to use the Nth console as default, we simply need to
>> connect all the N - 1 consoles to the null chardev.
>>
>> Add an index argument, so we can use a specific serial console as
>> default.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>> v2:
>> - renamed 'console_index', added docstring (Cleber)
>> - reworded description (pm215)
>> ---
>>   python/qemu/machine.py | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
>> index 734efd8536..ef9f5b213f 100644
>> --- a/python/qemu/machine.py
>> +++ b/python/qemu/machine.py
>> @@ -241,6 +241,8 @@ class QEMUMachine(object):
>>                            'chardev=mon,mode=control'])
>>           if self._machine is not None:
>>               args.extend(['-machine', self._machine])
>> +        for i in range(self._console_index):
>> +            args.extend(['-serial', 'null'])
>
> This patch is not sufficient, we have to initialize _console_index in 
> __init__():
>
> -- >8 --
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index ef9f5b213f..183d8f3d38 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -112,6 +112,7 @@ class QEMUMachine(object):
>          self._sock_dir = sock_dir
>          self._launched = False
>          self._machine = None
> +        self._console_index = 0
>          self._console_set = False
>          self._console_device_type = None
>          self._console_address = None
> ---
>

OK, with that fix:

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


> Else for tests not calling self.set_console() we get:
>
>  'QEMUMachine' object has no attribute '_console_index'
>
>>           if self._console_set:
>>               self._console_address = os.path.join(self._sock_dir,
>>                                                    self._name + 
>> "-console.sock")
>> @@ -527,7 +529,7 @@ class QEMUMachine(object):
>>           """
>>           self._machine = machine_type
>>   -    def set_console(self, device_type=None):
>> +    def set_console(self, device_type=None, console_index=0):
>>           """
>>           Sets the device type for a console device
>>   @@ -548,9 +550,14 @@ class QEMUMachine(object):
>>                               chardev:console" command line argument 
>> will
>>                               be used instead, resorting to the 
>> machine's
>>                               default device type.
>> +        @param console_index: the index of the console device to use.
>> +                              If not zero, the command line will create
>> +                              'index - 1' consoles and connect them to
>> +                              the 'null' backing character device.
>>           """
>>           self._console_set = True
>>           self._console_device_type = device_type
>> +        self._console_index = console_index
>>         @property
>>       def console_socket(self):
>>
>
>

Re: [PATCH v4 2/7] Acceptance tests: Extract _console_interaction()

[Wainer dos Santos Moschetta]

On 1/20/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> Since we are going to re-use the code shared between
> wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
> extract the common part into a local function.
>
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/avocado_qemu/__init__.py | 31 +++++++++++++----------
>   1 file changed, 17 insertions(+), 14 deletions(-)
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 6618ea67c1..0a50fcf2be 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -55,19 +55,14 @@ def pick_default_qemu_bin(arch=None):
>           return qemu_bin_from_src_dir_path
>   
>   
> -def wait_for_console_pattern(test, success_message, failure_message=None):
> -    """
> -    Waits for messages to appear on the console, while logging the content
> -
> -    :param test: an Avocado test containing a VM that will have its console
> -                 read and probed for a success or failure message
> -    :type test: :class:`avocado_qemu.Test`
> -    :param success_message: if this message appears, test succeeds
> -    :param failure_message: if this message appears, test fails
> -    """
> +def _console_interaction(test, success_message, failure_message,
> +                         send_string):


Why not just add send_string as a parameter? Like:

def wait_for_console_pattern(test, success_message, 
failure_message=None, send_msg=None)


>       console = test.vm.console_socket.makefile()
>       console_logger = logging.getLogger('console')
>       while True:
> +        if send_string:
> +            test.vm.console_socket.sendall(send_string.encode())
> +            send_string = None # send only once

If it is going to send the message once, then put it before the loop.

>           msg = console.readline().strip()
>           if not msg:
>               continue
> @@ -79,6 +74,17 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
>               fail = 'Failure message found in console: %s' % failure_message
>               test.fail(fail)
>   
> +def wait_for_console_pattern(test, success_message, failure_message=None):
> +    """
> +    Waits for messages to appear on the console, while logging the content
> +
> +    :param test: an Avocado test containing a VM that will have its console
> +                 read and probed for a success or failure message
> +    :type test: :class:`avocado_qemu.Test`
> +    :param success_message: if this message appears, test succeeds
> +    :param failure_message: if this message appears, test fails
> +    """
> +    _console_interaction(test, success_message, failure_message, None)
>   
>   def exec_command_and_wait_for_pattern(test, command,
>                                         success_message, failure_message=None):
> @@ -94,10 +100,7 @@ def exec_command_and_wait_for_pattern(test, command,
>       :param success_message: if this message appears, test succeeds
>       :param failure_message: if this message appears, test fails
>       """
> -    command += '\r'
> -    test.vm.console_socket.sendall(command.encode())
> -    wait_for_console_pattern(test, success_message, failure_message)
> -
> +    _console_interaction(test, success_message, failure_message, command + '\r')
>   
>   class Test(avocado.Test):
>       def _get_unique_tag_val(self, tag_name):

Re: [PATCH v4 2/7] Acceptance tests: Extract _console_interaction()

[Wainer dos Santos Moschetta]

On 1/28/20 5:34 PM, Wainer dos Santos Moschetta wrote:
>
> On 1/20/20 9:51 PM, Philippe Mathieu-Daudé wrote:
>> Since we are going to re-use the code shared between
>> wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
>> extract the common part into a local function.
>>
>> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   tests/acceptance/avocado_qemu/__init__.py | 31 +++++++++++++----------
>>   1 file changed, 17 insertions(+), 14 deletions(-)
>>
>> diff --git a/tests/acceptance/avocado_qemu/__init__.py 
>> b/tests/acceptance/avocado_qemu/__init__.py
>> index 6618ea67c1..0a50fcf2be 100644
>> --- a/tests/acceptance/avocado_qemu/__init__.py
>> +++ b/tests/acceptance/avocado_qemu/__init__.py
>> @@ -55,19 +55,14 @@ def pick_default_qemu_bin(arch=None):
>>           return qemu_bin_from_src_dir_path
>>     -def wait_for_console_pattern(test, success_message, 
>> failure_message=None):
>> -    """
>> -    Waits for messages to appear on the console, while logging the 
>> content
>> -
>> -    :param test: an Avocado test containing a VM that will have its 
>> console
>> -                 read and probed for a success or failure message
>> -    :type test: :class:`avocado_qemu.Test`
>> -    :param success_message: if this message appears, test succeeds
>> -    :param failure_message: if this message appears, test fails
>> -    """
>> +def _console_interaction(test, success_message, failure_message,
>> +                         send_string):
>
>
> Why not just add send_string as a parameter? Like:
>
> def wait_for_console_pattern(test, success_message, 
> failure_message=None, send_msg=None)
>
>
>>       console = test.vm.console_socket.makefile()
>>       console_logger = logging.getLogger('console')
>>       while True:
>> +        if send_string:
>> + test.vm.console_socket.sendall(send_string.encode())
>> +            send_string = None # send only once
>
> If it is going to send the message once, then put it before the loop.

OK, now that I read the next patch in this series (patch 03), I 
understood what you trying to accomplish here. So disregard my comments.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
>
>>           msg = console.readline().strip()
>>           if not msg:
>>               continue
>> @@ -79,6 +74,17 @@ def wait_for_console_pattern(test, 
>> success_message, failure_message=None):
>>               fail = 'Failure message found in console: %s' % 
>> failure_message
>>               test.fail(fail)
>>   +def wait_for_console_pattern(test, success_message, 
>> failure_message=None):
>> +    """
>> +    Waits for messages to appear on the console, while logging the 
>> content
>> +
>> +    :param test: an Avocado test containing a VM that will have its 
>> console
>> +                 read and probed for a success or failure message
>> +    :type test: :class:`avocado_qemu.Test`
>> +    :param success_message: if this message appears, test succeeds
>> +    :param failure_message: if this message appears, test fails
>> +    """
>> +    _console_interaction(test, success_message, failure_message, None)
>>     def exec_command_and_wait_for_pattern(test, command,
>>                                         success_message, 
>> failure_message=None):
>> @@ -94,10 +100,7 @@ def exec_command_and_wait_for_pattern(test, command,
>>       :param success_message: if this message appears, test succeeds
>>       :param failure_message: if this message appears, test fails
>>       """
>> -    command += '\r'
>> -    test.vm.console_socket.sendall(command.encode())
>> -    wait_for_console_pattern(test, success_message, failure_message)
>> -
>> +    _console_interaction(test, success_message, failure_message, 
>> command + '\r')
>>     class Test(avocado.Test):
>>       def _get_unique_tag_val(self, tag_name):
>
>

Re: [PATCH v4 3/7] Acceptance tests: Add interrupt_interactive_console_until_pattern()

[Wainer dos Santos Moschetta]

On 1/20/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> We need a function to interrupt interactive consoles.
>
> Example: Interrupt U-Boot to set different environment values.
>
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/avocado_qemu/__init__.py | 32 +++++++++++++++++++++--
>   1 file changed, 30 insertions(+), 2 deletions(-)

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 0a50fcf2be..d4358eb431 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -56,13 +56,15 @@ def pick_default_qemu_bin(arch=None):
>   
>   
>   def _console_interaction(test, success_message, failure_message,
> -                         send_string):
> +                         send_string, keep_sending=False):
> +    assert not keep_sending or send_string
>       console = test.vm.console_socket.makefile()
>       console_logger = logging.getLogger('console')
>       while True:
>           if send_string:
>               test.vm.console_socket.sendall(send_string.encode())
> -            send_string = None # send only once
> +            if not keep_sending:
> +                send_string = None # send only once
>           msg = console.readline().strip()
>           if not msg:
>               continue
> @@ -74,6 +76,32 @@ def _console_interaction(test, success_message, failure_message,
>               fail = 'Failure message found in console: %s' % failure_message
>               test.fail(fail)
>   
> +def interrupt_interactive_console_until_pattern(test, success_message,
> +                                                failure_message=None,
> +                                                interrupt_string='\r'):
> +    """
> +    Keep sending a string to interrupt a console prompt, while logging the
> +    console output. Typical use case is to break a boot loader prompt, such:
> +
> +        Press a key within 5 seconds to interrupt boot process.
> +        5
> +        4
> +        3
> +        2
> +        1
> +        Booting default image...
> +
> +    :param test: an Avocado test containing a VM that will have its console
> +                 read and probed for a success or failure message
> +    :type test: :class:`avocado_qemu.Test`
> +    :param success_message: if this message appears, test succeeds
> +    :param failure_message: if this message appears, test fails
> +    :param interrupt_string: a string to send to the console before trying
> +                             to read a new line
> +    """
> +    _console_interaction(test, success_message, failure_message,
> +                         interrupt_string, True)
> +
>   def wait_for_console_pattern(test, success_message, failure_message=None):
>       """
>       Waits for messages to appear on the console, while logging the content

Re: [PATCH v4 5/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 2

[Wainer dos Santos Moschetta]

On 1/20/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> This test runs U-Boot on the Raspberry Pi 2.
> It is very simple and fast:
>
>    $ avocado --show=app,console run -t raspi2 -t u-boot tests/acceptance/
>    JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
>     (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot:
>    console: MMC:   sdhci@7e300000: 0
>    console: Loading Environment from FAT... Card did not respond to voltage select!
>    console: In:    serial
>    console: Out:   vidconsole
>    console: Err:   vidconsole
>    console: Net:   No ethernet found.
>    console: starting USB...
>    console: USB0:   Port not available.
>    console: Hit any key to stop autoboot:  0
>    console: U-Boot>
>    console: U-Boot> bdinfo
>    console: arch_number = 0x00000000
>    console: boot_params = 0x00000100
>    console: DRAM bank   = 0x00000000
>    console: -> start    = 0x00000000
>    console: -> size     = 0x3c000000
>    console: baudrate    = 115200 bps
>    console: TLB addr    = 0x3bff0000
>    console: relocaddr   = 0x3bf64000
>    console: reloc off   = 0x3bf5c000
>    console: irq_sp      = 0x3bb5fec0
>    console: sp start    = 0x3bb5feb0
>    console: Early malloc usage: 2a4 / 400
>    console: fdt_blob    = 0x3bfbdfb0
>    console: U-Boot> version
>    console: U-Boot 2019.01+dfsg-7 (May 14 2019 - 02:07:44 +0000)
>    console: gcc (Debian 8.3.0-7) 8.3.0
>    console: GNU ld (GNU Binutils for Debian) 2.31.1
>    console: U-Boot> reset
>    console: resetting ...
>    PASS (0.46 s)
>
> U-Boot is built by the Debian project, see:
> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/boot_linux_console.py | 28 ++++++++++++++++++++++++++
>   1 file changed, 28 insertions(+)


Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index e40b84651b..682b801b4f 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -16,6 +16,7 @@ import shutil
>   from avocado import skipUnless
>   from avocado_qemu import Test
>   from avocado_qemu import exec_command_and_wait_for_pattern
> +from avocado_qemu import interrupt_interactive_console_until_pattern
>   from avocado_qemu import wait_for_console_pattern
>   from avocado.utils import process
>   from avocado.utils import archive
> @@ -485,6 +486,33 @@ class BootLinuxConsole(Test):
>           exec_command_and_wait_for_pattern(self, 'reboot',
>                                                   'reboot: Restarting system')
>   
> +    def test_arm_raspi2_uboot(self):
> +        """
> +        :avocado: tags=arch:arm
> +        :avocado: tags=machine:raspi2
> +        :avocado: tags=u-boot
> +        """
> +        deb_url = ('https://snapshot.debian.org/archive/debian/'
> +                   '20190514T084354Z/pool/main/u/u-boot/'
> +                   'u-boot-rpi_2019.01%2Bdfsg-7_armhf.deb')
> +        deb_hash = 'ad858cf3afe623b6c3fa2e20dcdd1768fcb9ae83'
> +        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> +        uboot_path = '/usr/lib/u-boot/rpi_2/uboot.elf'
> +        uboot_path = self.extract_from_deb(deb_path, uboot_path)
> +
> +        self.vm.set_console()
> +        self.vm.add_args('-kernel', uboot_path,
> +                         # VideoCore starts CPU with only 1 core enabled
> +                         '-global', 'bcm2836.enabled-cpus=1',
> +                         '-no-reboot')
> +        self.vm.launch()
> +        interrupt_interactive_console_until_pattern(self,
> +                                       'Hit any key to stop autoboot:',
> +                                       'Config file not found')
> +        exec_command_and_wait_for_pattern(self, 'bdinfo', 'U-Boot')
> +        exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
> +        exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
> +
>       def test_s390x_s390_ccw_virtio(self):
>           """
>           :avocado: tags=arch:s390x

Re: [PATCH v4 6/7] tests/boot_linux_console: Test booting U-Boot on the Raspberry Pi 3

[Wainer dos Santos Moschetta]

On 1/20/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> This test runs U-Boot on the Raspberry Pi 3.
> It is very simple and fast:
>
>    $ avocado --show=app,console run -t raspi3 -t u-boot tests/acceptance/
>    JOB LOG    : avocado/job-results/job-2020-01-20T23.40-2424777/job.log
>     (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot:
>    console: MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
>    console: Loading Environment from FAT... WARNING at drivers/mmc/bcm2835_sdhost.c:410/bcm2835_send_command()!
>    console: WARNING at drivers/mmc/bcm2835_sdhost.c:410/bcm2835_send_command()!
>    console: Card did not respond to voltage select!
>    console: In:    serial
>    console: Out:   vidconsole
>    console: Err:   vidconsole
>    console: Net:   No ethernet found.
>    console: starting USB...
>    console: Bus usb@7e980000: Port not available.
>    console: Hit any key to stop autoboot:  0
>    console: U-Boot>
>    console: U-Boot>
>    console: U-Boot> bdinfo
>    console: arch_number = 0x0000000000000000
>    console: boot_params = 0x0000000000000100
>    console: DRAM bank   = 0x0000000000000000
>    console: -> start    = 0x0000000000000000
>    console: -> size     = 0x000000003c000000
>    console: baudrate    = 115200 bps
>    console: TLB addr    = 0x000000003bff0000
>    console: relocaddr   = 0x000000003bf57000
>    console: reloc off   = 0x000000003bed7000
>    console: irq_sp      = 0x000000003bb52dd0
>    console: sp start    = 0x000000003bb52dd0
>    console: FB base     = 0x0000000000000000
>    console: Early malloc usage: 7b0 / 2000
>    console: fdt_blob    = 0x000000003bfbf200
>    console: U-Boot> version
>    console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000)
>    console: gcc (Debian 9.2.1-22) 9.2.1 20200104
>    console: GNU ld (GNU Binutils for Debian) 2.33.1
>    console: U-Boot> reset
>    console: resetting ...
>    PASS (1.79 s)
>
> U-Boot is built by the Debian project, see:
> https://wiki.debian.org/InstallingDebianOn/Allwinner#Creating_a_bootable_SD_Card_with_u-boot
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/boot_linux_console.py | 25 +++++++++++++++++++++++++
>   1 file changed, 25 insertions(+)


Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 682b801b4f..22b360118d 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -513,6 +513,31 @@ class BootLinuxConsole(Test):
>           exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
>           exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
>   
> +    def test_aarch64_raspi3_uboot(self):
> +        """
> +        :avocado: tags=arch:aarch64
> +        :avocado: tags=machine:raspi3
> +        :avocado: tags=u-boot
> +        """
> +        deb_url = ('https://snapshot.debian.org/archive/debian/'
> +                   '20200108T145233Z/pool/main/u/u-boot/'
> +                   'u-boot-rpi_2020.01%2Bdfsg-1_arm64.deb')
> +        deb_hash = 'f394386e02469d52f2eb3c07a2325b1c95aeb00b'
> +        deb_path = self.fetch_asset(deb_url, asset_hash=deb_hash)
> +        uboot_path = '/usr/lib/u-boot/rpi_3/u-boot.bin'
> +        uboot_path = self.extract_from_deb(deb_path, uboot_path)
> +
> +        self.vm.set_console(console_index=1)
> +        self.vm.add_args('-kernel', uboot_path,
> +                         '-no-reboot')
> +        self.vm.launch()
> +        interrupt_interactive_console_until_pattern(self,
> +                                       'Hit any key to stop autoboot:',
> +                                       'Config file not found')
> +        exec_command_and_wait_for_pattern(self, 'bdinfo', 'U-Boot')
> +        exec_command_and_wait_for_pattern(self, 'version', 'U-Boot')
> +        exec_command_and_wait_for_pattern(self, 'reset', 'resetting ...')
> +
>       def test_s390x_s390_ccw_virtio(self):
>           """
>           :avocado: tags=arch:s390x

Re: [PATCH v4 7/7] tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'

[Wainer dos Santos Moschetta]

On 1/27/20 2:05 PM, Thomas Huth wrote:
> On 21/01/2020 00.51, Philippe Mathieu-Daudé wrote:
>> Avocado tags are handy to automatically select tests matching
>> the tags. Since this test also runs U-Boot, tag it.
>>
>> We can run all the tests using U-Boot as once with:
>>
>>    $ avocado --show=app run -t u-boot tests/acceptance/
>>    JOB LOG    : avocado/job-results/job-2020-01-21T00.16-ee9344e/job.log
>>     (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2: PASS (16.59 s)
>>     (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot: PASS (0.47 s)
>>     (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot: PASS (2.43 s)
>>    RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>>    JOB TIME   : 19.78 s
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   tests/acceptance/boot_linux_console.py | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
>> index 22b360118d..4a4cf9d0ea 100644
>> --- a/tests/acceptance/boot_linux_console.py
>> +++ b/tests/acceptance/boot_linux_console.py
>> @@ -305,6 +305,7 @@ class BootLinuxConsole(Test):
>>           :avocado: tags=arch:arm
>>           :avocado: tags=machine:emcraft-sf2
>>           :avocado: tags=endian:little
>> +        :avocado: tags=u-boot
>>           """
>>           uboot_url = ('https://raw.githubusercontent.com/'
>>                        'Subbaraya-Sundeep/qemu-test-binaries/'
>>
> We should maybe move that test to another file, too, since it is not
> using Linux. Or should we maybe rename boot_linux_console.py to simply
> boot_console.py ?

Thinking aloud: maybe create a directory 'boot' (or 'boot_tests') and 
group tests in files according to the 'arch' tag.

Example:

[wainersm@localhost boot]$ tree
.
└── arm.py

0 directories, 1 file
[wainersm@localhost boot]$ cat arm.py
from avocado_qemu import Test

class BootArmTest(Test):
     """
     :avocado: tags=arch:arm
     """
     def test_linux_virt()
         """
         :avocado: tags=machine:virt
         :avocado: tags=linux-boot
         """
         # Test here
         pass

     def test_uboot_emcraft_sf2()
         """
         :avocado: tags=machine:emcraft-sf2
         :avocado: tags=endian:little
         :avocado: tags=u-boot
         """
         # Test here
         pass
--------------


Back to this patch...

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


>
>   Thomas
>
>

Re: [PATCH v4 0/7] hw/arm/raspi: Run U-Boot on the raspi machines

[Peter Maydell]

On Tue, 28 Jan 2020 at 06:53, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> Hi Peter,
>
> (Cc'ed Wainer from the Python part).
>
> On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> > Following Laurent report:
> > https://www.mail-archive.com/qemu-devel@nongnu.org/msg639950.html
> >
> > The SYS_timer is already merged, see:
> > https://git.qemu.org/?p=qemu.git;a=commit;h=d05be883fc
> > "hw/timer/bcm2835: Add the BCM2835 SYS_timer"
> >
> > The first patch should fix Laurent other issue.
> > Then few python patches are require to break into U-Boot console,
> > and the last patches add U-Boot tests for Raspi2 and Raspi3.
> >
> > Laurent, if you successfully test U-Boot with this patchset again,
> > do you mind replying with a "Tested-by:" tag?
> >
> > Regards,
> >
> > Phil.
> >
> > Since v3:
> > - rewrote '-smp' fix.
> > - tests use Debian 'trustable' u-boot.elf
> >
> > previous feedbacks from Peter on v3:
> > https://www.mail-archive.com/qemu-devel@nongnu.org/msg655415.html
> >
> > v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg653807.html
> > Supersedes: <20191019234715.25750-1-f4bug@amsat.org>
> >
> > Philippe Mathieu-Daudé (7):
> >    hw/arm/raspi: Remove obsolete use of -smp to set the soc 'enabled-cpus'
>
> While the first patch is reviewed by Alistair, the rest (acceptance
> tests) still requires an eye from Cleber/Eduardo.
>
> Can you queue the first patch via your qemu-arm tree?

OK, I've queued patch 1 (and dropped the rest from my
to-review queue on the assumption that somebody else will
pick them up and/or you'll send a respin at some point).

-- PMM

Re: [PATCH v4 4/7] python/qemu/machine: Allow to use other serial consoles than default

[Philippe Mathieu-Daudé]

On 1/21/20 9:20 AM, Philippe Mathieu-Daudé wrote:
> On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
>> Currently the QEMU Python module limits the QEMUMachine class to
>> use the first serial console.
>>
>> Some machines/guest might use another console than the first one as
>> the 'boot console'. For example the Raspberry Pi uses the second
>> (AUX) console.
>>
>> To be able to use the Nth console as default, we simply need to
>> connect all the N - 1 consoles to the null chardev.
>>
>> Add an index argument, so we can use a specific serial console as
>> default.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>> v2:
>> - renamed 'console_index', added docstring (Cleber)
>> - reworded description (pm215)
>> ---
>>   python/qemu/machine.py | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
>> index 734efd8536..ef9f5b213f 100644
>> --- a/python/qemu/machine.py
>> +++ b/python/qemu/machine.py
>> @@ -241,6 +241,8 @@ class QEMUMachine(object):
>>                            'chardev=mon,mode=control'])
>>           if self._machine is not None:
>>               args.extend(['-machine', self._machine])
>> +        for i in range(self._console_index):
>> +            args.extend(['-serial', 'null'])
> 
> This patch is not sufficient, we have to initialize _console_index in 
> __init__():
> 
> -- >8 --
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index ef9f5b213f..183d8f3d38 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -112,6 +112,7 @@ class QEMUMachine(object):
>           self._sock_dir = sock_dir
>           self._launched = False
>           self._machine = None
> +        self._console_index = 0
>           self._console_set = False
>           self._console_device_type = None
>           self._console_address = None
> ---
> 
> Else for tests not calling self.set_console() we get:
> 
>   'QEMUMachine' object has no attribute '_console_index'
> 
>>           if self._console_set:
>>               self._console_address = os.path.join(self._sock_dir,
>>                                                    self._name + 
>> "-console.sock")
>> @@ -527,7 +529,7 @@ class QEMUMachine(object):
>>           """
>>           self._machine = machine_type
>> -    def set_console(self, device_type=None):
>> +    def set_console(self, device_type=None, console_index=0):
>>           """
>>           Sets the device type for a console device
>> @@ -548,9 +550,14 @@ class QEMUMachine(object):
>>                               chardev:console" command line argument will
>>                               be used instead, resorting to the machine's
>>                               default device type.
>> +        @param console_index: the index of the console device to use.
>> +                              If not zero, the command line will create
>> +                              'index - 1' consoles and connect them to
>> +                              the 'null' backing character device.
>>           """
>>           self._console_set = True
>>           self._console_device_type = device_type
>> +        self._console_index = console_index
>>       @property
>>       def console_socket(self):
>>

Thanks, added missing line and applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

Re: [PATCH v4 2/7] Acceptance tests: Extract _console_interaction()

[Philippe Mathieu-Daudé]

On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> Since we are going to re-use the code shared between
> wait_for_console_pattern() and exec_command_and_wait_for_pattern(),
> extract the common part into a local function.
> 
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/avocado_qemu/__init__.py | 31 +++++++++++++----------
>   1 file changed, 17 insertions(+), 14 deletions(-)
> 
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 6618ea67c1..0a50fcf2be 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -55,19 +55,14 @@ def pick_default_qemu_bin(arch=None):
>           return qemu_bin_from_src_dir_path
>   
>   
> -def wait_for_console_pattern(test, success_message, failure_message=None):
> -    """
> -    Waits for messages to appear on the console, while logging the content
> -
> -    :param test: an Avocado test containing a VM that will have its console
> -                 read and probed for a success or failure message
> -    :type test: :class:`avocado_qemu.Test`
> -    :param success_message: if this message appears, test succeeds
> -    :param failure_message: if this message appears, test fails
> -    """
> +def _console_interaction(test, success_message, failure_message,
> +                         send_string):
>       console = test.vm.console_socket.makefile()
>       console_logger = logging.getLogger('console')
>       while True:
> +        if send_string:
> +            test.vm.console_socket.sendall(send_string.encode())
> +            send_string = None # send only once
>           msg = console.readline().strip()
>           if not msg:
>               continue
> @@ -79,6 +74,17 @@ def wait_for_console_pattern(test, success_message, failure_message=None):
>               fail = 'Failure message found in console: %s' % failure_message
>               test.fail(fail)
>   
> +def wait_for_console_pattern(test, success_message, failure_message=None):
> +    """
> +    Waits for messages to appear on the console, while logging the content
> +
> +    :param test: an Avocado test containing a VM that will have its console
> +                 read and probed for a success or failure message
> +    :type test: :class:`avocado_qemu.Test`
> +    :param success_message: if this message appears, test succeeds
> +    :param failure_message: if this message appears, test fails
> +    """
> +    _console_interaction(test, success_message, failure_message, None)
>   
>   def exec_command_and_wait_for_pattern(test, command,
>                                         success_message, failure_message=None):
> @@ -94,10 +100,7 @@ def exec_command_and_wait_for_pattern(test, command,
>       :param success_message: if this message appears, test succeeds
>       :param failure_message: if this message appears, test fails
>       """
> -    command += '\r'
> -    test.vm.console_socket.sendall(command.encode())
> -    wait_for_console_pattern(test, success_message, failure_message)
> -
> +    _console_interaction(test, success_message, failure_message, command + '\r')
>   
>   class Test(avocado.Test):
>       def _get_unique_tag_val(self, tag_name):
> 

Thanks, applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

Re: [PATCH v4 3/7] Acceptance tests: Add interrupt_interactive_console_until_pattern()

[Philippe Mathieu-Daudé]

On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> We need a function to interrupt interactive consoles.
> 
> Example: Interrupt U-Boot to set different environment values.
> 
> Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/avocado_qemu/__init__.py | 32 +++++++++++++++++++++--
>   1 file changed, 30 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py
> index 0a50fcf2be..d4358eb431 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -56,13 +56,15 @@ def pick_default_qemu_bin(arch=None):
>   
>   
>   def _console_interaction(test, success_message, failure_message,
> -                         send_string):
> +                         send_string, keep_sending=False):
> +    assert not keep_sending or send_string
>       console = test.vm.console_socket.makefile()
>       console_logger = logging.getLogger('console')
>       while True:
>           if send_string:
>               test.vm.console_socket.sendall(send_string.encode())
> -            send_string = None # send only once
> +            if not keep_sending:
> +                send_string = None # send only once
>           msg = console.readline().strip()
>           if not msg:
>               continue
> @@ -74,6 +76,32 @@ def _console_interaction(test, success_message, failure_message,
>               fail = 'Failure message found in console: %s' % failure_message
>               test.fail(fail)
>   
> +def interrupt_interactive_console_until_pattern(test, success_message,
> +                                                failure_message=None,
> +                                                interrupt_string='\r'):
> +    """
> +    Keep sending a string to interrupt a console prompt, while logging the
> +    console output. Typical use case is to break a boot loader prompt, such:
> +
> +        Press a key within 5 seconds to interrupt boot process.
> +        5
> +        4
> +        3
> +        2
> +        1
> +        Booting default image...
> +
> +    :param test: an Avocado test containing a VM that will have its console
> +                 read and probed for a success or failure message
> +    :type test: :class:`avocado_qemu.Test`
> +    :param success_message: if this message appears, test succeeds
> +    :param failure_message: if this message appears, test fails
> +    :param interrupt_string: a string to send to the console before trying
> +                             to read a new line
> +    """
> +    _console_interaction(test, success_message, failure_message,
> +                         interrupt_string, True)
> +
>   def wait_for_console_pattern(test, success_message, failure_message=None):
>       """
>       Waits for messages to appear on the console, while logging the content
> 

Thanks, applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

Re: [PATCH v4 7/7] tests/boot_linux_console: Tag Emcraft Smartfusion2 as running 'u-boot'

[Philippe Mathieu-Daudé]

On 1/21/20 12:51 AM, Philippe Mathieu-Daudé wrote:
> Avocado tags are handy to automatically select tests matching
> the tags. Since this test also runs U-Boot, tag it.
> 
> We can run all the tests using U-Boot as once with:
> 
>    $ avocado --show=app run -t u-boot tests/acceptance/
>    JOB LOG    : avocado/job-results/job-2020-01-21T00.16-ee9344e/job.log
>     (1/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2: PASS (16.59 s)
>     (2/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_uboot: PASS (0.47 s)
>     (3/3) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_uboot: PASS (2.43 s)
>    RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
>    JOB TIME   : 19.78 s
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   tests/acceptance/boot_linux_console.py | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 22b360118d..4a4cf9d0ea 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -305,6 +305,7 @@ class BootLinuxConsole(Test):
>           :avocado: tags=arch:arm
>           :avocado: tags=machine:emcraft-sf2
>           :avocado: tags=endian:little
> +        :avocado: tags=u-boot
>           """
>           uboot_url = ('https://raw.githubusercontent.com/'
>                        'Subbaraya-Sundeep/qemu-test-binaries/'
> 

Thanks, applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next