[Buildroot] [PATCH v4 8/9] support/scripts/boot-qemu-image.py: boot Qemu images with Qemu-system.

[Romain Naour <romain.naour@smile.fr>]

From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>

This script is intended to be used by gitlab CI to test
at runtime Qemu images generated by Buildroot's Qemu defconfigs.

This allows to troubleshoot different issues that may be
associated with defective builds by lanching a qemu machine,
sending root password, waiting for login shell and then perform
a shutdown.

This script is inspired by toolchain builder [1] and the Buildroot
testing infrastructure.

Usage: boot-qemu-image.py <qemu_arch_defconfig>

The gitlab CI will call this script for each defconfig build but
only Qemu defconfig will be runtime tested, all others defconfig are
ignored. The script retrieve the qemu command line from start-qemu.sh
script. The start-qemu.sh script is no executed directly since we
want to handle the serial output with pexpect to detect the
shell, enter the password and powerdown the system.
If the script is missing, error out with a message in the log.

The script check is the corresponding Qemu binary (qemu-system) is
available, either built by Buildroot or available from the host.
Some Qemu defconfig must be used with a specific Qemu version (fork)
that is not always available. The script doesn't error out when
qemu-system is missing but a message is added in the log.

Finally, the script start Qemu like it's done for the Buildroot
testing infrastructure (using pexpect).

Note:
We noticed some timeout issues with pexpect when the Qemu machine is
powered off. That's because Qemu process doesn't stop even if the
system is halted (after "System halted"). So the script doesn't error
out when such timeout occure. The behaviour depends on the architecture
emulated by Qemu.

[1] https://github.com/bootlin/toolchains-builder/blob/master/build.sh

Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>

wip
---
 support/scripts/boot-qemu-image.py | 105 +++++++++++++++++++++++++++++
 1 file changed, 105 insertions(+)
 create mode 100755 support/scripts/boot-qemu-image.py

diff --git a/support/scripts/boot-qemu-image.py b/support/scripts/boot-qemu-image.py
new file mode 100755
index 0000000000..abaf88d446
--- /dev/null
+++ b/support/scripts/boot-qemu-image.py
@@ -0,0 +1,105 @@
+#!/usr/bin/env python3
+
+# This script expect to run from the Buildroot top directory.
+
+import pexpect
+import sys
+import os
+import re
+import shlex
+import shutil
+import subprocess
+
+argc = len(sys.argv)
+if not (argc == 2):
+    print("Error: incorrect number of arguments")
+    print("""Usage: boot-qemu-image.py <qemu_arch_defconfig>""")
+    sys.exit(1)
+
+defconfig_filename = sys.argv[1]
+
+# Ignore non Qemu defconfig
+if defconfig_filename.startswith('qemu_') is False:
+    sys.exit(0)
+
+qemu_start_script_filepath = os.path.join(os.getcwd(), 'output/images/start-qemu.sh')
+
+# Ignore if start-qemu.sh is missing, we can't test.
+if not os.path.exists(qemu_start_script_filepath):
+    print("Error: " + qemu_start_script_filepath) + " file is missing."
+    sys.exit(1)
+
+qemu_cmd = ""
+
+with open(qemu_start_script_filepath, 'r') as script_file:
+    for line in script_file:
+        if re.search("qemu-system", line):
+            qemu_cmd = line
+            break
+
+if not qemu_cmd:
+    print("Error: No QEMU command line found in " + qemu_start_script_filepath)
+    sys.exit(1)
+
+# Replace bashism
+qemu_cmd = line.replace("${IMAGE_DIR}", "output/images")
+
+# pexpect needs a list, convert a sting to a list and escape quoted substring.
+qemu_cmd = shlex.split(qemu_cmd)
+
+# Use host Qemu if provided by Buildroot.
+os.environ["PATH"] = os.getcwd() + "/output/host/bin" + os.pathsep + os.environ["PATH"]
+
+# Ignore when no Qemu emulation is available
+if not shutil.which(qemu_cmd[0]):
+    print("No " + qemu_cmd[0] + " binary available, THIS DEFCONFIG CAN NOT BE TESTED!")
+    sys.exit(0)
+
+
+def main():
+    global child
+
+    try:
+        child.expect(["buildroot login:", pexpect.TIMEOUT], timeout=60)
+    except pexpect.EOF:
+        print("Connection problem, exiting.")
+        sys.exit(1)
+    except pexpect.TIMEOUT:
+        print("System did not boot in time, exiting.")
+        sys.exit(1)
+
+    child.sendline("root\r")
+
+    try:
+        child.expect(["# ", pexpect.TIMEOUT], timeout=60)
+    except pexpect.EOF:
+        print("Cannot connect to shell")
+        sys.exit(1)
+    except pexpect.TIMEOUT:
+        print("Timeout while waiting for shell")
+        sys.exit(1)
+
+    child.sendline("poweroff\r")
+
+    try:
+        child.expect(["System halted", pexpect.TIMEOUT], timeout=60)
+        child.expect(pexpect.EOF)
+    except pexpect.EOF:
+        sys.exit(0)
+    except pexpect.TIMEOUT:
+        print("Cannot halt machine")
+        # Qemu may not exit properly after "System halted", ignore.
+        sys.exit(0)
+
+
+# Log the Qemu version
+subprocess.call([qemu_cmd[0], "--version"])
+
+# Log the Qemu command line
+print(qemu_cmd)
+
+child = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:], timeout=5, encoding='utf-8',
+                      env={"QEMU_AUDIO_DRV": "none", 'PATH': os.environ["PATH"]})
+# We want only stdout into the log to avoid double echo
+child.logfile = sys.stdout
+main()
-- 
2.24.1