[PATCH 1/6] runfvp: reset the process group on startup

[PATCH 1/6] runfvp: reset the process group on startup

[Ross Burton]

So that it is easy to kill runfvp and everything it starts (such as
telnet or the FVP itself), reset the process group on startup.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 scripts/runfvp | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/runfvp b/scripts/runfvp
index a5a436e8..97836726 100755
--- a/scripts/runfvp
+++ b/scripts/runfvp
@@ -217,6 +217,9 @@ def runfvp(cli_args):
 
 if __name__ == "__main__":
     try:
+        # Set the process group so that it's possible to kill runfvp and
+        # everything it spawns easily.
+        os.setpgid(0, 0)
         runfvp(sys.argv[1:])
     except KeyboardInterrupt:
         pass
-- 
2.25.1

[PATCH 2/6] arm/oeqa: add FVP testimage controller target

[Ross Burton]

Add a new oeqa.core.target.OETarget subclass for testimage which starts
a FVP using runfvp. This uses --console to connect to the console so
telnet is needed on the host.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .gitignore                                |  1 +
 meta-arm/lib/oeqa/controllers/__init__.py |  0
 meta-arm/lib/oeqa/controllers/fvp.py      | 87 +++++++++++++++++++++++
 3 files changed, 88 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 meta-arm/lib/oeqa/controllers/__init__.py
 create mode 100644 meta-arm/lib/oeqa/controllers/fvp.py

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 00000000..bee8a64b
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+__pycache__
diff --git a/meta-arm/lib/oeqa/controllers/__init__.py b/meta-arm/lib/oeqa/controllers/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/meta-arm/lib/oeqa/controllers/fvp.py b/meta-arm/lib/oeqa/controllers/fvp.py
new file mode 100644
index 00000000..1e26a519
--- /dev/null
+++ b/meta-arm/lib/oeqa/controllers/fvp.py
@@ -0,0 +1,87 @@
+import asyncio
+import os
+import pathlib
+import signal
+import subprocess
+
+import oeqa.core.target.ssh
+
+class OEFVPTarget(oeqa.core.target.ssh.OESSHTarget):
+
+    # meta-arm/scripts isn't on PATH, so work out where it is
+    metaarm = pathlib.Path(__file__).parents[4]
+
+    def __init__(self, logger, target_ip, server_ip, timeout=300, user='root',
+                 port=None, server_port=0, dir_image=None, rootfs=None, bootlog=None,
+                 **kwargs):
+        super().__init__(logger, target_ip, server_ip, timeout, user, port)
+        image_dir = pathlib.Path(dir_image)
+        basename = pathlib.Path(rootfs).stem
+        self.fvpconf = image_dir / (basename + ".fvpconf")
+
+        if not self.fvpconf.exists():
+            raise FileNotFoundError(f"Cannot find {self.fvpconf}")
+        # FVPs boot slowly, so allow ten minutes
+        self.boot_timeout = 10 * 60
+
+        self.logfile = bootlog and open(bootlog, "wb") or None
+
+    async def boot_fvp(self):
+        cmd = [OEFVPTarget.metaarm / "scripts" / "runfvp", "--console", "--verbose", self.fvpconf]
+        # Python 3.7 needs the command items to be str
+        cmd = [str(c) for c in cmd]
+        self.logger.debug(f"Starting {cmd}")
+
+        # TODO: refactor runfvp so this can import it and directly hook to the
+        # console callback, then use telnetlib directly to access the console.
+
+        # As we're using --console, telnet expects stdin to be readable too.
+        self.fvp = await asyncio.create_subprocess_exec(*cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdin=subprocess.PIPE)
+        self.logger.debug(f"Started runfvp PID {self.fvp.pid}")
+
+        async def wait_for_login():
+            bootlog = bytearray()
+            while True:
+                line = await self.fvp.stdout.read(1024)
+                if not line:
+                    self.logger.debug("runfvp terminated")
+                    return False, bootlog
+
+                self.logger.debug(f"Read line [{line}]")
+
+                bootlog += line
+                if self.logfile:
+                    self.logfile.write(line)
+
+                if b" login:" in bootlog:
+                    self.logger.debug("Found login prompt")
+                    return True, bootlog
+            return False, bootlog
+
+        try:
+            found, bootlog = await asyncio.wait_for(wait_for_login(), self.boot_timeout)
+            if found:
+                return
+        except asyncio.TimeoutError:
+            self.logger.info("Timed out waiting for login prompt.")
+        self.logger.info(b"".join(bootlog.splitlines()[-20:]).decode("utf-8", errors="replace"))
+        raise RuntimeError("Failed to start FVP.")
+
+    def start(self, **kwargs):
+        # When we can assume Py3.7+, this can simply be asyncio.run()
+        loop = asyncio.get_event_loop()
+        loop.run_until_complete(asyncio.gather(self.boot_fvp()))
+
+    def stop(self, **kwargs):
+        loop = asyncio.get_event_loop()
+
+        # Kill the process group so that the telnet and FVP die too
+        gid = os.getpgid(self.fvp.pid)
+
+        try:
+            self.logger.debug(f"Sending SIGTERM to {gid}")
+            os.killpg(gid, signal.SIGTERM)
+            loop.run_until_complete(asyncio.wait_for(self.fvp.wait(), 10))
+        except TimeoutError:
+            self.logger.debug(f"Timed out, sending SIGKILL to {gid}")
+            os.killpg(gid, signal.SIGKILL)
-- 
2.25.1

[PATCH 3/6] arm-bsp/fvp: enable virtio drivers

[Ross Burton]

All FVPs can use virtio networking devices, so enable virtio in all of
the FVP kernels.

Remove our fvp/fvp-virtio.cfg as there's cfg/virtio.scc we can use.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .../linux/arm-platforms-kmeta/bsp/arm-platforms/fvp-arm32.scc | 3 +--
 .../linux/arm-platforms-kmeta/bsp/arm-platforms/fvp.scc       | 3 +--
 .../arm-platforms-kmeta/bsp/arm-platforms/fvp/fvp-virtio.cfg  | 4 ----
 3 files changed, 2 insertions(+), 8 deletions(-)
 delete mode 100644 meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp/fvp-virtio.cfg

diff --git a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp-arm32.scc b/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp-arm32.scc
index 85659aa7..15bb4831 100644
--- a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp-arm32.scc
+++ b/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp-arm32.scc
@@ -1,14 +1,13 @@
 include features/input/input.scc
 include cfg/timer/no_hz.scc
+include cfg/virtio.scc
 
 kconf hardware fvp-arm32/fvp-board.cfg
 kconf hardware fvp-arm32/fvp-features.cfg
 kconf hardware fvp/fvp-net.cfg
 kconf hardware fvp/fvp-rtc.cfg
 kconf hardware fvp/fvp-serial.cfg
-kconf hardware fvp/fvp-virtio.cfg
 kconf hardware fvp/fvp-cfi.cfg
 kconf hardware fvp/fvp-drm.cfg
 kconf hardware fvp/fvp-timer.cfg
-kconf hardware fvp/fvp-virtio.cfg
 kconf hardware fvp/fvp-watchdog.cfg
diff --git a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp.scc b/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp.scc
index 79e3a69a..80b85819 100644
--- a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp.scc
+++ b/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp.scc
@@ -1,14 +1,13 @@
 include features/input/input.scc
 include features/net/net.scc
 include cfg/timer/no_hz.scc
+include cfg/virtio.scc
 
 kconf hardware fvp/fvp-board.cfg
 kconf hardware fvp/fvp-net.cfg
 kconf hardware fvp/fvp-rtc.cfg
 kconf hardware fvp/fvp-serial.cfg
-kconf hardware fvp/fvp-virtio.cfg
 kconf hardware fvp/fvp-cfi.cfg
 kconf hardware fvp/fvp-drm.cfg
 kconf hardware fvp/fvp-timer.cfg
-kconf hardware fvp/fvp-virtio.cfg
 kconf hardware fvp/fvp-watchdog.cfg
diff --git a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp/fvp-virtio.cfg b/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp/fvp-virtio.cfg
deleted file mode 100644
index afc76a09..00000000
--- a/meta-arm-bsp/recipes-kernel/linux/arm-platforms-kmeta/bsp/arm-platforms/fvp/fvp-virtio.cfg
+++ /dev/null
@@ -1,4 +0,0 @@
-CONFIG_VIRTIO=y
-CONFIG_VIRTIO_MMIO=y
-CONFIG_BLOCK=y
-CONFIG_VIRTIO_BLK=y
-- 
2.25.1

[PATCH 4/6] arm-bsp/fvp-common: set TEST_TARGET to OEFVPTarget

[Ross Burton]

Set TEST_TARGET so that all FVP machines use the new FVP target out of
the box, instead of attempting to use qemu.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta-arm-bsp/conf/machine/fvp-common.inc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta-arm-bsp/conf/machine/fvp-common.inc b/meta-arm-bsp/conf/machine/fvp-common.inc
index 2579d94a..9adb4309 100644
--- a/meta-arm-bsp/conf/machine/fvp-common.inc
+++ b/meta-arm-bsp/conf/machine/fvp-common.inc
@@ -25,3 +25,5 @@ EXTRA_IMAGEDEPENDS += "virtual/trusted-firmware-a u-boot"
 # initialise) and install the pre-generated keys.
 PACKAGECONFIG:remove:pn-openssh = "rng-tools"
 MACHINE_EXTRA_RRECOMMENDS += "ssh-pregen-hostkeys"
+
+TEST_TARGET = "OEFVPTarget"
-- 
2.25.1

[PATCH 5/6] CI: install telnet into the images

[Ross Burton]

runfvp currently needs telnet to communicate with the guest, so install
telnet into the image.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .gitlab-ci.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 04e65ffe..f00bdcf8 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -27,6 +27,7 @@ stages:
     - mkdir --verbose --parents $KAS_WORK_DIR $KAS_REPO_REF_DIR $SSTATE_DIR $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR
     # Must do this here, as it's the only way to make sure the toolchain is installed on the same builder
     - ./ci/get-binary-toolchains $DL_DIR $TOOLCHAIN_DIR $TOOLCHAIN_LINK_DIR
+    - sudo apt update && sudo apt install telnet -y
 
 # Generalised fragment to do a Kas build
 .build:
-- 
2.25.1

[PATCH 6/6] CI: enable testimage for fvp-base

[Ross Burton]

Inherit fvpboot so that the FVP binary is fetched and configuration
generated.

Configure the FVP to forward host port 8022 to port 22 locally, and
tell testimage to SSH to localhost:8022. This has the limitation that
only one testimage can run per machine, but this will be removed shortly.

Disable the parselogs test case, as there are some harmless warnings in
the dmesg which cause it to fail.  Currently meta-arm can't extend the
whitelist of ignorable warnings.

The FVP binaries are x86-64 only, so tag the job appropriately.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .gitlab-ci.yml  | 4 +++-
 ci/fvp-base.yml | 9 +++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index f00bdcf8..4a2b3366 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -109,8 +109,10 @@ corstone700-fvp:
 corstone700-mps3:
   extends: .build
 
-fvp-base:
+fvp-base/testimage:
   extends: .build
+  tags:
+  - x86_64
 
 fvp-base-arm32:
   extends: .build
diff --git a/ci/fvp-base.yml b/ci/fvp-base.yml
index 1c3675fa..47d123e3 100644
--- a/ci/fvp-base.yml
+++ b/ci/fvp-base.yml
@@ -5,3 +5,12 @@ header:
 
 machine: fvp-base
 
+local_conf_header:
+  testimagefvp: |
+    INHERIT = "fvpboot"
+    # This fails but we can't add to the ignorelist from meta-arm yet
+    # https://bugzilla.yoctoproject.org/show_bug.cgi?id=14604
+    TEST_SUITES:remove = "parselogs"
+    # Tell testimage to connect to localhost:8022, and forward that to SSH in the FVP.
+    TEST_TARGET_IP = "localhost:8022"
+    FVP_CONFIG[bp.virtio_net.hostbridge.userNetPorts] ?= "8022=22"
-- 
2.25.1