[PATCH v2 0/9] Add barebox bootloader support (and testing)

[PATCH v2 0/9] Add barebox bootloader support (and testing)

[Enrico Jorns]

This adds support for the barebox bootloader (and tools) to oe-core.

In order to have proper testing, this extends oe-selftest to allow
basic testing of bootloaders. While at it, cover both barebox and u-boot.

v2:
  * set myself as maintainer in maintainers.inc
  * move doc from documentation.conf to recipe
  * update to barebox v2023.03 (including fixes for musl, etc.)
  * set standard configs for qemu machines to allow building them
  * add better efi dir support in barebox recipe
  * enable testing bootloaders in oeqa
  * add test cases for barebox (and u-boot)

Enrico Jorns (7):
  barebox: set default BAREBOX_CONFIG for qemu machines
  oeqa/utils/qemurunner: support ignoring vt100 escape sequences
  oeqa/utils/qemurunner: simplify output parsing and make
    crlf-compatible
  oeqa/utils/commands: document runqemu context manager
  oeqa: support passing custom boot patterns to runqemu
  oeqa/selftest/cases: add barebox tests
  oeqa/selftest/cases: add basic u-boot test

Marco Felsch (2):
  barebox: add initial support
  barebox-tools: add initial barebox tools support

 meta/conf/distro/include/maintainers.inc      |   2 +
 meta/lib/oeqa/selftest/cases/barebox.py       |  72 +++++++++
 meta/lib/oeqa/selftest/cases/uboot.py         |  40 +++++
 meta/lib/oeqa/targetcontrol.py                |   5 +-
 meta/lib/oeqa/utils/commands.py               |  22 ++-
 meta/lib/oeqa/utils/qemurunner.py             |  37 +++--
 .../barebox/barebox-tools_2023.03.0.bb        |  56 +++++++
 meta/recipes-bsp/barebox/barebox.inc          | 138 ++++++++++++++++++
 meta/recipes-bsp/barebox/barebox_2023.03.0.bb |   3 +
 9 files changed, 351 insertions(+), 24 deletions(-)
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
 create mode 100644 meta/recipes-bsp/barebox/barebox.inc
 create mode 100644 meta/recipes-bsp/barebox/barebox_2023.03.0.bb

-- 
2.39.2

[PATCH v2 1/9] barebox: add initial support

[Enrico Jorns]

From: Marco Felsch <m.felsch@pengutronix.de>

This adds the support for the barebox bootloader to oe-core. The recipe
is based on the recipe found in meta-ptx [1] with a few minor adaptions.

This basic support includes the bootloader and the target tools to
interact with the bootloader. The host tools support is not part of
this commit. This will be added later on as separate recipe.

[1] https://github.com/pengutronix/meta-ptx/tree/master/recipes-bsp/barebox

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/conf/distro/include/maintainers.inc      |   1 +
 meta/recipes-bsp/barebox/barebox.inc          | 133 ++++++++++++++++++
 meta/recipes-bsp/barebox/barebox_2023.03.0.bb |   3 +
 3 files changed, 137 insertions(+)
 create mode 100644 meta/recipes-bsp/barebox/barebox.inc
 create mode 100644 meta/recipes-bsp/barebox/barebox_2023.03.0.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 682ec2cfdf..86d5519489 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -54,6 +54,7 @@ RECIPE_MAINTAINER:pn-automake = "Robert Yang <liezhi.yang@windriver.com>"
 RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <alex.kanavin@gmail.com>"
+RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <yocto@pengutronix.de>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <alejandro@enedino.org>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <anuj.mittal@intel.com>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <anuj.mittal@intel.com>"
diff --git a/meta/recipes-bsp/barebox/barebox.inc b/meta/recipes-bsp/barebox/barebox.inc
new file mode 100644
index 0000000000..a6c790b84b
--- /dev/null
+++ b/meta/recipes-bsp/barebox/barebox.inc
@@ -0,0 +1,133 @@
+SUMMARY = "barebox is a bootloader designed for embedded systems. It runs on a variety of architectures including x86, ARM, MIPS, PowerPC and others."
+HOMEPAGE = "https://barebox.org/"
+SECTION = "bootloaders"
+PROVIDES = "virtual/bootloader"
+
+LICENSE = "GPL-2.0-only"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f5125d13e000b9ca1f0d3364286c4192"
+
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+require conf/image-uefi.conf
+
+inherit kernel-arch deploy cml1 pkgconfig
+
+# libusb1 is only required for usb tools like imx-usb-loader. These tools may
+# be enabled within the defconfig file so depend on it here to be more user
+# friendly (do not abort the build). The actual tools are provided by the
+# barebox-tools package.
+DEPENDS = "libusb1 bison-native flex-native"
+
+export KBUILD_BUILD_USER ?= "oe-user"
+export KBUILD_BUILD_HOST ?= "oe-host"
+
+SRC_URI = "https://barebox.org/download/barebox-${PV}.tar.bz2"
+
+S = "${WORKDIR}/barebox-${PV}"
+B = "${WORKDIR}/build"
+
+def get_layer_rev(path):
+    try:
+        rev, _ = bb.process.run("git describe --match='' --always --dirty --broken", cwd=path)
+    except bb.process.ExecutionError:
+        rev = ""
+    return rev.strip()
+
+BAREBOX_BUILDSYSTEM_VERSION[doc] = "Build system version to add to the barebox image. By default this is the git description of the containing layer."
+BAREBOX_BUILDSYSTEM_VERSION ??= "${@get_layer_rev(os.path.dirname(d.getVar('FILE')))}"
+
+BAREBOX_FIRMWARE_DIR[doc] = "Overwrite barebox' firmware blobs search directory (CONFIG_EXTRA_FIRMWARE_DIR) with this path, default ${B}/firmware"
+BAREBOX_FIRMWARE_DIR ??= "${B}/firmware"
+
+EXTRA_OEMAKE = " \
+    CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
+    BUILDSYSTEM_VERSION=${BAREBOX_BUILDSYSTEM_VERSION} \
+    CONFIG_EXTRA_FIRMWARE_DIR=${BAREBOX_FIRMWARE_DIR} \
+"
+
+BAREBOX_CONFIG[doc] = "The barebox kconfig defconfig file. Not used if a file called defconfig is added to the SRC_URI."
+BAREBOX_CONFIG ?= ""
+
+do_configure() {
+        if [ -e ${WORKDIR}/defconfig ]; then
+                cp ${WORKDIR}/defconfig ${B}/.config
+        else
+                if [ -n "${BAREBOX_CONFIG}" ]; then
+                        oe_runmake ${BAREBOX_CONFIG}
+                else
+                        bbfatal "No defconfig given. Either add file 'file://defconfig' to SRC_URI or set BAREBOX_CONFIG"
+                fi
+        fi
+
+        ${S}/scripts/kconfig/merge_config.sh -m .config ${@" ".join(find_cfgs(d))}
+        cml1_do_configure
+}
+
+BAREBOX_ENV_DIR[doc] = "Overlay the barebox built-in environment with the environment provided by the BSP if specified."
+BAREBOX_ENV_DIR ??= "${WORKDIR}/env/"
+
+do_compile () {
+        export userccflags="${TARGET_LDFLAGS}${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+        unset LDFLAGS
+        unset CFLAGS
+        unset CPPFLAGS
+        unset CXXFLAGS
+        unset MACHINE
+
+        if [ -d ${BAREBOX_ENV_DIR} ]; then
+                BAREBOX_DEFAULT_ENV="$(grep ^CONFIG_DEFAULT_ENVIRONMENT_PATH .config | cut -d '=' -f 2 | tr -d '"')"
+                oe_runmake CONFIG_DEFAULT_ENVIRONMENT_PATH="\"${BAREBOX_DEFAULT_ENV} ${BAREBOX_ENV_DIR}\""
+        else
+                oe_runmake
+        fi
+}
+
+BAREBOX_BINARY[doc] = "Specify the barebox binary to install. If not specified all barebox artifacts are installed."
+BAREBOX_BINARY ??= "${@'barebox.efi' if d.getVar('EFI_PROVIDER') == 'barebox' else ''}"
+BAREBOX_SUFFIX[doc] = "Specify the suffix for ${BAREBOX_IMAGE}."
+BAREBOX_SUFFIX ??= "img"
+BAREBOX_IMAGE[doc] = "A unique barebox image name. Unused if ${BAREBOX_BINARY} is not set."
+BAREBOX_IMAGE_DEFAULT ?= "${PN}-${MACHINE}-${PV}-${PR}.${BAREBOX_SUFFIX}"
+BAREBOX_IMAGE ?= "${@'${EFI_BOOT_IMAGE}' if d.getVar('EFI_PROVIDER') == 'barebox' else '${BAREBOX_IMAGE_DEFAULT}'}"
+
+BAREBOX_INSTALL_PATH ?= "${@'${EFI_FILES_PATH}' if d.getVar('EFI_PROVIDER') == 'barebox' else '/boot'}"
+
+do_install () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        bbfatal "Failed to locate ${BAREBOX_BINARY}"
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${D}${BAREBOX_INSTALL_PATH}/${BAREBOX_BINARY}
+        else
+                install -d ${D}${BAREBOX_INSTALL_PATH}/
+                for image in $(cat ${B}/barebox-flash-images); do
+                        install -m 644 ${B}/${image} ${D}${BAREBOX_INSTALL_PATH}/
+                done
+        fi
+}
+FILES:${PN} = "${BAREBOX_INSTALL_PATH}"
+
+do_deploy () {
+        if [ -n "${BAREBOX_BINARY}" ]; then
+
+                BAREBOX_BIN=${B}/${BAREBOX_BINARY}
+                if [ ! -f "${BAREBOX_BIN}" ]; then
+                        BAREBOX_BIN=${B}/images/${BAREBOX_BINARY}
+                fi
+
+                install -D -m 644 ${BAREBOX_BIN} ${DEPLOYDIR}/${BAREBOX_IMAGE}
+                ln -sf ${BAREBOX_IMAGE} ${DEPLOYDIR}/${BAREBOX_BINARY}
+        else
+                for image in $(cat ${B}/barebox-flash-images); do
+                        cp ${B}/${image} ${DEPLOYDIR}
+                done
+        fi
+}
+addtask deploy after do_compile
diff --git a/meta/recipes-bsp/barebox/barebox_2023.03.0.bb b/meta/recipes-bsp/barebox/barebox_2023.03.0.bb
new file mode 100644
index 0000000000..860b68a95f
--- /dev/null
+++ b/meta/recipes-bsp/barebox/barebox_2023.03.0.bb
@@ -0,0 +1,3 @@
+require barebox.inc
+
+SRC_URI[sha256sum] = "d0f78a69ba240327247c8fd0e1d45287e4a0dff99ed847e9a696cc2da0cf388c"
-- 
2.39.2

[PATCH v2 2/9] barebox-tools: add initial barebox tools support

[Enrico Jorns]

From: Marco Felsch <m.felsch@pengutronix.de>

Add initial support to build the barebox tools for the host and the
target.

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/conf/distro/include/maintainers.inc      |  1 +
 .../barebox/barebox-tools_2023.03.0.bb        | 56 +++++++++++++++++++
 2 files changed, 57 insertions(+)
 create mode 100644 meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 86d5519489..1a0ba10d3b 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -55,6 +55,7 @@ RECIPE_MAINTAINER:pn-avahi = "Yi Zhao <yi.zhao@windriver.com>"
 RECIPE_MAINTAINER:pn-babeltrace = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-babeltrace2 = "Alexander Kanavin <alex.kanavin@gmail.com>"
 RECIPE_MAINTAINER:pn-barebox = "Enrico Jörns <yocto@pengutronix.de>"
+RECIPE_MAINTAINER:pn-barebox-tools = "Enrico Jörns <yocto@pengutronix.de>"
 RECIPE_MAINTAINER:pn-baremetal-helloworld = "Alejandro Hernandez <alejandro@enedino.org>"
 RECIPE_MAINTAINER:pn-base-files = "Anuj Mittal <anuj.mittal@intel.com>"
 RECIPE_MAINTAINER:pn-base-passwd = "Anuj Mittal <anuj.mittal@intel.com>"
diff --git a/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb b/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
new file mode 100644
index 0000000000..9ab1b6303d
--- /dev/null
+++ b/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
@@ -0,0 +1,56 @@
+SUMMARY = "barebox bootloader tools"
+HOMEPAGE = "https://barebox.org/"
+
+LICENSE = "GPL-2.0-only"
+LIC_FILES_CHKSUM = "file://COPYING;md5=f5125d13e000b9ca1f0d3364286c4192"
+
+DEPENDS = "libusb1 bison-native flex-native"
+
+SRC_URI = "https://barebox.org/download/barebox-${PV}.tar.bz2"
+SRC_URI[sha256sum] = "d0f78a69ba240327247c8fd0e1d45287e4a0dff99ed847e9a696cc2da0cf388c"
+
+S = "${WORKDIR}/barebox-${PV}"
+B = "${WORKDIR}/build"
+
+inherit pkgconfig
+
+EXTRA_OEMAKE = " \
+  ARCH=sandbox \
+  CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
+  CROSS_PKG_CONFIG=pkg-config \
+  "
+
+do_compile:class-target () {
+    export userccflags="${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
+    export userldflags="${TARGET_LDFLAGS}${TOOLCHAIN_OPTIONS}"
+    oe_runmake targettools_defconfig
+    oe_runmake scripts
+}
+
+do_compile:class-native () {
+    oe_runmake hosttools_defconfig
+    oe_runmake scripts
+}
+
+BAREBOX_TOOLS = " \
+         bareboxenv \
+         bareboxcrc32 \
+         kernel-install \
+         bareboximd \
+         omap3-usb-loader \
+         omap4_usbboot \
+         imx/imx-usb-loader \
+         "
+
+BAREBOX_TOOLS_SUFFIX = ""
+BAREBOX_TOOLS_SUFFIX:class-target = "-target"
+
+do_install () {
+	install -d ${D}${bindir}
+
+	for tool in ${BAREBOX_TOOLS}; do
+		install -m 0755 scripts/${tool}${BAREBOX_TOOLS_SUFFIX} ${D}${bindir}/${tool##*/}
+	done
+}
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.39.2

[PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines

[Enrico Jorns]

These are set in the barebox recipe rather in the corresponding machines
(where they would belong otherwise) to keep the impact of barebox to
oe-core minimal for now.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/recipes-bsp/barebox/barebox.inc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta/recipes-bsp/barebox/barebox.inc b/meta/recipes-bsp/barebox/barebox.inc
index a6c790b84b..4827f7b3a4 100644
--- a/meta/recipes-bsp/barebox/barebox.inc
+++ b/meta/recipes-bsp/barebox/barebox.inc
@@ -48,6 +48,11 @@ EXTRA_OEMAKE = " \
 BAREBOX_CONFIG[doc] = "The barebox kconfig defconfig file. Not used if a file called defconfig is added to the SRC_URI."
 BAREBOX_CONFIG ?= ""
 
+# set defaults for oe-core MACHINEs to allow building barebox
+BAREBOX_CONFIG:qemuarm = "vexpress_defconfig"
+BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig"
+BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
+
 do_configure() {
         if [ -e ${WORKDIR}/defconfig ]; then
                 cp ${WORKDIR}/defconfig ${B}/.config
-- 
2.39.2

[PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences

[Enrico Jorns]

If we talk to terminals that like colors, we need to ignore the vt100
escape sequences when matching strings.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/utils/qemurunner.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 05385763ac..95c9e6596c 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -30,6 +30,7 @@ control_range = list(range(0,32))+list(range(127,160))
 control_chars = [chr(x) for x in control_range
                 if chr(x) not in string.printable]
 re_control_char = re.compile('[%s]' % re.escape("".join(control_chars)))
+re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
 
 class QemuRunner:
 
@@ -662,7 +663,7 @@ class QemuRunner:
                 time.sleep(0.1)
                 answer = self.server_socket.recv(1024)
                 if answer:
-                    data += answer.decode('utf-8')
+                    data += re_vt100.sub("", answer.decode('utf-8'))
                     # Search the prompt to stop
                     if re.search(self.boot_patterns['search_cmd_finished'], data):
                         break
-- 
2.39.2

[PATCH v2 5/9] oeqa/utils/qemurunner: simplify output parsing and make crlf-compatible

[Enrico Jorns]

Reduces the nesting depth and replaces the manual newline matching by
built-in splitlines() method.

This makes it compatible with shells that use windows-compatible line
breaks, e.g. for EFI loaders.

More comments and an early return handling should make the code a bit
more readable.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/utils/qemurunner.py | 32 +++++++++++++++----------------
 1 file changed, 15 insertions(+), 17 deletions(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 95c9e6596c..1e544acc90 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -672,23 +672,21 @@ class QemuRunner:
                         return (1, "")
                     raise Exception("No data on serial console socket, connection closed?")
 
-        if data:
-            if raw:
-                status = 1
-            else:
-                # Remove first line (command line) and last line (prompt)
-                data = data[data.find('$?\r\n')+4:data.rfind('\r\n')]
-                index = data.rfind('\r\n')
-                if index == -1:
-                    status_cmd = data
-                    data = ""
-                else:
-                    status_cmd = data[index+2:]
-                    data = data[:index]
-                if (status_cmd == "0"):
-                    status = 1
-        return (status, str(data))
-
+        # If we got no data, we assume something went wrong and return 0
+        if not data:
+            return (0, str(None))
+
+        # in raw mode, we cannot check exit status output and thus assume success
+        if raw:
+            return (1, str(data))
+
+        # Split lines into array and remove first line (command line) and last line (prompt)
+        # Also remove empty lines to ease catching results
+        outlines = list(filter(None, data.splitlines()[1:-1]))
+        # Remaining last line contains exit code output
+        if (outlines[-1] == "0"):
+            status = 1
+        return (status, "\n".join(outlines[0:-1]))
 
     def _dump_host(self):
         self.host_dumper.create_dir("qemu")
-- 
2.39.2

[PATCH v2 6/9] oeqa/utils/commands: document runqemu context manager

[Enrico Jorns]

Usage and arguments are not necessarily self-explaining.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/utils/commands.py | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index f4daea2507..efbb6ab6f2 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -314,7 +314,22 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec=
 @contextlib.contextmanager
 def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True):
     """
-    launch_cmd means directly run the command, don't need set rootfs or env vars.
+    Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource.
+    The underlying Qemu will be booted into a shell when the generator yields.
+
+    Usage:
+
+        with runqemu('core-image-minimal') as qemu:
+            qemu.run_serial('cat /proc/cpuinfo')
+
+    Args:
+        pn (str): (image) recipe to run on
+        ssh (boolean): whether or not to enable SSH (network access)
+        runqemuparams (str): space-separated list of params to pass to 'runqemu' script (like 'nographics', 'ovmf', etc.)
+        image_fstype (str): IMAGE_FSTYPE to use
+        launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation
+        overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables
+        discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image
     """
 
     import bb.tinfoil
-- 
2.39.2

[PATCH v2 7/9] oeqa: support passing custom boot patterns to runqemu

[Enrico Jorns]

This allows defining non-standard patterns from QEMU tests just as they
are already supported by the testimage.bbclass.

Will allow testing non-Linux shells in QEMU, too, e.g. a U-Boot or other
bootloader shell.

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/targetcontrol.py    | 5 +++--
 meta/lib/oeqa/utils/commands.py   | 5 +++--
 meta/lib/oeqa/utils/qemurunner.py | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index 1fdff82889..43a213ac0e 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -92,7 +92,7 @@ class QemuTarget(BaseTarget):
 
     supported_image_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic']
 
-    def __init__(self, d, logger, image_fstype=None):
+    def __init__(self, d, logger, image_fstype=None, boot_patterns=None):
 
         import oe.types
 
@@ -148,7 +148,8 @@ class QemuTarget(BaseTarget):
                             dump_host_cmds = d.getVar("testimage_dump_host"),
                             logger = logger,
                             tmpfsdir = d.getVar("RUNQEMU_TMPFS_DIR"),
-                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()))
+                            serial_ports = len(d.getVar("SERIAL_CONSOLES").split()),
+                            boot_patterns = boot_patterns)
 
         self.target_dumper = TargetDumper(dump_target_cmds, dump_dir, self.runner)
         self.monitor_dumper = MonitorDumper(dump_monitor_cmds, dump_dir, self.runner)
diff --git a/meta/lib/oeqa/utils/commands.py b/meta/lib/oeqa/utils/commands.py
index efbb6ab6f2..1f6a617367 100644
--- a/meta/lib/oeqa/utils/commands.py
+++ b/meta/lib/oeqa/utils/commands.py
@@ -312,7 +312,7 @@ def create_temp_layer(templayerdir, templayername, priority=999, recipepathspec=
         f.write('LAYERSERIES_COMPAT_%s = "%s"\n' % (templayername, corenames))
 
 @contextlib.contextmanager
-def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, discard_writes=True):
+def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemuparams=None, overrides={}, boot_patterns = {}, discard_writes=True):
     """
     Starts a context manager for a 'oeqa.targetcontrol.QemuTarget' resource.
     The underlying Qemu will be booted into a shell when the generator yields.
@@ -329,6 +329,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
         image_fstype (str): IMAGE_FSTYPE to use
         launch_cmd (str): directly run this command and bypass automatic runqemu parameter generation
         overrides (dict): dict of "'<bitbake-variable>': value" pairs that allows overriding bitbake variables
+        boot_patterns (dict): dict of "'<pattern-name>': value" pairs to override default boot patterns, e.g. when not booting Linux
         discard_writes (boolean): enables qemu -snapshot feature to prevent modifying original image
     """
 
@@ -360,7 +361,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None,
 
         logdir = recipedata.getVar("TEST_LOG_DIR")
 
-        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype)
+        qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype, boot_patterns=boot_patterns)
     finally:
         # We need to shut down tinfoil early here in case we actually want
         # to run tinfoil-using utilities with the running QEMU instance.
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 1e544acc90..0d64df7253 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -93,7 +93,7 @@ class QemuRunner:
 
         # Only override patterns that were set e.g. login user TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
         for pattern in accepted_patterns:
-            if not self.boot_patterns[pattern]:
+            if pattern not in self.boot_patterns or not self.boot_patterns[pattern]:
                 self.boot_patterns[pattern] = default_boot_patterns[pattern]
 
     def create_socket(self):
-- 
2.39.2

[PATCH v2 8/9] oeqa/selftest/cases: add barebox tests

[Enrico Jorns]

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/selftest/cases/barebox.py | 72 +++++++++++++++++++++++++
 1 file changed, 72 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/barebox.py

diff --git a/meta/lib/oeqa/selftest/cases/barebox.py b/meta/lib/oeqa/selftest/cases/barebox.py
new file mode 100644
index 0000000000..497cd6e8dd
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/barebox.py
@@ -0,0 +1,72 @@
+# Qemu-based barebox bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator.data import skipIfNotArch
+from oeqa.core.decorator import OETestTag
+
+barebox_boot_patterns = {
+        'search_reached_prompt': r"stop autoboot",
+        'search_login_succeeded': r"barebox@[^:]+:[^ ]+ ",
+        'search_cmd_finished': r"barebox@[a-zA-Z0-9\-\s]+:/"
+        }
+
+
+class BareboxTest(OESelftestTestCase):
+
+    @skipIfNotArch(['arm', 'aarch64'])
+    @OETestTag("runqemu")
+    def test_boot_barebox(self):
+        """
+        Tests building barebox and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_KERNEL = "barebox-dt-2nd.img"
+PREFERRED_PROVIDER_virtual/bootloader = "barebox"
+""")
+
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=barebox_boot_patterns) as qemu:
+
+            # test if barebox console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("barebox" in output, msg=output)
+
+    @skipIfNotArch(['x86_64'])
+    @OETestTag("runqemu")
+    def test_boot_barebox_efi(self):
+        """
+        Tests building barebox for UEFI and booting it as EFI payload
+        with QEMU + OVMF
+        """
+        image = "core-image-minimal"
+
+        self.write_config("""
+IMAGE_INSTALL:append = " barebox"
+MACHINE_FEATURES:append = " pcbios efi"
+EXTRA_IMAGEDEPENDS += "ovmf"
+EFI_PROVIDER = "barebox"
+IMAGE_FSTYPES += "wic"
+WKS_FILE = "efi-bootdisk.wks.in"
+""")
+
+        bitbake(image)
+
+        with runqemu(image, ssh=False, runqemuparams='nographic ovmf',
+                     boot_patterns=barebox_boot_patterns, image_fstype='wic') as qemu:
+
+            # test if barebox console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("barebox" in output, msg=output)
-- 
2.39.2

[PATCH v2 9/9] oeqa/selftest/cases: add basic u-boot test

[Enrico Jorns]

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
---
 meta/lib/oeqa/selftest/cases/uboot.py | 40 +++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/uboot.py

diff --git a/meta/lib/oeqa/selftest/cases/uboot.py b/meta/lib/oeqa/selftest/cases/uboot.py
new file mode 100644
index 0000000000..d3c60c3052
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/uboot.py
@@ -0,0 +1,40 @@
+# Qemu-based u-boot bootloader integration testing
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, runqemu
+from oeqa.core.decorator import OETestTag
+
+uboot_boot_patterns = {
+        'search_reached_prompt': "stop autoboot",
+        'search_login_succeeded': "=>",
+        'search_cmd_finished': "=>"
+        }
+
+
+class UBootTest(OESelftestTestCase):
+
+    @OETestTag("runqemu")
+    def test_boot_uboot(self):
+        """
+        Tests building u-boot and booting it with QEMU
+        """
+
+        self.write_config("""
+QB_DEFAULT_BIOS = "u-boot.bin"
+PREFERRED_PROVIDER_virtual/bootloader = "u-boot"
+""")
+        bitbake("virtual/bootloader core-image-minimal")
+
+        with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic',
+                     boot_patterns=uboot_boot_patterns) as qemu:
+
+            # test if u-boot console works
+            cmd = "version"
+            status, output = qemu.run_serial(cmd)
+            self.assertEqual(status, 1, msg=output)
+            self.assertTrue("U-Boot" in output, msg=output)
-- 
2.39.2

Re: [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Alexander Kanavin]

On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> v2:
>   * set myself as maintainer in maintainers.inc
>   * move doc from documentation.conf to recipe
>   * update to barebox v2023.03 (including fixes for musl, etc.)
>   * set standard configs for qemu machines to allow building them
>   * add better efi dir support in barebox recipe
>   * enable testing bootloaders in oeqa
>   * add test cases for barebox (and u-boot)

Thanks for persevering with this! The patchset is mostly fine (I'll
write a couple comments now), and just needs to get a verdict from the
autobuilder.

Alex

Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines

[Alexander Kanavin]

On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> +# set defaults for oe-core MACHINEs to allow building barebox
> +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig"
> +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig"
> +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"

As all three are different perhaps there could be commentary in the
recipe explaining why that and how they differ? Note that there's also
a qemuall override if there's a single configuration that works for
all qemu targets.

Alex

Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences

[Alexander Kanavin]

On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> +re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')

This piece of 'magic' needs to be explained :) What does it define?

Alex

Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines

[Enrico Jörns]

Hi Alex,

Am Freitag, dem 31.03.2023 um 16:04 +0200 schrieb Alexander Kanavin:
> On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> > +# set defaults for oe-core MACHINEs to allow building barebox
> > +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig"
> > +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig"
> > +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
> 
> As all three are different perhaps there could be commentary in the
> recipe explaining why that and how they differ? Note that there's also
> a qemuall override if there's a single configuration that works for
> all qemu targets.

as these three are different archs/machines, I do not find it surprising that they require different
configs. I assumed that this was self-explaining.

Note that for UBOOT, you have

in qemux86-64.conf:

  UBOOT_MACHINE ?= "qemu-x86_64_defconfig"

in qemuarm.conf:

  UBOOT_MACHINE ?= "qemu_arm_defconfig"

in qemuarm64.conf:

  UBOOT_MACHINE ?= "qemu_arm64_defconfig"

which is pretty much the same as for barebox, isn't it? Only the naming scheme is a bit more
consistent probably.


Thanks and regards

Enrico

> 
> Alex
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences

[Enrico Jörns]

Hi Alex,

Am Freitag, dem 31.03.2023 um 16:05 +0200 schrieb Alexander Kanavin:
> On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> > +re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
> 
> This piece of 'magic' needs to be explained :) What does it define?

this is actually 'stolen' from labgrid's implementation for barebox[1] or shell[2] drivers which
have exactly the same needs: They need to remove the ansii (color) control codes from the strings in
order to match the text only.

The unprocessed barebox console prompt would look like:

  ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m

where we cannot match for something like "barebox@ARM QEMU virt64:/".
The same applies to colored linux terminal output of course.

The "\x1b\[" from the regex catches the standard start of ansii escape sequence while the rest
catches the actual command code executed.


Regards, Enrico

[1] https://github.com/labgrid-project/labgrid/blob/e05f530c22513cd775b4f84e28f6c8c920b95102/labgrid/driver/bareboxdriver.py#L44 
[2] https://github.com/labgrid-project/labgrid/blob/5d3d5714976b37df0f9a9b769c581fe1f83ccbdc/labgrid/driver/shelldriver.py#L61

> Alex
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Richard Purdie]

On Fri, 2023-03-31 at 12:40 +0200, Enrico Jorns wrote:
> This adds support for the barebox bootloader (and tools) to oe-core.
> 
> In order to have proper testing, this extends oe-selftest to allow
> basic testing of bootloaders. While at it, cover both barebox and u-boot.
> 
> v2:
>   * set myself as maintainer in maintainers.inc
>   * move doc from documentation.conf to recipe
>   * update to barebox v2023.03 (including fixes for musl, etc.)
>   * set standard configs for qemu machines to allow building them
>   * add better efi dir support in barebox recipe
>   * enable testing bootloaders in oeqa
>   * add test cases for barebox (and u-boot)
> 
> Enrico Jorns (7):
>   barebox: set default BAREBOX_CONFIG for qemu machines
>   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
>   oeqa/utils/qemurunner: simplify output parsing and make
>     crlf-compatible
>   oeqa/utils/commands: document runqemu context manager
>   oeqa: support passing custom boot patterns to runqemu
>   oeqa/selftest/cases: add barebox tests
>   oeqa/selftest/cases: add basic u-boot test
> 
> Marco Felsch (2):
>   barebox: add initial support
>   barebox-tools: add initial barebox tools support

Testing on the autobuilder did trigger a few issues:

on musl:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932

and some failing oe-selftests:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
[there were more but look similar]

Cheers,

Richard

Re: [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Enrico Jörns]

Hi Richard,

Am Montag, dem 03.04.2023 um 16:49 +0100 schrieb Richard Purdie:
> On Fri, 2023-03-31 at 12:40 +0200, Enrico Jorns wrote:
> > This adds support for the barebox bootloader (and tools) to oe-core.
> > 
> > In order to have proper testing, this extends oe-selftest to allow
> > basic testing of bootloaders. While at it, cover both barebox and u-boot.
> > 
> > v2:
> >   * set myself as maintainer in maintainers.inc
> >   * move doc from documentation.conf to recipe
> >   * update to barebox v2023.03 (including fixes for musl, etc.)
> >   * set standard configs for qemu machines to allow building them
> >   * add better efi dir support in barebox recipe
> >   * enable testing bootloaders in oeqa
> >   * add test cases for barebox (and u-boot)
> > 
> > Enrico Jorns (7):
> >   barebox: set default BAREBOX_CONFIG for qemu machines
> >   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
> >   oeqa/utils/qemurunner: simplify output parsing and make
> >     crlf-compatible
> >   oeqa/utils/commands: document runqemu context manager
> >   oeqa: support passing custom boot patterns to runqemu
> >   oeqa/selftest/cases: add barebox tests
> >   oeqa/selftest/cases: add basic u-boot test
> > 
> > Marco Felsch (2):
> >   barebox: add initial support
> >   barebox-tools: add initial barebox tools support
> 
> Testing on the autobuilder did trigger a few issues:
> 
> on musl:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
> https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932
> 
> and some failing oe-selftests:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
> [there were more but look similar]

thank you for the autobuilder logs!

I fear we are running into a conceptional issue that I already feared we could run into (but did not
test for yet).

If I traced this down correctly, then the 'issue' is:

| NOTE: Multiple providers are available for virtual/bootloader (barebox, u-boot)
| Consider defining a PREFERRED_PROVIDER entry to match virtual/bootloader

The fitimage tests building "virtual/bootloader", silently assuming this is equal to u-boot (which
it actually was so far).

Now, the quick fix for this would be to properly set the correct PREFERRED_PROVIDER here, but I
could imagine that we run into similar issues in other builds, too.

Do you have a good solution for this at hand? Like defining a standard preferred provider for
virtual/bootloader?

I guess one could argue possibly that the mechanism does exactly what it is meant for. And a few
machines (like beaglebone-yocto or qemuloongarch) actually set the PREFERRED_PROVIDER, but not all
and this might lead to silently switching the bootloader (like in our tests).

I hope you have a better overview on the topic than I have and could give me a hint on how you would
like to see this being resolved?

Thanks in advance and best regards

Enrico


> Cheers,
> 
> Richard
> 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [OE-core] [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Khem Raj]

On Mon, Apr 3, 2023 at 2:20 PM Enrico Jörns <ejo@pengutronix.de> wrote:
>
> Hi Richard,
>
> Am Montag, dem 03.04.2023 um 16:49 +0100 schrieb Richard Purdie:
> > On Fri, 2023-03-31 at 12:40 +0200, Enrico Jorns wrote:
> > > This adds support for the barebox bootloader (and tools) to oe-core.
> > >
> > > In order to have proper testing, this extends oe-selftest to allow
> > > basic testing of bootloaders. While at it, cover both barebox and u-boot.
> > >
> > > v2:
> > >   * set myself as maintainer in maintainers.inc
> > >   * move doc from documentation.conf to recipe
> > >   * update to barebox v2023.03 (including fixes for musl, etc.)
> > >   * set standard configs for qemu machines to allow building them
> > >   * add better efi dir support in barebox recipe
> > >   * enable testing bootloaders in oeqa
> > >   * add test cases for barebox (and u-boot)
> > >
> > > Enrico Jorns (7):
> > >   barebox: set default BAREBOX_CONFIG for qemu machines
> > >   oeqa/utils/qemurunner: support ignoring vt100 escape sequences
> > >   oeqa/utils/qemurunner: simplify output parsing and make
> > >     crlf-compatible
> > >   oeqa/utils/commands: document runqemu context manager
> > >   oeqa: support passing custom boot patterns to runqemu
> > >   oeqa/selftest/cases: add barebox tests
> > >   oeqa/selftest/cases: add basic u-boot test
> > >
> > > Marco Felsch (2):
> > >   barebox: add initial support
> > >   barebox-tools: add initial barebox tools support
> >
> > Testing on the autobuilder did trigger a few issues:
> >
> > on musl:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/6932
> >
> > and some failing oe-selftests:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/4971
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5023
> > [there were more but look similar]
>
> thank you for the autobuilder logs!
>
> I fear we are running into a conceptional issue that I already feared we could run into (but did not
> test for yet).
>
> If I traced this down correctly, then the 'issue' is:
>
> | NOTE: Multiple providers are available for virtual/bootloader (barebox, u-boot)
> | Consider defining a PREFERRED_PROVIDER entry to match virtual/bootloader
>
> The fitimage tests building "virtual/bootloader", silently assuming this is equal to u-boot (which
> it actually was so far).
>
> Now, the quick fix for this would be to properly set the correct PREFERRED_PROVIDER here, but I
> could imagine that we run into similar issues in other builds, too.
>
> Do you have a good solution for this at hand? Like defining a standard preferred provider for
> virtual/bootloader?
>
> I guess one could argue possibly that the mechanism does exactly what it is meant for. And a few
> machines (like beaglebone-yocto or qemuloongarch) actually set the PREFERRED_PROVIDER, but not all
> and this might lead to silently switching the bootloader (like in our tests).
>
> I hope you have a better overview on the topic than I have and could give me a hint on how you would
> like to see this being resolved?

I see that it fails to build with clang
https://errors.yoctoproject.org/Errors/Details/699467/

is it buildable with clang ? if not we can pin it to use gcc always in
meta-clang but I want to make sure first.


>
> Thanks in advance and best regards
>
> Enrico
>
>
> > Cheers,
> >
> > Richard
> >
> >
>
> --
> Pengutronix e.K.                           | Enrico Jörns                |
> Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
> Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Enrico Jörns]

Hi Khem,

> Am Montag, dem 03.04.2023 um 19:33 -0700 schrieb Khem Raj:


> [...]

> I see that it fails to build with clang
> https://errors.yoctoproject.org/Errors/Details/699467/

thanks for testing this!

> is it buildable with clang ? if not we can pin it to use gcc always in
> meta-clang but I want to make sure first.

barebox itself is unfortunately not buildable with clang since on some architectures like ARM32 it
uses GCC extensions that are not implemented by clang.

The barebox-tools however that are built for 'sandbox' should be buildable with clang.
It's just that when passing CROSS_COMPILE, barebox cannot guess if it has to append 'clang' or 'gcc'
suffix. The kernel seems to have an explicit handling for this by checking for "ifneq ($(LLVM),)"
and plugging together CC differently.

With barebox we could however still override the Makefile's default "CC = $(CROSS_COMPILE)gcc" when
passing CC explicitly to EXTRA_OEMAKE. We have just successfully tested this patch:

--- a/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
+++ b/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
@@ -18,6 +18,8 @@ EXTRA_OEMAKE = " \
   ARCH=sandbox \
   CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
   CROSS_PKG_CONFIG=pkg-config \
+  CC='${CC}' \
+  LD='${LD}' \
   "


If that looks fine for you, I could add it to the recipe.


Best regards, Enrico

> 
> > 
> > Thanks in advance and best regards
> > 
> > Enrico
> > 
> > 
> > > Cheers,
> > > 
> > > Richard
> > > 
> > > 
> > 
> > --
> > Pengutronix e.K.                           | Enrico Jörns                |
> > Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> > Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> > 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
> > 
> > 
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
> > Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> > 
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |

Re: [OE-core] [PATCH v2 0/9] Add barebox bootloader support (and testing)

[Khem Raj]

On Tue, Apr 4, 2023 at 4:13 AM Enrico Jörns <ejo@pengutronix.de> wrote:
>
> Hi Khem,
>
> > Am Montag, dem 03.04.2023 um 19:33 -0700 schrieb Khem Raj:
>
>
> > [...]
>
> > I see that it fails to build with clang
> > https://errors.yoctoproject.org/Errors/Details/699467/
>
> thanks for testing this!
>
> > is it buildable with clang ? if not we can pin it to use gcc always in
> > meta-clang but I want to make sure first.
>
> barebox itself is unfortunately not buildable with clang since on some architectures like ARM32 it
> uses GCC extensions that are not implemented by clang.
>
> The barebox-tools however that are built for 'sandbox' should be buildable with clang.
> It's just that when passing CROSS_COMPILE, barebox cannot guess if it has to append 'clang' or 'gcc'
> suffix. The kernel seems to have an explicit handling for this by checking for "ifneq ($(LLVM),)"
> and plugging together CC differently.
>
> With barebox we could however still override the Makefile's default "CC = $(CROSS_COMPILE)gcc" when
> passing CC explicitly to EXTRA_OEMAKE. We have just successfully tested this patch:
>
> --- a/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
> +++ b/meta/recipes-bsp/barebox/barebox-tools_2023.03.0.bb
> @@ -18,6 +18,8 @@ EXTRA_OEMAKE = " \
>    ARCH=sandbox \
>    CROSS_COMPILE=${TARGET_PREFIX} -C ${S} O=${B} \
>    CROSS_PKG_CONFIG=pkg-config \
> +  CC='${CC}' \
> +  LD='${LD}' \
>    "
>
>
> If that looks fine for you, I could add it to the recipe.
>

yeah I was testing one with CC alone and that worked too but also
containing LD is good, so please include above
changes in next patchset

>
> Best regards, Enrico
>
> >
> > >
> > > Thanks in advance and best regards
> > >
> > > Enrico
> > >
> > >
> > > > Cheers,
> > > >
> > > > Richard
> > > >
> > > >
> > >
> > > --
> > > Pengutronix e.K.                           | Enrico Jörns                |
> > > Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> > > Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> > > 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
> > >
> > >
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > > Links: You receive all messages sent to this group.
> > > View/Reply Online (#179655): https://lists.openembedded.org/g/openembedded-core/message/179655
> > > Mute This Topic: https://lists.openembedded.org/mt/97970644/1997914
> > > Group Owner: openembedded-core+owner@lists.openembedded.org
> > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > > -=-=-=-=-=-=-=-=-=-=-=-
> > >
> >
>
> --
> Pengutronix e.K.                           | Enrico Jörns                |
> Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
>

Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines

[Alexander Kanavin]

On Mon, 3 Apr 2023 at 13:42, Enrico Jörns <ejo@pengutronix.de> wrote:
> > > +# set defaults for oe-core MACHINEs to allow building barebox
> > > +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig"
> > > +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig"
> > > +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
> >
> > As all three are different perhaps there could be commentary in the
> > recipe explaining why that and how they differ? Note that there's also
> > a qemuall override if there's a single configuration that works for
> > all qemu targets.
>
> as these three are different archs/machines, I do not find it surprising that they require different
> configs. I assumed that this was self-explaining.

Not at all. What is this 'vexpress' thingy? What does 'express' mean?
Why does it apply to arm, but not arm64? Why arm64 is using
'qemu_virt64', but not any other targets, if that config does not seem
arm-specific? Why is x86 using 'efi' but not the arm targets?

The choices you made here should be explained.

Alex

Re: [PATCH v2 4/9] oeqa/utils/qemurunner: support ignoring vt100 escape sequences

[Alexander Kanavin]

Right, but this information should be recorded in commit message, or
the file itself.

Alex

On Mon, 3 Apr 2023 at 15:02, Enrico Jörns <ejo@pengutronix.de> wrote:
>
> Hi Alex,
>
> Am Freitag, dem 31.03.2023 um 16:05 +0200 schrieb Alexander Kanavin:
> > On Fri, 31 Mar 2023 at 12:40, Enrico Jorns <ejo@pengutronix.de> wrote:
> > > +re_vt100 = re.compile(r'(\x1b\[|\x9b)[^@-_a-z]*[@-_a-z]|\x1b[@-_a-z]')
> >
> > This piece of 'magic' needs to be explained :) What does it define?
>
> this is actually 'stolen' from labgrid's implementation for barebox[1] or shell[2] drivers which
> have exactly the same needs: They need to remove the ansii (color) control codes from the strings in
> order to match the text only.
>
> The unprocessed barebox console prompt would look like:
>
>   ESC[1;32mbarebox@ESC[1;36mARM QEMU virt64:/ESC[0m
>
> where we cannot match for something like "barebox@ARM QEMU virt64:/".
> The same applies to colored linux terminal output of course.
>
> The "\x1b\[" from the regex catches the standard start of ansii escape sequence while the rest
> catches the actual command code executed.
>
>
> Regards, Enrico
>
> [1] https://github.com/labgrid-project/labgrid/blob/e05f530c22513cd775b4f84e28f6c8c920b95102/labgrid/driver/bareboxdriver.py#L44
> [2] https://github.com/labgrid-project/labgrid/blob/5d3d5714976b37df0f9a9b769c581fe1f83ccbdc/labgrid/driver/shelldriver.py#L61
>
> > Alex
> >
>
> --
> Pengutronix e.K.                           | Enrico Jörns                |
> Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
> Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
> 31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |
>

Re: [PATCH v2 3/9] barebox: set default BAREBOX_CONFIG for qemu machines

[Enrico Jörns]

Am Dienstag, dem 11.04.2023 um 16:16 +0200 schrieb Alexander Kanavin:
> On Mon, 3 Apr 2023 at 13:42, Enrico Jörns <ejo@pengutronix.de> wrote:
> > > > +# set defaults for oe-core MACHINEs to allow building barebox
> > > > +BAREBOX_CONFIG:qemuarm = "vexpress_defconfig"
> > > > +BAREBOX_CONFIG:qemuarm64 = "qemu_virt64_defconfig"
> > > > +BAREBOX_CONFIG:qemux86-64 = "efi_defconfig"
> > > 
> > > As all three are different perhaps there could be commentary in the
> > > recipe explaining why that and how they differ? Note that there's also
> > > a qemuall override if there's a single configuration that works for
> > > all qemu targets.
> > 
> > as these three are different archs/machines, I do not find it surprising that they require
> > different
> > configs. I assumed that this was self-explaining.
> 
> Not at all. What is this 'vexpress' thingy? What does 'express' mean?
> Why does it apply to arm, but not arm64? Why arm64 is using
> 'qemu_virt64', but not any other targets, if that config does not seem
> arm-specific? Why is x86 using 'efi' but not the arm targets?
> 
> The choices you made here should be explained.

Vexpress is "Versatile Express" which once was the ARMv7 reference platform and is one of the
platforms emulated in QEMU (when using "-machine vexpress-a9" or "-machine vexpress-a15").

virt, or virt64 are the generic QEMU ARM platform. For historical reasons, there is only a dedicated
vexpress defconfig in barebox.

EFI is used for x86 only, since on ARM barebox is normally used as the primary loader only.


However, since barebox builds are multi-platform builds by default, it should be possible to use the
standard multi_v7_defconfig and multi_v8_defconfig instead (for arm). I'll give it a try.

Regards, Enrico

> 
> Alex
> 

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Embedded Linux Consulting & Support        | https://www.pengutronix.de/ |
Steuerwalder Str. 21                       | Phone: +49-5121-206917-180  |
31137 Hildesheim, Germany                  | Fax:   +49-5121-206917-9    |