[meta-virtualization][PATCH 0/2] Xen: qemuboot device tree support (Arm 64-bit)

[meta-virtualization][PATCH 0/2] Xen: qemuboot device tree support (Arm 64-bit)

[Christopher Clark]

This series introduces tools and class structure in the layer for
enabling testing Xen system images with Qemu, booting with a device tree
that describes the target system.

A device tree is required for launching Xen systems on Arm platforms and
is part of the Hyperlaunch design for Xen.
  https://wiki.xenproject.org/wiki/Hyperlaunch

This series adds a recipe for Lopper, a device tree tool, and its
dependencies. It provides a class that implements a bitbake task to
generate a device tree for qemuboot, and a set of defaults to enable
'runqemu xen-image-minimal nographic slirp' to boot images compiled for
Arm 64-bit machines.

Christopher Clark (2):
  lopper, python-dtc: add new recipes for device tree tools
  qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images

 classes/qemuboot-xen-defaults.bbclass        |  29 +++
 classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
 conf/distro/include/meta-virt-xen.inc        |   3 +
 recipes-extended/images/xen-image-minimal.bb |   6 +-
 recipes-kernel/dtc/python3-dtc_1.6.1.bb      |  26 +++
 recipes-kernel/lopper/lopper.bb              |  45 ++++
 6 files changed, 316 insertions(+), 2 deletions(-)
 create mode 100644 classes/qemuboot-xen-defaults.bbclass
 create mode 100644 classes/qemuboot-xen-dtb.bbclass
 create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
 create mode 100644 recipes-kernel/lopper/lopper.bb

-- 
2.25.1

[meta-virtualization][PATCH 1/2] lopper, python-dtc: add new recipes for device tree tools

[Christopher Clark]

From: Christopher Clark <christopher.clark@starlab.io>

This device tree tooling is being added as a prequisite for enabling
qemuboot to run Xen system images with device trees generated for Qemu.

lopper: a tool for performing operations on device tree files.
A new recipe inspired by the original from meta-xilinx-bsp.

python-dtc: a python library for the Device Tree compiler.
This is a prerequisite for lopper. Imported from meta-xilinx-bsp with an
updated SRCREV and DESCRIPTION for the latest release, version 1.6.1.

Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
---
 recipes-kernel/dtc/python3-dtc_1.6.1.bb | 26 ++++++++++++++
 recipes-kernel/lopper/lopper.bb         | 45 +++++++++++++++++++++++++
 2 files changed, 71 insertions(+)
 create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
 create mode 100644 recipes-kernel/lopper/lopper.bb

diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
new file mode 100644
index 0000000..5374029
--- /dev/null
+++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
@@ -0,0 +1,26 @@
+SUMMARY = "Python Library for the Device Tree Compiler"
+HOMEPAGE = "https://devicetree.org/"
+DESCRIPTION = "A python library for the Device Tree Compiler, a tool used to manipulate Device Tree files which contain a data structure for describing hardware."
+SECTION = "bootloader"
+LICENSE = "GPLv2 | BSD"
+
+DEPENDS = "flex-native bison-native swig-native libyaml dtc"
+
+SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git"
+
+UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
+
+LIC_FILES_CHKSUM = "file://libfdt.i;beginline=1;endline=6;md5=afda088c974174a29108c8d80b5dce90"
+
+SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
+
+S = "${WORKDIR}/git/pylibfdt"
+
+inherit distutils3
+
+do_configure_prepend() {
+    make -C "${S}/.." version_gen.h
+    mv "${S}/../version_gen.h" "${S}/"
+}
+
+BBCLASSEXTEND = "native nativesdk"
diff --git a/recipes-kernel/lopper/lopper.bb b/recipes-kernel/lopper/lopper.bb
new file mode 100644
index 0000000..274d5b9
--- /dev/null
+++ b/recipes-kernel/lopper/lopper.bb
@@ -0,0 +1,45 @@
+SUMMARY = "Device Tree Lopper"
+DESCRIPTION = "Tool for manipulation of system device tree files"
+LICENSE = "BSD-3-Clause"
+SECTION = "bootloader"
+
+SRC_URI = "git://github.com/devicetree-org/lopper.git"
+SRCREV = "0c23e51d585608ab04009ccf3499f168e2f4d478"
+S = "${WORKDIR}/git"
+
+LIC_FILES_CHKSUM = "file://LICENSE.md;md5=8e5f5f691f01c9fdfa7a7f2d535be619"
+
+RDEPENDS_${PN} = " \
+    python3-core \
+    python3-dtc \
+    python3-humanfriendly \
+    "
+
+do_install() {
+    install -d "${D}/${bindir}"
+    install -d "${D}/${datadir}/${BPN}"
+
+    install -m 0644 "${S}/README" "${D}/${datadir}/${BPN}"
+    install -m 0644 "${S}/README-architecture.txt" "${D}/${datadir}/${BPN}"
+    install -m 0644 "${S}/README.pydoc" "${D}/${datadir}/${BPN}"
+    install -m 0644 "${S}/LICENSE.md" "${D}/${datadir}/${BPN}"
+
+    install -d "${D}/${datadir}/${BPN}/assists"
+    install -m 0644 "${S}/assists/"* "${D}/${datadir}/${BPN}/assists/"
+
+    install -d "${D}/${datadir}/${BPN}/lops"
+    install -m 0644 "${S}/lops/"* "${D}/${datadir}/${BPN}/lops/"
+
+    install -d "${D}/${datadir}/${BPN}/device-trees"
+    install -m 0644 "${S}/device-trees/"* "${D}/${datadir}/${BPN}/device-trees/"
+
+    install -m 0755 "${S}/"lopper*.py "${D}/${datadir}/${BPN}/"
+
+    datadir_relpath=${@os.path.relpath(d.getVar('datadir'), d.getVar('bindir'))}
+    ln -s "${datadir_relpath}/${BPN}/lopper.py" "${D}/${bindir}/"
+}
+
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.25.1

[meta-virtualization][PATCH 2/2] qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images

[Christopher Clark]

From: Christopher Clark <christopher.clark@starlab.io>

The Xen hypervisor built for Arm 64-bit targets can be launched with
runqemu by providing a Device Tree binary and configuration for Qemu,
which enables interactive testing of Xen images.

Add qemuboot-xen-dtb.bbclass to add a new bitbake task for generating
the dtb file by using lopper on a device tree produced by Qemu.

Add qemuboot-xen-defaults.bbclass to supply working default parameters
for the qemuarm64 machine and general support for qemuboot for Xen.

Add the new qemuboot-xen classes to the xen-image-minimal image, along
with variable settings needed for qemuboot to run the image for
qemuarm64.

The memory setting for Qemu has been increased to 512M after testing
with successful boots. Default fstype for qemux86-64 was found to need a
stronger override in testing. Provide an override for the serial console
setting for the qemuarm64 machine when the Xen distro feature is
enabled for the console in qemuboot, following the example of similar
machine settings in meta-arm-bsp.

Development aided by this script by Stewart Hildebrand of DornerWorks:
https://gist.github.com/stewdk/110f43e0cc1d905fc6ed4c7e10d8d35e

Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
---
 classes/qemuboot-xen-defaults.bbclass        |  29 +++
 classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
 conf/distro/include/meta-virt-xen.inc        |   3 +
 recipes-extended/images/xen-image-minimal.bb |   6 +-
 4 files changed, 245 insertions(+), 2 deletions(-)
 create mode 100644 classes/qemuboot-xen-defaults.bbclass
 create mode 100644 classes/qemuboot-xen-dtb.bbclass

diff --git a/classes/qemuboot-xen-defaults.bbclass b/classes/qemuboot-xen-defaults.bbclass
new file mode 100644
index 0000000..82cad49
--- /dev/null
+++ b/classes/qemuboot-xen-defaults.bbclass
@@ -0,0 +1,29 @@
+# Set defaults for booting Xen images with qemuboot
+
+# Xen and Dom0 command line options
+QB_XEN_CMDLINE_EXTRA ??= "dom0_mem=192M"
+QB_XEN_DOM0_BOOTARGS ??= \
+    "console=hvc0 earlyprintk clk_ignore_unused root=/dev/vda"
+
+# Launch with one initial domain, dom0, with one boot module, the kernel
+DOM0_KERNEL="${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}"
+DOM0_KERNEL_LOAD_ADDR="0x45000000"
+QB_XEN_DOMAIN_MODULES = "${DOM0_KERNEL}:${DOM0_KERNEL_LOAD_ADDR}:multiboot,kernel"
+
+# Qemuboot for 64-bit Arm uses the QB_DEFAULT_KERNEL method to load Xen
+# and the device loader option for the dom0 kernel:
+QB_OPT_APPEND_append_aarch64 = " \
+    -device loader,file=${DOM0_KERNEL},addr=${DOM0_KERNEL_LOAD_ADDR} \
+    "
+QB_DEFAULT_KERNEL_aarch64 = "xen-${MACHINE}"
+
+# 64-bit Arm: gic version 3
+QB_MACHINE_aarch64 = "-machine virt,gic-version=3 -machine virtualization=true"
+
+# Use overrides for stronger assignment here as default memory set by the
+# machines is not high enough
+QB_MEM_qemuarm64 = "-m 512"
+
+# 64-bit Arm: qemuboot with a device tree binary
+QB_DTB_aarch64 = "${IMAGE_NAME}.qemuboot.dtb"
+QB_DTB_LINK_aarch64 = "${IMAGE_LINK_NAME}.qemuboot.dtb"
diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
new file mode 100644
index 0000000..08f9b02
--- /dev/null
+++ b/classes/qemuboot-xen-dtb.bbclass
@@ -0,0 +1,209 @@
+# Enable booting Xen with qemuboot / runqemu: generate device tree
+#
+# Copyright (c) 2021 Star Lab Corp. All rights reserved.
+#
+# Author: Christopher Clark <christopher.clark@starlab.io>
+
+# Interface variables:
+#
+# QB_DTB : defined in qemuboot.bbclass.
+# If set, this class will generate the specified device tree file.
+#
+# QB_XEN_CMDLINE_EXTRA :
+# A string to be appended to the default Xen hypervisor boot command line,
+# for supplying Xen boot options.
+# The device tree that this bbclass generates will contain Xen command
+# line options to connect the Xen console to the Qemu serial port.
+#
+# QB_XEN_DOMAIN_MODULES :
+# A space-separated list of colon-separated entries:
+#   "<file for the module>:<load memory address>:<module compatibility string>"
+#
+# QB_XEN_DOM0_BOOTARGS :
+# A string for specifying Dom0 boot options for the Xen section of the device
+# tree.
+#
+# QB_XEN_DTB_TASK_DEPENDS :
+# The task dependencies for the DTB generation. A default is provided.
+#
+# See also: Other QB_ variables as defined by the qemuboot.bbclass.
+
+write_lops_xen_section() {
+    cat <<EOF >"$1"
+/dts-v1/;
+/ {
+    compatible = "system-device-tree-v1";
+    lops {
+        /* Connect the Xen console to Qemu dtuart */
+        lop_1 {
+            compatible = "system-device-tree-v1,lop,code-v1";
+            code = "
+                chosen = node.tree['/chosen']
+                stdout_path = str(chosen['stdout-path'].value[0])
+                chosen['xen,xen-bootargs'] = \
+                        'console=dtuart dtuart=%s' % stdout_path
+                return True
+            ";
+        };
+        lop_2 {
+            compatible = "system-device-tree-v1,lop,modify";
+            modify = "/chosen:xen,dom0-bootargs:${QB_XEN_DOM0_BOOTARGS}";
+        };
+        lop_3 {
+            compatible = "system-device-tree-v1,lop,modify";
+            modify = "/chosen:#address-cells:<1>";
+        };
+        lop_4 {
+            compatible = "system-device-tree-v1,lop,modify";
+            modify = "/chosen:#size-cells:<1>";
+        };
+     };
+};
+EOF
+}
+
+write_lop_add_to_xen_cmdline() {
+    EXTRA_XEN_BOOTARGS="$2"
+    cat <<EOF >"$1"
+/dts-v1/;
+/ {
+    compatible = "system-device-tree-v1";
+    lops {
+        lop_1 {
+            compatible = "system-device-tree-v1,lop,code-v1";
+            options = "extra_xen_bootargs:${EXTRA_XEN_BOOTARGS}";
+            code = "
+                chosen = node.tree['/chosen']
+                xen_bootargs = str(chosen['xen,xen-bootargs'].value)
+                chosen['xen,xen-bootargs'] = '%s %s' % \
+                     (xen_bootargs, extra_xen_bootargs)
+                return True
+            ";
+        };
+     };
+};
+EOF
+}
+
+write_lop_add_chosen_module() {
+    ADDR="$2"
+    SIZE="$3"
+    MODULE_TYPE="$4"
+    cat <<EOF >"$1"
+/dts-v1/;
+/ {
+    compatible = "system-device-tree-v1";
+    lops {
+        lop_1 {
+            compatible = "system-device-tree-v1,lop,add";
+            node_src = "module@${ADDR}";
+            node_dest = "/chosen/module@${ADDR}";
+
+            module@${ADDR} {
+                compatible = "multiboot,module", "${MODULE_TYPE}";
+                reg = <${ADDR} ${SIZE}>;
+            };
+        };
+     };
+};
+EOF
+}
+
+generate_xen_qemuboot_dtb() {
+    # First: invoke qemu to generate an initial device tree.
+    # Parameters supplied here inspired by inspection of:
+    #   runqemu "${IMAGE_BASENAME}" nographic slirp \
+    #            qemuparams='-dtb "" -machine dumpdtb=${B}/qemu-dumped.dtb'
+    ${QB_SYSTEM_NAME} \
+        -device qemu-xhci \
+        -device usb-tablet \
+        -device usb-kbd \
+        -machine virt,gic-version=3 \
+        -machine virtualization=true \
+        ${QB_CPU} \
+        ${QB_SMP} \
+        ${QB_MEM} \
+        -nographic \
+        -serial mon:stdio \
+        -machine "dumpdtb=${B}/qemu-dumped.dtb"
+
+    # Lopper generates temporary files in cwd, so run it within ${B}
+    cd "${B}"
+    write_lops_xen_section "${B}/lop-insert-xen-section.dts"
+
+    write_lop_add_to_xen_cmdline "${B}/lop-xen-cmdline.dts" \
+        "${QB_XEN_CMDLINE_EXTRA}"
+
+    if [ -z "${QB_XEN_DOMAIN_MODULES}" ]; then
+        bbwarn "No domain modules: please set QB_XEN_DOMAIN_MODULES"
+    fi
+
+    for DOMAIN_MODULE in ${QB_XEN_DOMAIN_MODULES}
+    do
+        MODULE_FILE="$(echo ${DOMAIN_MODULE} | cut -f1 -d:)"
+        ADDR="$(echo ${DOMAIN_MODULE} | cut -f2 -d:)"
+        MODULE_TYPE="$(echo ${DOMAIN_MODULE} | cut -f3 -d:)"
+        RESOLVED_FILE="$(readlink -f ${MODULE_FILE})"
+        SIZE=$(printf '0x%x\n' $(stat -c '%s' "${RESOLVED_FILE}"))
+        [ "x${SIZE}" != "x0x0" ] || bbfatal No module: "${MODULE_FILE}"
+        write_lop_add_chosen_module "${B}/lop-add-module-${ADDR}.dts" \
+            "${ADDR}" "${SIZE}" "${MODULE_TYPE}"
+        LOP_MODULE_ARGS="${LOP_MODULE_ARGS} -i ${B}/lop-add-module-${ADDR}.dts"
+    done
+
+    QEMUBOOT_DTB="${IMGDEPLOYDIR}/${QB_DTB}"
+    QEMUBOOT_DTB_LINK="${IMGDEPLOYDIR}/${QB_DTB_LINK}"
+
+    lopper.py --werror --verbose \
+        -i "${B}/lop-insert-xen-section.dts" \
+        -i "${B}/lop-xen-cmdline.dts" \
+        ${LOP_MODULE_ARGS} \
+        -f -o "${QEMUBOOT_DTB}" \
+            "${B}/qemu-dumped.dtb"
+
+    # To assist debugging:
+    dtc -I dtb -O dts -o "${B}/output.dts" "${QEMUBOOT_DTB}"
+
+    if [ "${QEMUBOOT_DTB_LINK}" != "${QEMUBOOT_DTB}" ] ; then
+        if [ -e "${QEMUBOOT_DTB_LINK}" ] ; then
+            rm "${QEMUBOOT_DTB_LINK}"
+        fi
+        ln -s "${QB_DTB}" "${QEMUBOOT_DTB_LINK}"
+    fi
+}
+
+do_write_xen_qemuboot_dtb() {
+    # Not all architectures qemuboot with a device tree binary, so check
+    # to see if one is needed. This allows this bbclass file to be used
+    # in the same image recipe for multiple architectures.
+    if [ -n "${QB_DTB}" ] ; then
+        generate_xen_qemuboot_dtb
+    fi
+}
+
+addtask do_write_xen_qemuboot_dtb after do_write_qemuboot_conf before do_image
+# Task dependency:
+# An expected common case is that the kernel for at least one of the initial
+# domains (eg. dom0) is deployed from the virtual/kernel recipe, so
+# add virtual/kernel:do_deploy as a task dependency here since the kernel size
+# needs to be known for generating the device tree.
+# Dependencies are only introduced if a device tree will be generated.
+QB_XEN_DTB_TASK_DEPENDS ?= " \
+    ${@[ ' \
+        qemu-helper-native:do_populate_sysroot \
+        lopper-native:do_populate_sysroot \
+        dtc-native:do_populate_sysroot \
+        virtual/kernel:do_deploy \
+    ', ''][d.getVar('QB_DTB') == '' or d.getVar('QB_DTB') is None]} \
+    "
+do_write_xen_qemuboot_dtb[depends] = "${QB_XEN_DTB_TASK_DEPENDS}"
+
+def qemuboot_dtb_vars(d):
+    build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE',
+                'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME',
+                'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE',
+                'STAGING_DIR_HOST', 'SERIAL_CONSOLES']
+    return build_vars + [k for k in d.keys() if k.startswith('QB_')]
+
+do_write_qemuboot_dtb[vardeps] += "${@' '.join(qemuboot_dtb_vars(d))}"
+do_write_qemuboot_dtb[vardepsexclude] += "TOPDIR"
diff --git a/conf/distro/include/meta-virt-xen.inc b/conf/distro/include/meta-virt-xen.inc
index 02a7810..2becba1 100644
--- a/conf/distro/include/meta-virt-xen.inc
+++ b/conf/distro/include/meta-virt-xen.inc
@@ -10,3 +10,6 @@ XEN_RPI4_64_CONFIG_PATH ??= \
     'dynamic-layers/raspberrypi/conf/distro/include/xen-raspberrypi4-64.inc'
 include ${@bb.utils.contains('MACHINE', 'raspberrypi4-64', \
     '${XEN_RPI4_64_CONFIG_PATH}', '', d)}
+
+# Set serial for working qemuboot console
+SERIAL_CONSOLES_qemuarm64 ?= "115200;ttyAMA0"
diff --git a/recipes-extended/images/xen-image-minimal.bb b/recipes-extended/images/xen-image-minimal.bb
index 6733801..2175e63 100644
--- a/recipes-extended/images/xen-image-minimal.bb
+++ b/recipes-extended/images/xen-image-minimal.bb
@@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
 
 LICENSE = "MIT"
 
-inherit core-image
+inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
 
 do_check_xen_state() {
     if [ "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' yes', 'no', d)}" = "no" ]; then
@@ -74,10 +74,12 @@ build_syslinux_cfg () {
 
 # Enable runqemu. eg: runqemu xen-image-minimal nographic slirp
 WKS_FILE_x86-64 = "directdisk-xen.wks"
-QB_MEM ?= "-m 400"
+QB_MEM ?= "-m 512"
 QB_DEFAULT_KERNEL ?= "none"
 QB_DEFAULT_FSTYPE ?= "wic"
+QB_DEFAULT_FSTYPE_qemux86-64 = "wic"
 QB_FSINFO ?= "wic:kernel-in-fs"
+QB_SERIAL_OPT = "-serial mon:stdio"
 # qemux86-64 machine does not include 'wic' in IMAGE_FSTYPES, which is needed
 # to boot this image, so add it here:
 IMAGE_FSTYPES_qemux86-64 += "wic"
-- 
2.25.1

Re: [meta-virtualization][PATCH 0/2] Xen: qemuboot device tree support (Arm 64-bit)

[Bruce Ashfield]

On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
<christopher.w.clark@gmail.com> wrote:
>
> This series introduces tools and class structure in the layer for
> enabling testing Xen system images with Qemu, booting with a device tree
> that describes the target system.
>
> A device tree is required for launching Xen systems on Arm platforms and
> is part of the Hyperlaunch design for Xen.
>   https://wiki.xenproject.org/wiki/Hyperlaunch
>
> This series adds a recipe for Lopper, a device tree tool, and its
> dependencies. It provides a class that implements a bitbake task to
> generate a device tree for qemuboot, and a set of defaults to enable
> 'runqemu xen-image-minimal nographic slirp' to boot images compiled for
> Arm 64-bit machines.

This looks great, and something that I've been looking for as well.

I just need to carve out a bit more time to take it for a spin, but I
don't see any issues with the changes. (I've reviewed the new recipes,
etc, and nothing major jumped out).

Bruce

>
> Christopher Clark (2):
>   lopper, python-dtc: add new recipes for device tree tools
>   qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images
>
>  classes/qemuboot-xen-defaults.bbclass        |  29 +++
>  classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
>  conf/distro/include/meta-virt-xen.inc        |   3 +
>  recipes-extended/images/xen-image-minimal.bb |   6 +-
>  recipes-kernel/dtc/python3-dtc_1.6.1.bb      |  26 +++
>  recipes-kernel/lopper/lopper.bb              |  45 ++++
>  6 files changed, 316 insertions(+), 2 deletions(-)
>  create mode 100644 classes/qemuboot-xen-defaults.bbclass
>  create mode 100644 classes/qemuboot-xen-dtb.bbclass
>  create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
>  create mode 100644 recipes-kernel/lopper/lopper.bb
>
> --
> 2.25.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 1/2] lopper, python-dtc: add new recipes for device tree tools

[Bruce Ashfield]

On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
<christopher.w.clark@gmail.com> wrote:
>
> From: Christopher Clark <christopher.clark@starlab.io>
>
> This device tree tooling is being added as a prequisite for enabling
> qemuboot to run Xen system images with device trees generated for Qemu.
>
> lopper: a tool for performing operations on device tree files.
> A new recipe inspired by the original from meta-xilinx-bsp.
>
> python-dtc: a python library for the Device Tree compiler.
> This is a prerequisite for lopper. Imported from meta-xilinx-bsp with an
> updated SRCREV and DESCRIPTION for the latest release, version 1.6.1.
>
> Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> ---
>  recipes-kernel/dtc/python3-dtc_1.6.1.bb | 26 ++++++++++++++
>  recipes-kernel/lopper/lopper.bb         | 45 +++++++++++++++++++++++++
>  2 files changed, 71 insertions(+)
>  create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
>  create mode 100644 recipes-kernel/lopper/lopper.bb
>
> diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> new file mode 100644
> index 0000000..5374029
> --- /dev/null
> +++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> @@ -0,0 +1,26 @@
> +SUMMARY = "Python Library for the Device Tree Compiler"
> +HOMEPAGE = "https://devicetree.org/"
> +DESCRIPTION = "A python library for the Device Tree Compiler, a tool used to manipulate Device Tree files which contain a data structure for describing hardware."
> +SECTION = "bootloader"
> +LICENSE = "GPLv2 | BSD"
> +
> +DEPENDS = "flex-native bison-native swig-native libyaml dtc"
> +
> +SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git"
> +
> +UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
> +
> +LIC_FILES_CHKSUM = "file://libfdt.i;beginline=1;endline=6;md5=afda088c974174a29108c8d80b5dce90"
> +
> +SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
> +
> +S = "${WORKDIR}/git/pylibfdt"
> +
> +inherit distutils3
> +
> +do_configure_prepend() {

Sorry for being so slow on the review of this, I was deep into getting
the umoci backend to work for OCI image creation and am just getting
to reviews now!

> +    make -C "${S}/.." version_gen.h

Two minor thoughts on the line above:

 - did you try oe_runmake ? versus the bare 'make' call ? I always
forget to do that myself.
 - the ${S}/.. concerned me at first, until I took a close look at
${S}. Could we use "${WORKDIR}/git/" instead ? That doesn't make it
look like we are scribbling outside of ${S}.

> +    mv "${S}/../version_gen.h" "${S}/"
> +}
> +
> +BBCLASSEXTEND = "native nativesdk"
> diff --git a/recipes-kernel/lopper/lopper.bb b/recipes-kernel/lopper/lopper.bb
> new file mode 100644
> index 0000000..274d5b9
> --- /dev/null
> +++ b/recipes-kernel/lopper/lopper.bb
> @@ -0,0 +1,45 @@
> +SUMMARY = "Device Tree Lopper"
> +DESCRIPTION = "Tool for manipulation of system device tree files"
> +LICENSE = "BSD-3-Clause"
> +SECTION = "bootloader"
> +
> +SRC_URI = "git://github.com/devicetree-org/lopper.git"
> +SRCREV = "0c23e51d585608ab04009ccf3499f168e2f4d478"
> +S = "${WORKDIR}/git"
> +
> +LIC_FILES_CHKSUM = "file://LICENSE.md;md5=8e5f5f691f01c9fdfa7a7f2d535be619"
> +
> +RDEPENDS_${PN} = " \
> +    python3-core \
> +    python3-dtc \
> +    python3-humanfriendly \
> +    "
> +
> +do_install() {
> +    install -d "${D}/${bindir}"
> +    install -d "${D}/${datadir}/${BPN}"
> +
> +    install -m 0644 "${S}/README" "${D}/${datadir}/${BPN}"
> +    install -m 0644 "${S}/README-architecture.txt" "${D}/${datadir}/${BPN}"
> +    install -m 0644 "${S}/README.pydoc" "${D}/${datadir}/${BPN}"
> +    install -m 0644 "${S}/LICENSE.md" "${D}/${datadir}/${BPN}"
> +
> +    install -d "${D}/${datadir}/${BPN}/assists"
> +    install -m 0644 "${S}/assists/"* "${D}/${datadir}/${BPN}/assists/"
> +
> +    install -d "${D}/${datadir}/${BPN}/lops"
> +    install -m 0644 "${S}/lops/"* "${D}/${datadir}/${BPN}/lops/"
> +
> +    install -d "${D}/${datadir}/${BPN}/device-trees"
> +    install -m 0644 "${S}/device-trees/"* "${D}/${datadir}/${BPN}/device-trees/"
> +
> +    install -m 0755 "${S}/"lopper*.py "${D}/${datadir}/${BPN}/"
> +
> +    datadir_relpath=${@os.path.relpath(d.getVar('datadir'), d.getVar('bindir'))}
> +    ln -s "${datadir_relpath}/${BPN}/lopper.py" "${D}/${bindir}/"

This looks good to me. I'm working on getting lopper into a pypi
format, but I'll take care of updating this recipe once that work is
complete.

Bruce

> +}
> +
> +do_configure[noexec] = "1"
> +do_compile[noexec] = "1"
> +
> +BBCLASSEXTEND = "native nativesdk"
> --
> 2.25.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 2/2] qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images

[Bruce Ashfield]

On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
<christopher.w.clark@gmail.com> wrote:
>
> From: Christopher Clark <christopher.clark@starlab.io>
>
> The Xen hypervisor built for Arm 64-bit targets can be launched with
> runqemu by providing a Device Tree binary and configuration for Qemu,
> which enables interactive testing of Xen images.
>
> Add qemuboot-xen-dtb.bbclass to add a new bitbake task for generating
> the dtb file by using lopper on a device tree produced by Qemu.
>
> Add qemuboot-xen-defaults.bbclass to supply working default parameters
> for the qemuarm64 machine and general support for qemuboot for Xen.
>
> Add the new qemuboot-xen classes to the xen-image-minimal image, along
> with variable settings needed for qemuboot to run the image for
> qemuarm64.
>
> The memory setting for Qemu has been increased to 512M after testing
> with successful boots. Default fstype for qemux86-64 was found to need a
> stronger override in testing. Provide an override for the serial console
> setting for the qemuarm64 machine when the Xen distro feature is
> enabled for the console in qemuboot, following the example of similar
> machine settings in meta-arm-bsp.
>
> Development aided by this script by Stewart Hildebrand of DornerWorks:
> https://gist.github.com/stewdk/110f43e0cc1d905fc6ed4c7e10d8d35e
>
> Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> ---
>  classes/qemuboot-xen-defaults.bbclass        |  29 +++
>  classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
>  conf/distro/include/meta-virt-xen.inc        |   3 +
>  recipes-extended/images/xen-image-minimal.bb |   6 +-
>  4 files changed, 245 insertions(+), 2 deletions(-)
>  create mode 100644 classes/qemuboot-xen-defaults.bbclass
>  create mode 100644 classes/qemuboot-xen-dtb.bbclass
>
> diff --git a/classes/qemuboot-xen-defaults.bbclass b/classes/qemuboot-xen-defaults.bbclass
> new file mode 100644
> index 0000000..82cad49
> --- /dev/null
> +++ b/classes/qemuboot-xen-defaults.bbclass
> @@ -0,0 +1,29 @@
> +# Set defaults for booting Xen images with qemuboot
> +
> +# Xen and Dom0 command line options
> +QB_XEN_CMDLINE_EXTRA ??= "dom0_mem=192M"
> +QB_XEN_DOM0_BOOTARGS ??= \
> +    "console=hvc0 earlyprintk clk_ignore_unused root=/dev/vda"
> +
> +# Launch with one initial domain, dom0, with one boot module, the kernel
> +DOM0_KERNEL="${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}"
> +DOM0_KERNEL_LOAD_ADDR="0x45000000"
> +QB_XEN_DOMAIN_MODULES = "${DOM0_KERNEL}:${DOM0_KERNEL_LOAD_ADDR}:multiboot,kernel"
> +
> +# Qemuboot for 64-bit Arm uses the QB_DEFAULT_KERNEL method to load Xen
> +# and the device loader option for the dom0 kernel:
> +QB_OPT_APPEND_append_aarch64 = " \
> +    -device loader,file=${DOM0_KERNEL},addr=${DOM0_KERNEL_LOAD_ADDR} \
> +    "
> +QB_DEFAULT_KERNEL_aarch64 = "xen-${MACHINE}"
> +
> +# 64-bit Arm: gic version 3
> +QB_MACHINE_aarch64 = "-machine virt,gic-version=3 -machine virtualization=true"
> +
> +# Use overrides for stronger assignment here as default memory set by the
> +# machines is not high enough
> +QB_MEM_qemuarm64 = "-m 512"

What about making that memory value a variable (with 512 as the
default) ? That allows us to change it without needing to always
modify the bbclass.

Almost all of my K8S runqemu launches need 512 or 1G of memory, so
this is definitely not uncommon!

> +
> +# 64-bit Arm: qemuboot with a device tree binary
> +QB_DTB_aarch64 = "${IMAGE_NAME}.qemuboot.dtb"
> +QB_DTB_LINK_aarch64 = "${IMAGE_LINK_NAME}.qemuboot.dtb"
> diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
> new file mode 100644
> index 0000000..08f9b02
> --- /dev/null
> +++ b/classes/qemuboot-xen-dtb.bbclass
> @@ -0,0 +1,209 @@
> +# Enable booting Xen with qemuboot / runqemu: generate device tree
> +#
> +# Copyright (c) 2021 Star Lab Corp. All rights reserved.
> +#
> +# Author: Christopher Clark <christopher.clark@starlab.io>
> +
> +# Interface variables:
> +#
> +# QB_DTB : defined in qemuboot.bbclass.
> +# If set, this class will generate the specified device tree file.
> +#
> +# QB_XEN_CMDLINE_EXTRA :
> +# A string to be appended to the default Xen hypervisor boot command line,
> +# for supplying Xen boot options.
> +# The device tree that this bbclass generates will contain Xen command
> +# line options to connect the Xen console to the Qemu serial port.
> +#
> +# QB_XEN_DOMAIN_MODULES :
> +# A space-separated list of colon-separated entries:
> +#   "<file for the module>:<load memory address>:<module compatibility string>"
> +#
> +# QB_XEN_DOM0_BOOTARGS :
> +# A string for specifying Dom0 boot options for the Xen section of the device
> +# tree.
> +#
> +# QB_XEN_DTB_TASK_DEPENDS :
> +# The task dependencies for the DTB generation. A default is provided.
> +#
> +# See also: Other QB_ variables as defined by the qemuboot.bbclass.
> +
> +write_lops_xen_section() {
> +    cat <<EOF >"$1"
> +/dts-v1/;
> +/ {
> +    compatible = "system-device-tree-v1";
> +    lops {
> +        /* Connect the Xen console to Qemu dtuart */
> +        lop_1 {
> +            compatible = "system-device-tree-v1,lop,code-v1";
> +            code = "
> +                chosen = node.tree['/chosen']
> +                stdout_path = str(chosen['stdout-path'].value[0])
> +                chosen['xen,xen-bootargs'] = \
> +                        'console=dtuart dtuart=%s' % stdout_path
> +                return True
> +            ";
> +        };
> +        lop_2 {
> +            compatible = "system-device-tree-v1,lop,modify";
> +            modify = "/chosen:xen,dom0-bootargs:${QB_XEN_DOM0_BOOTARGS}";
> +        };
> +        lop_3 {
> +            compatible = "system-device-tree-v1,lop,modify";
> +            modify = "/chosen:#address-cells:<1>";
> +        };
> +        lop_4 {
> +            compatible = "system-device-tree-v1,lop,modify";
> +            modify = "/chosen:#size-cells:<1>";
> +        };
> +     };
> +};
> +EOF
> +}
> +
> +write_lop_add_to_xen_cmdline() {
> +    EXTRA_XEN_BOOTARGS="$2"
> +    cat <<EOF >"$1"
> +/dts-v1/;
> +/ {
> +    compatible = "system-device-tree-v1";
> +    lops {
> +        lop_1 {
> +            compatible = "system-device-tree-v1,lop,code-v1";
> +            options = "extra_xen_bootargs:${EXTRA_XEN_BOOTARGS}";
> +            code = "
> +                chosen = node.tree['/chosen']
> +                xen_bootargs = str(chosen['xen,xen-bootargs'].value)
> +                chosen['xen,xen-bootargs'] = '%s %s' % \
> +                     (xen_bootargs, extra_xen_bootargs)
> +                return True
> +            ";
> +        };
> +     };
> +};
> +EOF
> +}
> +
> +write_lop_add_chosen_module() {
> +    ADDR="$2"
> +    SIZE="$3"
> +    MODULE_TYPE="$4"
> +    cat <<EOF >"$1"
> +/dts-v1/;
> +/ {
> +    compatible = "system-device-tree-v1";
> +    lops {
> +        lop_1 {
> +            compatible = "system-device-tree-v1,lop,add";
> +            node_src = "module@${ADDR}";
> +            node_dest = "/chosen/module@${ADDR}";
> +
> +            module@${ADDR} {
> +                compatible = "multiboot,module", "${MODULE_TYPE}";
> +                reg = <${ADDR} ${SIZE}>;
> +            };
> +        };
> +     };
> +};
> +EOF
> +}
> +
> +generate_xen_qemuboot_dtb() {
> +    # First: invoke qemu to generate an initial device tree.
> +    # Parameters supplied here inspired by inspection of:
> +    #   runqemu "${IMAGE_BASENAME}" nographic slirp \
> +    #            qemuparams='-dtb "" -machine dumpdtb=${B}/qemu-dumped.dtb'
> +    ${QB_SYSTEM_NAME} \
> +        -device qemu-xhci \
> +        -device usb-tablet \
> +        -device usb-kbd \
> +        -machine virt,gic-version=3 \
> +        -machine virtualization=true \
> +        ${QB_CPU} \
> +        ${QB_SMP} \
> +        ${QB_MEM} \
> +        -nographic \
> +        -serial mon:stdio \
> +        -machine "dumpdtb=${B}/qemu-dumped.dtb"
> +
> +    # Lopper generates temporary files in cwd, so run it within ${B}
> +    cd "${B}"
> +    write_lops_xen_section "${B}/lop-insert-xen-section.dts"
> +
> +    write_lop_add_to_xen_cmdline "${B}/lop-xen-cmdline.dts" \
> +        "${QB_XEN_CMDLINE_EXTRA}"
> +
> +    if [ -z "${QB_XEN_DOMAIN_MODULES}" ]; then
> +        bbwarn "No domain modules: please set QB_XEN_DOMAIN_MODULES"
> +    fi
> +
> +    for DOMAIN_MODULE in ${QB_XEN_DOMAIN_MODULES}
> +    do
> +        MODULE_FILE="$(echo ${DOMAIN_MODULE} | cut -f1 -d:)"
> +        ADDR="$(echo ${DOMAIN_MODULE} | cut -f2 -d:)"
> +        MODULE_TYPE="$(echo ${DOMAIN_MODULE} | cut -f3 -d:)"
> +        RESOLVED_FILE="$(readlink -f ${MODULE_FILE})"
> +        SIZE=$(printf '0x%x\n' $(stat -c '%s' "${RESOLVED_FILE}"))
> +        [ "x${SIZE}" != "x0x0" ] || bbfatal No module: "${MODULE_FILE}"
> +        write_lop_add_chosen_module "${B}/lop-add-module-${ADDR}.dts" \
> +            "${ADDR}" "${SIZE}" "${MODULE_TYPE}"
> +        LOP_MODULE_ARGS="${LOP_MODULE_ARGS} -i ${B}/lop-add-module-${ADDR}.dts"
> +    done
> +
> +    QEMUBOOT_DTB="${IMGDEPLOYDIR}/${QB_DTB}"
> +    QEMUBOOT_DTB_LINK="${IMGDEPLOYDIR}/${QB_DTB_LINK}"
> +
> +    lopper.py --werror --verbose \
> +        -i "${B}/lop-insert-xen-section.dts" \
> +        -i "${B}/lop-xen-cmdline.dts" \
> +        ${LOP_MODULE_ARGS} \
> +        -f -o "${QEMUBOOT_DTB}" \
> +            "${B}/qemu-dumped.dtb"
> +
> +    # To assist debugging:
> +    dtc -I dtb -O dts -o "${B}/output.dts" "${QEMUBOOT_DTB}"
> +
> +    if [ "${QEMUBOOT_DTB_LINK}" != "${QEMUBOOT_DTB}" ] ; then
> +        if [ -e "${QEMUBOOT_DTB_LINK}" ] ; then
> +            rm "${QEMUBOOT_DTB_LINK}"
> +        fi
> +        ln -s "${QB_DTB}" "${QEMUBOOT_DTB_LINK}"
> +    fi
> +}
> +
> +do_write_xen_qemuboot_dtb() {
> +    # Not all architectures qemuboot with a device tree binary, so check
> +    # to see if one is needed. This allows this bbclass file to be used
> +    # in the same image recipe for multiple architectures.
> +    if [ -n "${QB_DTB}" ] ; then
> +        generate_xen_qemuboot_dtb
> +    fi
> +}
> +
> +addtask do_write_xen_qemuboot_dtb after do_write_qemuboot_conf before do_image
> +# Task dependency:
> +# An expected common case is that the kernel for at least one of the initial
> +# domains (eg. dom0) is deployed from the virtual/kernel recipe, so
> +# add virtual/kernel:do_deploy as a task dependency here since the kernel size
> +# needs to be known for generating the device tree.
> +# Dependencies are only introduced if a device tree will be generated.
> +QB_XEN_DTB_TASK_DEPENDS ?= " \
> +    ${@[ ' \
> +        qemu-helper-native:do_populate_sysroot \
> +        lopper-native:do_populate_sysroot \
> +        dtc-native:do_populate_sysroot \
> +        virtual/kernel:do_deploy \
> +    ', ''][d.getVar('QB_DTB') == '' or d.getVar('QB_DTB') is None]} \
> +    "
> +do_write_xen_qemuboot_dtb[depends] = "${QB_XEN_DTB_TASK_DEPENDS}"
> +
> +def qemuboot_dtb_vars(d):
> +    build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE',
> +                'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME',
> +                'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE',
> +                'STAGING_DIR_HOST', 'SERIAL_CONSOLES']
> +    return build_vars + [k for k in d.keys() if k.startswith('QB_')]
> +
> +do_write_qemuboot_dtb[vardeps] += "${@' '.join(qemuboot_dtb_vars(d))}"
> +do_write_qemuboot_dtb[vardepsexclude] += "TOPDIR"
> diff --git a/conf/distro/include/meta-virt-xen.inc b/conf/distro/include/meta-virt-xen.inc
> index 02a7810..2becba1 100644
> --- a/conf/distro/include/meta-virt-xen.inc
> +++ b/conf/distro/include/meta-virt-xen.inc
> @@ -10,3 +10,6 @@ XEN_RPI4_64_CONFIG_PATH ??= \
>      'dynamic-layers/raspberrypi/conf/distro/include/xen-raspberrypi4-64.inc'
>  include ${@bb.utils.contains('MACHINE', 'raspberrypi4-64', \
>      '${XEN_RPI4_64_CONFIG_PATH}', '', d)}
> +
> +# Set serial for working qemuboot console
> +SERIAL_CONSOLES_qemuarm64 ?= "115200;ttyAMA0"
> diff --git a/recipes-extended/images/xen-image-minimal.bb b/recipes-extended/images/xen-image-minimal.bb
> index 6733801..2175e63 100644
> --- a/recipes-extended/images/xen-image-minimal.bb
> +++ b/recipes-extended/images/xen-image-minimal.bb
> @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
>
>  LICENSE = "MIT"
>
> -inherit core-image
> +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
>
>  do_check_xen_state() {
>      if [ "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' yes', 'no', d)}" = "no" ]; then
> @@ -74,10 +74,12 @@ build_syslinux_cfg () {
>
>  # Enable runqemu. eg: runqemu xen-image-minimal nographic slirp
>  WKS_FILE_x86-64 = "directdisk-xen.wks"
> -QB_MEM ?= "-m 400"
> +QB_MEM ?= "-m 512"

Since you are bumping the default to 512 here. Do we even need that
qemuarm64 specific override in the bbclass ?

Bruce

>  QB_DEFAULT_KERNEL ?= "none"
>  QB_DEFAULT_FSTYPE ?= "wic"
> +QB_DEFAULT_FSTYPE_qemux86-64 = "wic"
>  QB_FSINFO ?= "wic:kernel-in-fs"
> +QB_SERIAL_OPT = "-serial mon:stdio"
>  # qemux86-64 machine does not include 'wic' in IMAGE_FSTYPES, which is needed
>  # to boot this image, so add it here:
>  IMAGE_FSTYPES_qemux86-64 += "wic"
> --
> 2.25.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 0/2] Xen: qemuboot device tree support (Arm 64-bit)

[Bruce Ashfield]

On Wed, Jul 14, 2021 at 3:34 PM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
> <christopher.w.clark@gmail.com> wrote:
> >
> > This series introduces tools and class structure in the layer for
> > enabling testing Xen system images with Qemu, booting with a device tree
> > that describes the target system.
> >
> > A device tree is required for launching Xen systems on Arm platforms and
> > is part of the Hyperlaunch design for Xen.
> >   https://wiki.xenproject.org/wiki/Hyperlaunch
> >
> > This series adds a recipe for Lopper, a device tree tool, and its
> > dependencies. It provides a class that implements a bitbake task to
> > generate a device tree for qemuboot, and a set of defaults to enable
> > 'runqemu xen-image-minimal nographic slirp' to boot images compiled for
> > Arm 64-bit machines.
>
> This looks great, and something that I've been looking for as well.
>
> I just need to carve out a bit more time to take it for a spin, but I
> don't see any issues with the changes. (I've reviewed the new recipes,
> etc, and nothing major jumped out).

I finally got a chance to look at things in more detail .. and it
looks like a nice step forward.

I had some minor comments. If you can't get to them, or they aren't
valid .. let me know, and I'd have no issue merging this as-is and we
can fixup later as required.

Bruce

>
> Bruce
>
> >
> > Christopher Clark (2):
> >   lopper, python-dtc: add new recipes for device tree tools
> >   qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images
> >
> >  classes/qemuboot-xen-defaults.bbclass        |  29 +++
> >  classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
> >  conf/distro/include/meta-virt-xen.inc        |   3 +
> >  recipes-extended/images/xen-image-minimal.bb |   6 +-
> >  recipes-kernel/dtc/python3-dtc_1.6.1.bb      |  26 +++
> >  recipes-kernel/lopper/lopper.bb              |  45 ++++
> >  6 files changed, 316 insertions(+), 2 deletions(-)
> >  create mode 100644 classes/qemuboot-xen-defaults.bbclass
> >  create mode 100644 classes/qemuboot-xen-dtb.bbclass
> >  create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
> >  create mode 100644 recipes-kernel/lopper/lopper.bb
> >
> > --
> > 2.25.1
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 1/2] lopper, python-dtc: add new recipes for device tree tools

[Christopher Clark]

On Tue, Jul 20, 2021 at 7:41 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
> <christopher.w.clark@gmail.com> wrote:
> >
> > From: Christopher Clark <christopher.clark@starlab.io>
> >
> > This device tree tooling is being added as a prequisite for enabling
> > qemuboot to run Xen system images with device trees generated for Qemu.
> >
> > lopper: a tool for performing operations on device tree files.
> > A new recipe inspired by the original from meta-xilinx-bsp.
> >
> > python-dtc: a python library for the Device Tree compiler.
> > This is a prerequisite for lopper. Imported from meta-xilinx-bsp with an
> > updated SRCREV and DESCRIPTION for the latest release, version 1.6.1.
> >
> > Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> > ---
> >  recipes-kernel/dtc/python3-dtc_1.6.1.bb | 26 ++++++++++++++
> >  recipes-kernel/lopper/lopper.bb         | 45 +++++++++++++++++++++++++
> >  2 files changed, 71 insertions(+)
> >  create mode 100644 recipes-kernel/dtc/python3-dtc_1.6.1.bb
> >  create mode 100644 recipes-kernel/lopper/lopper.bb
> >
> > diff --git a/recipes-kernel/dtc/python3-dtc_1.6.1.bb b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> > new file mode 100644
> > index 0000000..5374029
> > --- /dev/null
> > +++ b/recipes-kernel/dtc/python3-dtc_1.6.1.bb
> > @@ -0,0 +1,26 @@
> > +SUMMARY = "Python Library for the Device Tree Compiler"
> > +HOMEPAGE = "https://devicetree.org/"
> > +DESCRIPTION = "A python library for the Device Tree Compiler, a tool used to manipulate Device Tree files which contain a data structure for describing hardware."
> > +SECTION = "bootloader"
> > +LICENSE = "GPLv2 | BSD"
> > +
> > +DEPENDS = "flex-native bison-native swig-native libyaml dtc"
> > +
> > +SRC_URI = "git://git.kernel.org/pub/scm/utils/dtc/dtc.git"
> > +
> > +UPSTREAM_CHECK_GITTAGREGEX = "v(?P<pver>\d+(\.\d+)+)"
> > +
> > +LIC_FILES_CHKSUM = "file://libfdt.i;beginline=1;endline=6;md5=afda088c974174a29108c8d80b5dce90"
> > +
> > +SRCREV = "ecaeb97fec013973360e94888a7de645f084345c"
> > +
> > +S = "${WORKDIR}/git/pylibfdt"
> > +
> > +inherit distutils3
> > +
> > +do_configure_prepend() {
>
> Sorry for being so slow on the review of this, I was deep into getting
> the umoci backend to work for OCI image creation and am just getting
> to reviews now!
>
> > +    make -C "${S}/.." version_gen.h
>
> Two minor thoughts on the line above:
>
>  - did you try oe_runmake ? versus the bare 'make' call ? I always
> forget to do that myself.

oops, yep, I forgot - I've checked with that done and it works fine,
so will update in v2

>  - the ${S}/.. concerned me at first, until I took a close look at
> ${S}. Could we use "${WORKDIR}/git/" instead ? That doesn't make it
> look like we are scribbling outside of ${S}.

Will fix. Thanks for the review.

Christopher

>
> > +    mv "${S}/../version_gen.h" "${S}/"
> > +}
> > +
> > +BBCLASSEXTEND = "native nativesdk"
> > diff --git a/recipes-kernel/lopper/lopper.bb b/recipes-kernel/lopper/lopper.bb
> > new file mode 100644
> > index 0000000..274d5b9
> > --- /dev/null
> > +++ b/recipes-kernel/lopper/lopper.bb
> > @@ -0,0 +1,45 @@
> > +SUMMARY = "Device Tree Lopper"
> > +DESCRIPTION = "Tool for manipulation of system device tree files"
> > +LICENSE = "BSD-3-Clause"
> > +SECTION = "bootloader"
> > +
> > +SRC_URI = "git://github.com/devicetree-org/lopper.git"
> > +SRCREV = "0c23e51d585608ab04009ccf3499f168e2f4d478"
> > +S = "${WORKDIR}/git"
> > +
> > +LIC_FILES_CHKSUM = "file://LICENSE.md;md5=8e5f5f691f01c9fdfa7a7f2d535be619"
> > +
> > +RDEPENDS_${PN} = " \
> > +    python3-core \
> > +    python3-dtc \
> > +    python3-humanfriendly \
> > +    "
> > +
> > +do_install() {
> > +    install -d "${D}/${bindir}"
> > +    install -d "${D}/${datadir}/${BPN}"
> > +
> > +    install -m 0644 "${S}/README" "${D}/${datadir}/${BPN}"
> > +    install -m 0644 "${S}/README-architecture.txt" "${D}/${datadir}/${BPN}"
> > +    install -m 0644 "${S}/README.pydoc" "${D}/${datadir}/${BPN}"
> > +    install -m 0644 "${S}/LICENSE.md" "${D}/${datadir}/${BPN}"
> > +
> > +    install -d "${D}/${datadir}/${BPN}/assists"
> > +    install -m 0644 "${S}/assists/"* "${D}/${datadir}/${BPN}/assists/"
> > +
> > +    install -d "${D}/${datadir}/${BPN}/lops"
> > +    install -m 0644 "${S}/lops/"* "${D}/${datadir}/${BPN}/lops/"
> > +
> > +    install -d "${D}/${datadir}/${BPN}/device-trees"
> > +    install -m 0644 "${S}/device-trees/"* "${D}/${datadir}/${BPN}/device-trees/"
> > +
> > +    install -m 0755 "${S}/"lopper*.py "${D}/${datadir}/${BPN}/"
> > +
> > +    datadir_relpath=${@os.path.relpath(d.getVar('datadir'), d.getVar('bindir'))}
> > +    ln -s "${datadir_relpath}/${BPN}/lopper.py" "${D}/${bindir}/"
>
> This looks good to me. I'm working on getting lopper into a pypi
> format, but I'll take care of updating this recipe once that work is
> complete.
>
> Bruce
>
> > +}
> > +
> > +do_configure[noexec] = "1"
> > +do_compile[noexec] = "1"
> > +
> > +BBCLASSEXTEND = "native nativesdk"
> > --
> > 2.25.1
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 2/2] qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images

[Christopher Clark]

On Tue, Jul 20, 2021 at 7:49 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
> <christopher.w.clark@gmail.com> wrote:
> >
> > From: Christopher Clark <christopher.clark@starlab.io>
> >
> > The Xen hypervisor built for Arm 64-bit targets can be launched with
> > runqemu by providing a Device Tree binary and configuration for Qemu,
> > which enables interactive testing of Xen images.
> >
> > Add qemuboot-xen-dtb.bbclass to add a new bitbake task for generating
> > the dtb file by using lopper on a device tree produced by Qemu.
> >
> > Add qemuboot-xen-defaults.bbclass to supply working default parameters
> > for the qemuarm64 machine and general support for qemuboot for Xen.
> >
> > Add the new qemuboot-xen classes to the xen-image-minimal image, along
> > with variable settings needed for qemuboot to run the image for
> > qemuarm64.
> >
> > The memory setting for Qemu has been increased to 512M after testing
> > with successful boots. Default fstype for qemux86-64 was found to need a
> > stronger override in testing. Provide an override for the serial console
> > setting for the qemuarm64 machine when the Xen distro feature is
> > enabled for the console in qemuboot, following the example of similar
> > machine settings in meta-arm-bsp.
> >
> > Development aided by this script by Stewart Hildebrand of DornerWorks:
> > https://gist.github.com/stewdk/110f43e0cc1d905fc6ed4c7e10d8d35e
> >
> > Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> > ---
> >  classes/qemuboot-xen-defaults.bbclass        |  29 +++
> >  classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
> >  conf/distro/include/meta-virt-xen.inc        |   3 +
> >  recipes-extended/images/xen-image-minimal.bb |   6 +-
> >  4 files changed, 245 insertions(+), 2 deletions(-)
> >  create mode 100644 classes/qemuboot-xen-defaults.bbclass
> >  create mode 100644 classes/qemuboot-xen-dtb.bbclass
> >
> > diff --git a/classes/qemuboot-xen-defaults.bbclass b/classes/qemuboot-xen-defaults.bbclass
> > new file mode 100644
> > index 0000000..82cad49
> > --- /dev/null
> > +++ b/classes/qemuboot-xen-defaults.bbclass
> > @@ -0,0 +1,29 @@
> > +# Set defaults for booting Xen images with qemuboot
> > +
> > +# Xen and Dom0 command line options
> > +QB_XEN_CMDLINE_EXTRA ??= "dom0_mem=192M"
> > +QB_XEN_DOM0_BOOTARGS ??= \
> > +    "console=hvc0 earlyprintk clk_ignore_unused root=/dev/vda"
> > +
> > +# Launch with one initial domain, dom0, with one boot module, the kernel
> > +DOM0_KERNEL="${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}"
> > +DOM0_KERNEL_LOAD_ADDR="0x45000000"
> > +QB_XEN_DOMAIN_MODULES = "${DOM0_KERNEL}:${DOM0_KERNEL_LOAD_ADDR}:multiboot,kernel"
> > +
> > +# Qemuboot for 64-bit Arm uses the QB_DEFAULT_KERNEL method to load Xen
> > +# and the device loader option for the dom0 kernel:
> > +QB_OPT_APPEND_append_aarch64 = " \
> > +    -device loader,file=${DOM0_KERNEL},addr=${DOM0_KERNEL_LOAD_ADDR} \
> > +    "
> > +QB_DEFAULT_KERNEL_aarch64 = "xen-${MACHINE}"
> > +
> > +# 64-bit Arm: gic version 3
> > +QB_MACHINE_aarch64 = "-machine virt,gic-version=3 -machine virtualization=true"
> > +
> > +# Use overrides for stronger assignment here as default memory set by the
> > +# machines is not high enough
> > +QB_MEM_qemuarm64 = "-m 512"
>
> What about making that memory value a variable (with 512 as the
> default) ? That allows us to change it without needing to always
> modify the bbclass.
>
> Almost all of my K8S runqemu launches need 512 or 1G of memory, so
> this is definitely not uncommon!

ack - will do that for v2

>
> > +
> > +# 64-bit Arm: qemuboot with a device tree binary
> > +QB_DTB_aarch64 = "${IMAGE_NAME}.qemuboot.dtb"
> > +QB_DTB_LINK_aarch64 = "${IMAGE_LINK_NAME}.qemuboot.dtb"
> > diff --git a/classes/qemuboot-xen-dtb.bbclass b/classes/qemuboot-xen-dtb.bbclass
> > new file mode 100644
> > index 0000000..08f9b02
> > --- /dev/null
> > +++ b/classes/qemuboot-xen-dtb.bbclass
> > @@ -0,0 +1,209 @@
> > +# Enable booting Xen with qemuboot / runqemu: generate device tree
> > +#
> > +# Copyright (c) 2021 Star Lab Corp. All rights reserved.
> > +#
> > +# Author: Christopher Clark <christopher.clark@starlab.io>
> > +
> > +# Interface variables:
> > +#
> > +# QB_DTB : defined in qemuboot.bbclass.
> > +# If set, this class will generate the specified device tree file.
> > +#
> > +# QB_XEN_CMDLINE_EXTRA :
> > +# A string to be appended to the default Xen hypervisor boot command line,
> > +# for supplying Xen boot options.
> > +# The device tree that this bbclass generates will contain Xen command
> > +# line options to connect the Xen console to the Qemu serial port.
> > +#
> > +# QB_XEN_DOMAIN_MODULES :
> > +# A space-separated list of colon-separated entries:
> > +#   "<file for the module>:<load memory address>:<module compatibility string>"
> > +#
> > +# QB_XEN_DOM0_BOOTARGS :
> > +# A string for specifying Dom0 boot options for the Xen section of the device
> > +# tree.
> > +#
> > +# QB_XEN_DTB_TASK_DEPENDS :
> > +# The task dependencies for the DTB generation. A default is provided.
> > +#
> > +# See also: Other QB_ variables as defined by the qemuboot.bbclass.
> > +
> > +write_lops_xen_section() {
> > +    cat <<EOF >"$1"
> > +/dts-v1/;
> > +/ {
> > +    compatible = "system-device-tree-v1";
> > +    lops {
> > +        /* Connect the Xen console to Qemu dtuart */
> > +        lop_1 {
> > +            compatible = "system-device-tree-v1,lop,code-v1";
> > +            code = "
> > +                chosen = node.tree['/chosen']
> > +                stdout_path = str(chosen['stdout-path'].value[0])
> > +                chosen['xen,xen-bootargs'] = \
> > +                        'console=dtuart dtuart=%s' % stdout_path
> > +                return True
> > +            ";
> > +        };
> > +        lop_2 {
> > +            compatible = "system-device-tree-v1,lop,modify";
> > +            modify = "/chosen:xen,dom0-bootargs:${QB_XEN_DOM0_BOOTARGS}";
> > +        };
> > +        lop_3 {
> > +            compatible = "system-device-tree-v1,lop,modify";
> > +            modify = "/chosen:#address-cells:<1>";
> > +        };
> > +        lop_4 {
> > +            compatible = "system-device-tree-v1,lop,modify";
> > +            modify = "/chosen:#size-cells:<1>";
> > +        };
> > +     };
> > +};
> > +EOF
> > +}
> > +
> > +write_lop_add_to_xen_cmdline() {
> > +    EXTRA_XEN_BOOTARGS="$2"
> > +    cat <<EOF >"$1"
> > +/dts-v1/;
> > +/ {
> > +    compatible = "system-device-tree-v1";
> > +    lops {
> > +        lop_1 {
> > +            compatible = "system-device-tree-v1,lop,code-v1";
> > +            options = "extra_xen_bootargs:${EXTRA_XEN_BOOTARGS}";
> > +            code = "
> > +                chosen = node.tree['/chosen']
> > +                xen_bootargs = str(chosen['xen,xen-bootargs'].value)
> > +                chosen['xen,xen-bootargs'] = '%s %s' % \
> > +                     (xen_bootargs, extra_xen_bootargs)
> > +                return True
> > +            ";
> > +        };
> > +     };
> > +};
> > +EOF
> > +}
> > +
> > +write_lop_add_chosen_module() {
> > +    ADDR="$2"
> > +    SIZE="$3"
> > +    MODULE_TYPE="$4"
> > +    cat <<EOF >"$1"
> > +/dts-v1/;
> > +/ {
> > +    compatible = "system-device-tree-v1";
> > +    lops {
> > +        lop_1 {
> > +            compatible = "system-device-tree-v1,lop,add";
> > +            node_src = "module@${ADDR}";
> > +            node_dest = "/chosen/module@${ADDR}";
> > +
> > +            module@${ADDR} {
> > +                compatible = "multiboot,module", "${MODULE_TYPE}";
> > +                reg = <${ADDR} ${SIZE}>;
> > +            };
> > +        };
> > +     };
> > +};
> > +EOF
> > +}
> > +
> > +generate_xen_qemuboot_dtb() {
> > +    # First: invoke qemu to generate an initial device tree.
> > +    # Parameters supplied here inspired by inspection of:
> > +    #   runqemu "${IMAGE_BASENAME}" nographic slirp \
> > +    #            qemuparams='-dtb "" -machine dumpdtb=${B}/qemu-dumped.dtb'
> > +    ${QB_SYSTEM_NAME} \
> > +        -device qemu-xhci \
> > +        -device usb-tablet \
> > +        -device usb-kbd \
> > +        -machine virt,gic-version=3 \
> > +        -machine virtualization=true \
> > +        ${QB_CPU} \
> > +        ${QB_SMP} \
> > +        ${QB_MEM} \
> > +        -nographic \
> > +        -serial mon:stdio \
> > +        -machine "dumpdtb=${B}/qemu-dumped.dtb"
> > +
> > +    # Lopper generates temporary files in cwd, so run it within ${B}
> > +    cd "${B}"
> > +    write_lops_xen_section "${B}/lop-insert-xen-section.dts"
> > +
> > +    write_lop_add_to_xen_cmdline "${B}/lop-xen-cmdline.dts" \
> > +        "${QB_XEN_CMDLINE_EXTRA}"
> > +
> > +    if [ -z "${QB_XEN_DOMAIN_MODULES}" ]; then
> > +        bbwarn "No domain modules: please set QB_XEN_DOMAIN_MODULES"
> > +    fi
> > +
> > +    for DOMAIN_MODULE in ${QB_XEN_DOMAIN_MODULES}
> > +    do
> > +        MODULE_FILE="$(echo ${DOMAIN_MODULE} | cut -f1 -d:)"
> > +        ADDR="$(echo ${DOMAIN_MODULE} | cut -f2 -d:)"
> > +        MODULE_TYPE="$(echo ${DOMAIN_MODULE} | cut -f3 -d:)"
> > +        RESOLVED_FILE="$(readlink -f ${MODULE_FILE})"
> > +        SIZE=$(printf '0x%x\n' $(stat -c '%s' "${RESOLVED_FILE}"))
> > +        [ "x${SIZE}" != "x0x0" ] || bbfatal No module: "${MODULE_FILE}"
> > +        write_lop_add_chosen_module "${B}/lop-add-module-${ADDR}.dts" \
> > +            "${ADDR}" "${SIZE}" "${MODULE_TYPE}"
> > +        LOP_MODULE_ARGS="${LOP_MODULE_ARGS} -i ${B}/lop-add-module-${ADDR}.dts"
> > +    done
> > +
> > +    QEMUBOOT_DTB="${IMGDEPLOYDIR}/${QB_DTB}"
> > +    QEMUBOOT_DTB_LINK="${IMGDEPLOYDIR}/${QB_DTB_LINK}"
> > +
> > +    lopper.py --werror --verbose \
> > +        -i "${B}/lop-insert-xen-section.dts" \
> > +        -i "${B}/lop-xen-cmdline.dts" \
> > +        ${LOP_MODULE_ARGS} \
> > +        -f -o "${QEMUBOOT_DTB}" \
> > +            "${B}/qemu-dumped.dtb"
> > +
> > +    # To assist debugging:
> > +    dtc -I dtb -O dts -o "${B}/output.dts" "${QEMUBOOT_DTB}"
> > +
> > +    if [ "${QEMUBOOT_DTB_LINK}" != "${QEMUBOOT_DTB}" ] ; then
> > +        if [ -e "${QEMUBOOT_DTB_LINK}" ] ; then
> > +            rm "${QEMUBOOT_DTB_LINK}"
> > +        fi
> > +        ln -s "${QB_DTB}" "${QEMUBOOT_DTB_LINK}"
> > +    fi
> > +}
> > +
> > +do_write_xen_qemuboot_dtb() {
> > +    # Not all architectures qemuboot with a device tree binary, so check
> > +    # to see if one is needed. This allows this bbclass file to be used
> > +    # in the same image recipe for multiple architectures.
> > +    if [ -n "${QB_DTB}" ] ; then
> > +        generate_xen_qemuboot_dtb
> > +    fi
> > +}
> > +
> > +addtask do_write_xen_qemuboot_dtb after do_write_qemuboot_conf before do_image
> > +# Task dependency:
> > +# An expected common case is that the kernel for at least one of the initial
> > +# domains (eg. dom0) is deployed from the virtual/kernel recipe, so
> > +# add virtual/kernel:do_deploy as a task dependency here since the kernel size
> > +# needs to be known for generating the device tree.
> > +# Dependencies are only introduced if a device tree will be generated.
> > +QB_XEN_DTB_TASK_DEPENDS ?= " \
> > +    ${@[ ' \
> > +        qemu-helper-native:do_populate_sysroot \
> > +        lopper-native:do_populate_sysroot \
> > +        dtc-native:do_populate_sysroot \
> > +        virtual/kernel:do_deploy \
> > +    ', ''][d.getVar('QB_DTB') == '' or d.getVar('QB_DTB') is None]} \
> > +    "
> > +do_write_xen_qemuboot_dtb[depends] = "${QB_XEN_DTB_TASK_DEPENDS}"
> > +
> > +def qemuboot_dtb_vars(d):
> > +    build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE',
> > +                'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME',
> > +                'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE',
> > +                'STAGING_DIR_HOST', 'SERIAL_CONSOLES']
> > +    return build_vars + [k for k in d.keys() if k.startswith('QB_')]
> > +
> > +do_write_qemuboot_dtb[vardeps] += "${@' '.join(qemuboot_dtb_vars(d))}"
> > +do_write_qemuboot_dtb[vardepsexclude] += "TOPDIR"
> > diff --git a/conf/distro/include/meta-virt-xen.inc b/conf/distro/include/meta-virt-xen.inc
> > index 02a7810..2becba1 100644
> > --- a/conf/distro/include/meta-virt-xen.inc
> > +++ b/conf/distro/include/meta-virt-xen.inc
> > @@ -10,3 +10,6 @@ XEN_RPI4_64_CONFIG_PATH ??= \
> >      'dynamic-layers/raspberrypi/conf/distro/include/xen-raspberrypi4-64.inc'
> >  include ${@bb.utils.contains('MACHINE', 'raspberrypi4-64', \
> >      '${XEN_RPI4_64_CONFIG_PATH}', '', d)}
> > +
> > +# Set serial for working qemuboot console
> > +SERIAL_CONSOLES_qemuarm64 ?= "115200;ttyAMA0"
> > diff --git a/recipes-extended/images/xen-image-minimal.bb b/recipes-extended/images/xen-image-minimal.bb
> > index 6733801..2175e63 100644
> > --- a/recipes-extended/images/xen-image-minimal.bb
> > +++ b/recipes-extended/images/xen-image-minimal.bb
> > @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = "kernel-module-xen-pciback"
> >
> >  LICENSE = "MIT"
> >
> > -inherit core-image
> > +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
> >
> >  do_check_xen_state() {
> >      if [ "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' yes', 'no', d)}" = "no" ]; then
> > @@ -74,10 +74,12 @@ build_syslinux_cfg () {
> >
> >  # Enable runqemu. eg: runqemu xen-image-minimal nographic slirp
> >  WKS_FILE_x86-64 = "directdisk-xen.wks"
> > -QB_MEM ?= "-m 400"
> > +QB_MEM ?= "-m 512"
>
> Since you are bumping the default to 512 here. Do we even need that
> qemuarm64 specific override in the bbclass ?

ack, I don't think we do.
I think the following works OK: put this into the qemuboot-xen-defaults.bbclass:

+# Increase the default qemu memory allocation to allow for the hypervisor.
+# Use a weak assignment to allow for change of default and override elsewhere.
+QB_MEM_VALUE ??= "512"
+QB_MEM = "-m ${QB_MEM_VALUE}"

and then setting QB_MEM at all can be dropped from the
xen-image-minimal recipe (since the default is now fine).

If an image does need to update the memory setting, it can set QB_MEM_VALUE
which has a weak default so is easily overridden. It's also
MACHINE-agnostic, which is preferable.

Does that sound OK?

Christopher


>
> Bruce
>
> >  QB_DEFAULT_KERNEL ?= "none"
> >  QB_DEFAULT_FSTYPE ?= "wic"
> > +QB_DEFAULT_FSTYPE_qemux86-64 = "wic"
> >  QB_FSINFO ?= "wic:kernel-in-fs"
> > +QB_SERIAL_OPT = "-serial mon:stdio"
> >  # qemux86-64 machine does not include 'wic' in IMAGE_FSTYPES, which is needed
> >  # to boot this image, so add it here:
> >  IMAGE_FSTYPES_qemux86-64 += "wic"
> > --
> > 2.25.1
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization][PATCH 2/2] qemuboot, xen-image-minimal: enable runqemu for qemuarm64 Xen images

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 16206 bytes --]

On Mon, Jul 26, 2021 at 6:37 PM Christopher Clark <
christopher.w.clark@gmail.com> wrote:

> On Tue, Jul 20, 2021 at 7:49 PM Bruce Ashfield <bruce.ashfield@gmail.com>
> wrote:
> >
> > On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark
> > <christopher.w.clark@gmail.com> wrote:
> > >
> > > From: Christopher Clark <christopher.clark@starlab.io>
> > >
> > > The Xen hypervisor built for Arm 64-bit targets can be launched with
> > > runqemu by providing a Device Tree binary and configuration for Qemu,
> > > which enables interactive testing of Xen images.
> > >
> > > Add qemuboot-xen-dtb.bbclass to add a new bitbake task for generating
> > > the dtb file by using lopper on a device tree produced by Qemu.
> > >
> > > Add qemuboot-xen-defaults.bbclass to supply working default parameters
> > > for the qemuarm64 machine and general support for qemuboot for Xen.
> > >
> > > Add the new qemuboot-xen classes to the xen-image-minimal image, along
> > > with variable settings needed for qemuboot to run the image for
> > > qemuarm64.
> > >
> > > The memory setting for Qemu has been increased to 512M after testing
> > > with successful boots. Default fstype for qemux86-64 was found to need
> a
> > > stronger override in testing. Provide an override for the serial
> console
> > > setting for the qemuarm64 machine when the Xen distro feature is
> > > enabled for the console in qemuboot, following the example of similar
> > > machine settings in meta-arm-bsp.
> > >
> > > Development aided by this script by Stewart Hildebrand of DornerWorks:
> > > https://gist.github.com/stewdk/110f43e0cc1d905fc6ed4c7e10d8d35e
> > >
> > > Signed-off-by: Christopher Clark <christopher.clark@starlab.io>
> > > ---
> > >  classes/qemuboot-xen-defaults.bbclass        |  29 +++
> > >  classes/qemuboot-xen-dtb.bbclass             | 209 +++++++++++++++++++
> > >  conf/distro/include/meta-virt-xen.inc        |   3 +
> > >  recipes-extended/images/xen-image-minimal.bb |   6 +-
> > >  4 files changed, 245 insertions(+), 2 deletions(-)
> > >  create mode 100644 classes/qemuboot-xen-defaults.bbclass
> > >  create mode 100644 classes/qemuboot-xen-dtb.bbclass
> > >
> > > diff --git a/classes/qemuboot-xen-defaults.bbclass
> b/classes/qemuboot-xen-defaults.bbclass
> > > new file mode 100644
> > > index 0000000..82cad49
> > > --- /dev/null
> > > +++ b/classes/qemuboot-xen-defaults.bbclass
> > > @@ -0,0 +1,29 @@
> > > +# Set defaults for booting Xen images with qemuboot
> > > +
> > > +# Xen and Dom0 command line options
> > > +QB_XEN_CMDLINE_EXTRA ??= "dom0_mem=192M"
> > > +QB_XEN_DOM0_BOOTARGS ??= \
> > > +    "console=hvc0 earlyprintk clk_ignore_unused root=/dev/vda"
> > > +
> > > +# Launch with one initial domain, dom0, with one boot module, the
> kernel
> > > +DOM0_KERNEL="${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}"
> > > +DOM0_KERNEL_LOAD_ADDR="0x45000000"
> > > +QB_XEN_DOMAIN_MODULES =
> "${DOM0_KERNEL}:${DOM0_KERNEL_LOAD_ADDR}:multiboot,kernel"
> > > +
> > > +# Qemuboot for 64-bit Arm uses the QB_DEFAULT_KERNEL method to load
> Xen
> > > +# and the device loader option for the dom0 kernel:
> > > +QB_OPT_APPEND_append_aarch64 = " \
> > > +    -device loader,file=${DOM0_KERNEL},addr=${DOM0_KERNEL_LOAD_ADDR} \
> > > +    "
> > > +QB_DEFAULT_KERNEL_aarch64 = "xen-${MACHINE}"
> > > +
> > > +# 64-bit Arm: gic version 3
> > > +QB_MACHINE_aarch64 = "-machine virt,gic-version=3 -machine
> virtualization=true"
> > > +
> > > +# Use overrides for stronger assignment here as default memory set by
> the
> > > +# machines is not high enough
> > > +QB_MEM_qemuarm64 = "-m 512"
> >
> > What about making that memory value a variable (with 512 as the
> > default) ? That allows us to change it without needing to always
> > modify the bbclass.
> >
> > Almost all of my K8S runqemu launches need 512 or 1G of memory, so
> > this is definitely not uncommon!
>
> ack - will do that for v2
>
> >
> > > +
> > > +# 64-bit Arm: qemuboot with a device tree binary
> > > +QB_DTB_aarch64 = "${IMAGE_NAME}.qemuboot.dtb"
> > > +QB_DTB_LINK_aarch64 = "${IMAGE_LINK_NAME}.qemuboot.dtb"
> > > diff --git a/classes/qemuboot-xen-dtb.bbclass
> b/classes/qemuboot-xen-dtb.bbclass
> > > new file mode 100644
> > > index 0000000..08f9b02
> > > --- /dev/null
> > > +++ b/classes/qemuboot-xen-dtb.bbclass
> > > @@ -0,0 +1,209 @@
> > > +# Enable booting Xen with qemuboot / runqemu: generate device tree
> > > +#
> > > +# Copyright (c) 2021 Star Lab Corp. All rights reserved.
> > > +#
> > > +# Author: Christopher Clark <christopher.clark@starlab.io>
> > > +
> > > +# Interface variables:
> > > +#
> > > +# QB_DTB : defined in qemuboot.bbclass.
> > > +# If set, this class will generate the specified device tree file.
> > > +#
> > > +# QB_XEN_CMDLINE_EXTRA :
> > > +# A string to be appended to the default Xen hypervisor boot command
> line,
> > > +# for supplying Xen boot options.
> > > +# The device tree that this bbclass generates will contain Xen command
> > > +# line options to connect the Xen console to the Qemu serial port.
> > > +#
> > > +# QB_XEN_DOMAIN_MODULES :
> > > +# A space-separated list of colon-separated entries:
> > > +#   "<file for the module>:<load memory address>:<module
> compatibility string>"
> > > +#
> > > +# QB_XEN_DOM0_BOOTARGS :
> > > +# A string for specifying Dom0 boot options for the Xen section of
> the device
> > > +# tree.
> > > +#
> > > +# QB_XEN_DTB_TASK_DEPENDS :
> > > +# The task dependencies for the DTB generation. A default is provided.
> > > +#
> > > +# See also: Other QB_ variables as defined by the qemuboot.bbclass.
> > > +
> > > +write_lops_xen_section() {
> > > +    cat <<EOF >"$1"
> > > +/dts-v1/;
> > > +/ {
> > > +    compatible = "system-device-tree-v1";
> > > +    lops {
> > > +        /* Connect the Xen console to Qemu dtuart */
> > > +        lop_1 {
> > > +            compatible = "system-device-tree-v1,lop,code-v1";
> > > +            code = "
> > > +                chosen = node.tree['/chosen']
> > > +                stdout_path = str(chosen['stdout-path'].value[0])
> > > +                chosen['xen,xen-bootargs'] = \
> > > +                        'console=dtuart dtuart=%s' % stdout_path
> > > +                return True
> > > +            ";
> > > +        };
> > > +        lop_2 {
> > > +            compatible = "system-device-tree-v1,lop,modify";
> > > +            modify =
> "/chosen:xen,dom0-bootargs:${QB_XEN_DOM0_BOOTARGS}";
> > > +        };
> > > +        lop_3 {
> > > +            compatible = "system-device-tree-v1,lop,modify";
> > > +            modify = "/chosen:#address-cells:<1>";
> > > +        };
> > > +        lop_4 {
> > > +            compatible = "system-device-tree-v1,lop,modify";
> > > +            modify = "/chosen:#size-cells:<1>";
> > > +        };
> > > +     };
> > > +};
> > > +EOF
> > > +}
> > > +
> > > +write_lop_add_to_xen_cmdline() {
> > > +    EXTRA_XEN_BOOTARGS="$2"
> > > +    cat <<EOF >"$1"
> > > +/dts-v1/;
> > > +/ {
> > > +    compatible = "system-device-tree-v1";
> > > +    lops {
> > > +        lop_1 {
> > > +            compatible = "system-device-tree-v1,lop,code-v1";
> > > +            options = "extra_xen_bootargs:${EXTRA_XEN_BOOTARGS}";
> > > +            code = "
> > > +                chosen = node.tree['/chosen']
> > > +                xen_bootargs = str(chosen['xen,xen-bootargs'].value)
> > > +                chosen['xen,xen-bootargs'] = '%s %s' % \
> > > +                     (xen_bootargs, extra_xen_bootargs)
> > > +                return True
> > > +            ";
> > > +        };
> > > +     };
> > > +};
> > > +EOF
> > > +}
> > > +
> > > +write_lop_add_chosen_module() {
> > > +    ADDR="$2"
> > > +    SIZE="$3"
> > > +    MODULE_TYPE="$4"
> > > +    cat <<EOF >"$1"
> > > +/dts-v1/;
> > > +/ {
> > > +    compatible = "system-device-tree-v1";
> > > +    lops {
> > > +        lop_1 {
> > > +            compatible = "system-device-tree-v1,lop,add";
> > > +            node_src = "module@${ADDR}";
> > > +            node_dest = "/chosen/module@${ADDR}";
> > > +
> > > +            module@${ADDR} {
> > > +                compatible = "multiboot,module", "${MODULE_TYPE}";
> > > +                reg = <${ADDR} ${SIZE}>;
> > > +            };
> > > +        };
> > > +     };
> > > +};
> > > +EOF
> > > +}
> > > +
> > > +generate_xen_qemuboot_dtb() {
> > > +    # First: invoke qemu to generate an initial device tree.
> > > +    # Parameters supplied here inspired by inspection of:
> > > +    #   runqemu "${IMAGE_BASENAME}" nographic slirp \
> > > +    #            qemuparams='-dtb "" -machine
> dumpdtb=${B}/qemu-dumped.dtb'
> > > +    ${QB_SYSTEM_NAME} \
> > > +        -device qemu-xhci \
> > > +        -device usb-tablet \
> > > +        -device usb-kbd \
> > > +        -machine virt,gic-version=3 \
> > > +        -machine virtualization=true \
> > > +        ${QB_CPU} \
> > > +        ${QB_SMP} \
> > > +        ${QB_MEM} \
> > > +        -nographic \
> > > +        -serial mon:stdio \
> > > +        -machine "dumpdtb=${B}/qemu-dumped.dtb"
> > > +
> > > +    # Lopper generates temporary files in cwd, so run it within ${B}
> > > +    cd "${B}"
> > > +    write_lops_xen_section "${B}/lop-insert-xen-section.dts"
> > > +
> > > +    write_lop_add_to_xen_cmdline "${B}/lop-xen-cmdline.dts" \
> > > +        "${QB_XEN_CMDLINE_EXTRA}"
> > > +
> > > +    if [ -z "${QB_XEN_DOMAIN_MODULES}" ]; then
> > > +        bbwarn "No domain modules: please set QB_XEN_DOMAIN_MODULES"
> > > +    fi
> > > +
> > > +    for DOMAIN_MODULE in ${QB_XEN_DOMAIN_MODULES}
> > > +    do
> > > +        MODULE_FILE="$(echo ${DOMAIN_MODULE} | cut -f1 -d:)"
> > > +        ADDR="$(echo ${DOMAIN_MODULE} | cut -f2 -d:)"
> > > +        MODULE_TYPE="$(echo ${DOMAIN_MODULE} | cut -f3 -d:)"
> > > +        RESOLVED_FILE="$(readlink -f ${MODULE_FILE})"
> > > +        SIZE=$(printf '0x%x\n' $(stat -c '%s' "${RESOLVED_FILE}"))
> > > +        [ "x${SIZE}" != "x0x0" ] || bbfatal No module:
> "${MODULE_FILE}"
> > > +        write_lop_add_chosen_module "${B}/lop-add-module-${ADDR}.dts"
> \
> > > +            "${ADDR}" "${SIZE}" "${MODULE_TYPE}"
> > > +        LOP_MODULE_ARGS="${LOP_MODULE_ARGS} -i
> ${B}/lop-add-module-${ADDR}.dts"
> > > +    done
> > > +
> > > +    QEMUBOOT_DTB="${IMGDEPLOYDIR}/${QB_DTB}"
> > > +    QEMUBOOT_DTB_LINK="${IMGDEPLOYDIR}/${QB_DTB_LINK}"
> > > +
> > > +    lopper.py --werror --verbose \
> > > +        -i "${B}/lop-insert-xen-section.dts" \
> > > +        -i "${B}/lop-xen-cmdline.dts" \
> > > +        ${LOP_MODULE_ARGS} \
> > > +        -f -o "${QEMUBOOT_DTB}" \
> > > +            "${B}/qemu-dumped.dtb"
> > > +
> > > +    # To assist debugging:
> > > +    dtc -I dtb -O dts -o "${B}/output.dts" "${QEMUBOOT_DTB}"
> > > +
> > > +    if [ "${QEMUBOOT_DTB_LINK}" != "${QEMUBOOT_DTB}" ] ; then
> > > +        if [ -e "${QEMUBOOT_DTB_LINK}" ] ; then
> > > +            rm "${QEMUBOOT_DTB_LINK}"
> > > +        fi
> > > +        ln -s "${QB_DTB}" "${QEMUBOOT_DTB_LINK}"
> > > +    fi
> > > +}
> > > +
> > > +do_write_xen_qemuboot_dtb() {
> > > +    # Not all architectures qemuboot with a device tree binary, so
> check
> > > +    # to see if one is needed. This allows this bbclass file to be
> used
> > > +    # in the same image recipe for multiple architectures.
> > > +    if [ -n "${QB_DTB}" ] ; then
> > > +        generate_xen_qemuboot_dtb
> > > +    fi
> > > +}
> > > +
> > > +addtask do_write_xen_qemuboot_dtb after do_write_qemuboot_conf before
> do_image
> > > +# Task dependency:
> > > +# An expected common case is that the kernel for at least one of the
> initial
> > > +# domains (eg. dom0) is deployed from the virtual/kernel recipe, so
> > > +# add virtual/kernel:do_deploy as a task dependency here since the
> kernel size
> > > +# needs to be known for generating the device tree.
> > > +# Dependencies are only introduced if a device tree will be generated.
> > > +QB_XEN_DTB_TASK_DEPENDS ?= " \
> > > +    ${@[ ' \
> > > +        qemu-helper-native:do_populate_sysroot \
> > > +        lopper-native:do_populate_sysroot \
> > > +        dtc-native:do_populate_sysroot \
> > > +        virtual/kernel:do_deploy \
> > > +    ', ''][d.getVar('QB_DTB') == '' or d.getVar('QB_DTB') is None]} \
> > > +    "
> > > +do_write_xen_qemuboot_dtb[depends] = "${QB_XEN_DTB_TASK_DEPENDS}"
> > > +
> > > +def qemuboot_dtb_vars(d):
> > > +    build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE',
> > > +                'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME',
> > > +                'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE',
> > > +                'STAGING_DIR_HOST', 'SERIAL_CONSOLES']
> > > +    return build_vars + [k for k in d.keys() if k.startswith('QB_')]
> > > +
> > > +do_write_qemuboot_dtb[vardeps] += "${@' '.join(qemuboot_dtb_vars(d))}"
> > > +do_write_qemuboot_dtb[vardepsexclude] += "TOPDIR"
> > > diff --git a/conf/distro/include/meta-virt-xen.inc
> b/conf/distro/include/meta-virt-xen.inc
> > > index 02a7810..2becba1 100644
> > > --- a/conf/distro/include/meta-virt-xen.inc
> > > +++ b/conf/distro/include/meta-virt-xen.inc
> > > @@ -10,3 +10,6 @@ XEN_RPI4_64_CONFIG_PATH ??= \
> > >
> 'dynamic-layers/raspberrypi/conf/distro/include/xen-raspberrypi4-64.inc'
> > >  include ${@bb.utils.contains('MACHINE', 'raspberrypi4-64', \
> > >      '${XEN_RPI4_64_CONFIG_PATH}', '', d)}
> > > +
> > > +# Set serial for working qemuboot console
> > > +SERIAL_CONSOLES_qemuarm64 ?= "115200;ttyAMA0"
> > > diff --git a/recipes-extended/images/xen-image-minimal.bb
> b/recipes-extended/images/xen-image-minimal.bb
> > > index 6733801..2175e63 100644
> > > --- a/recipes-extended/images/xen-image-minimal.bb
> > > +++ b/recipes-extended/images/xen-image-minimal.bb
> > > @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 =
> "kernel-module-xen-pciback"
> > >
> > >  LICENSE = "MIT"
> > >
> > > -inherit core-image
> > > +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb
> > >
> > >  do_check_xen_state() {
> > >      if [ "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' yes',
> 'no', d)}" = "no" ]; then
> > > @@ -74,10 +74,12 @@ build_syslinux_cfg () {
> > >
> > >  # Enable runqemu. eg: runqemu xen-image-minimal nographic slirp
> > >  WKS_FILE_x86-64 = "directdisk-xen.wks"
> > > -QB_MEM ?= "-m 400"
> > > +QB_MEM ?= "-m 512"
> >
> > Since you are bumping the default to 512 here. Do we even need that
> > qemuarm64 specific override in the bbclass ?
>
> ack, I don't think we do.
> I think the following works OK: put this into the
> qemuboot-xen-defaults.bbclass:
>
> +# Increase the default qemu memory allocation to allow for the hypervisor.
> +# Use a weak assignment to allow for change of default and override
> elsewhere.
> +QB_MEM_VALUE ??= "512"
> +QB_MEM = "-m ${QB_MEM_VALUE}"
>
> and then setting QB_MEM at all can be dropped from the
> xen-image-minimal recipe (since the default is now fine).
>
> If an image does need to update the memory setting, it can set QB_MEM_VALUE
> which has a weak default so is easily overridden. It's also
> MACHINE-agnostic, which is preferable.
>
> Does that sound OK?
>

Yes, sounds good to me!

Bruce



>
> Christopher
>
>
> >
> > Bruce
> >
> > >  QB_DEFAULT_KERNEL ?= "none"
> > >  QB_DEFAULT_FSTYPE ?= "wic"
> > > +QB_DEFAULT_FSTYPE_qemux86-64 = "wic"
> > >  QB_FSINFO ?= "wic:kernel-in-fs"
> > > +QB_SERIAL_OPT = "-serial mon:stdio"
> > >  # qemux86-64 machine does not include 'wic' in IMAGE_FSTYPES, which
> is needed
> > >  # to boot this image, so add it here:
> > >  IMAGE_FSTYPES_qemux86-64 += "wic"
> > > --
> > > 2.25.1
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 22484 bytes --]