[isar-cip-core][PATCH v3 0/3] Integrate Delta Update with rdiff_image and delta handler

[isar-cip-core][PATCH v3 0/3] Integrate Delta Update with rdiff_image and delta handler

[Adithya Balakumar]

These series of patches enable the support for delta update for the root file system with swupdate's 
rdiff_image and delta handler(zchunk).

A couple notes about the implementation:
- In this patchset, Delta update is only supported for the root file system.
- The newly added delta-update.bbclass handles the creation of the delta update artifacts.
- The update handler type i.e rdiff_image or delta handler is set in the variable "DELTA_UPDATE_TYPE"
in the swupdate.yml file along with some variables required for each handler.
- The creation of delta artifact for the rdiff_image handler requires a reference image (against which 
the delta is computed). Currently the reference artifact file (.squashfs/.verity) is passed to the 
build system by placing the artifact in the path "${TOPDIR}/previous-image" and setting the name of the
artifact in the variable DELTA_RDIFF_REF_IMAGE in the swupdate.yml file.

Changes in v3:
- Rebase to current next branch (61e998e1e0541e2713c05fcbf21df39fbc882a0a)
- Set "${TOPDIR}/previous-image" as the path for the previous image artifact for rdiff update.
- README updates to verify delta update.

Changes in v2:
- Removed zchunk support for bookworm (due to use of backports). Currently zchunk based update only supported in Sid.
- Removed delta-update.yml file. Related variables have default values and can be set in manually swupdate.yml file.
- Using .squashfs file (.verity file in the case of Secure boot) for rdiff delta creation instead of .wic file.
- rdiff is set as default for delta update type, the build system looks for the v1 artifact in a default path (image-v1/<artifact_name>).
- Created a separate template file for sw-description of delta update.
- Renamed delta_update.bbclass to delta-update.bbclass.

Adithya Balakumar (3):
  Add Delta update support with rdiff_image and delta handler
  swupdate.bbclass: Generate swu for delta updates
  doc/README.swupdate.md: Update steps to test Delta software Update

 classes/delta-update.bbclass                  | 97 +++++++++++++++++++
 classes/swupdate.bbclass                      | 32 +++++-
 conf/layer.conf                               |  2 +-
 doc/README.swupdate.md                        | 59 +++++++++++
 kas/opt/swupdate.yml                          |  6 ++
 .../images/swu/sw-description-delta.tmpl      | 39 ++++++++
 6 files changed, 233 insertions(+), 2 deletions(-)
 create mode 100644 classes/delta-update.bbclass
 create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl

-- 
2.39.2

[isar-cip-core][PATCH v3 1/3] Add Delta update support with rdiff_image and delta handler

[Adithya Balakumar]

swupdate supports delta updates with rdiff_image and delta(zchunk)
handler. This change adds support to use either of the handler
for creating delta update artifacts. zchunk based updates are supported
only for sid.

DELTA_UPDATE_TYPE, DELTA_RDIFF_REF_IMAGE and DELTA_ZCK_URL variables can
be modified based on the update type in the swupdate.yml file.

Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
---
 classes/delta-update.bbclass | 97 ++++++++++++++++++++++++++++++++++++
 conf/layer.conf              |  2 +-
 kas/opt/swupdate.yml         |  6 +++
 3 files changed, 104 insertions(+), 1 deletion(-)
 create mode 100644 classes/delta-update.bbclass

diff --git a/classes/delta-update.bbclass b/classes/delta-update.bbclass
new file mode 100644
index 0000000..87bf7ff
--- /dev/null
+++ b/classes/delta-update.bbclass
@@ -0,0 +1,97 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Toshiba Corporation 2024
+#
+# Authors:
+#  Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+IMAGER_INSTALL:delta_update .= "zchunk rdiff"
+
+FILESEXTRAPATHS:append = ":${TOPDIR}/previous-image"
+do_image_delta_update[cleandirs] += "${WORKDIR}/delta_interim_artifacts"
+
+DELTA_UPDATE_TYPE ??= "rdiff"
+DELTA_RDIFF_REF_IMAGE ??= "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
+DELTA_ZCK_URL ??= ""
+DELTA_PREV_IMAGE_PATH ??= "${TOPDIR}/previous-image"
+
+def disable_delta_update_tasks (d):
+    d.appendVarFlag("do_image_delta_update", "noexec", "1")
+    d.setVar("DELTA_UPDATE_TYPE", "")
+
+def delta_update_prereq_check (d):
+    import os
+    if d.getVar("DELTA_UPDATE_TYPE") == "rdiff":
+        path = d.getVar("DELTA_PREV_IMAGE_PATH") + "/" + d.getVar("DELTA_RDIFF_REF_IMAGE")
+        if not os.path.isfile(path):
+            disable_delta_update_tasks(d)
+        else:
+            return "file://" + d.getVar("DELTA_RDIFF_REF_IMAGE")
+    elif d.getVar("DELTA_UPDATE_TYPE") == "zchunk":
+        if d.getVar("BASE_DISTRO_CODENAME") != "sid":
+            disable_delta_update_tasks(d)
+    else:
+        disable_delta_update_tasks(d)
+    return ""
+
+SRC_URI += "${@delta_update_prereq_check(d)}"
+
+create_rdiff_delta_artifact() {
+    if [ -z "${DELTA_RDIFF_REF_IMAGE}" ]; then
+        bbfatal "You must set DELTA_RDIFF_REF_IMAGE and provide the required files as artifacts to this recipe"
+    fi
+
+    rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta
+    # create signature file with rdiff
+    ${SUDO_CHROOT} /usr/bin/rdiff signature ${WORKDIR}/${DELTA_RDIFF_REF_IMAGE} \
+        ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig
+
+    # create delta file with the signature file
+    ${SUDO_CHROOT} /usr/bin/rdiff delta ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig \
+        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE} ${PP_DEPLOY}/${IMAGE_FULLNAME}.delta
+
+    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.delta
+
+    # create a symbolic link as IMAGE_CMD expects a *.delta_update file in deploy image directory
+    ln -sf ${DELTA_ARTIFACT_SWU} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
+}
+
+create_zchunk_delta_artifact() {
+    # Create .zck file
+    ${SUDO_CHROOT} /bin/zck \
+        --output ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck \
+        -u --chunk-hash-type sha256 \
+        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}
+
+    # Calculate size of zck header
+    HSIZE="$(${SUDO_CHROOT} /bin/zck_read_header -v ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck | grep "Header size" | cut -d ':' -f 2)"
+
+    # Extract the zck header
+    ${SUDO_CHROOT} /bin/dd if="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck of="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck.header bs=1 count="$HSIZE" status=none
+
+    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.zck.header
+
+    # create a symbolic link as IMAGE_CMD expects a *.delta_update file in deploy image directory
+    ln -sf ${DELTA_ARTIFACT_SWU} ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
+}
+
+do_image_delta_update[depends] += "${PN}:do_transform_template"
+IMAGE_CMD:delta_update() {
+    case "${DELTA_UPDATE_TYPE}" in
+    "rdiff")
+        create_rdiff_delta_artifact
+        ;;
+    "zchunk")
+        create_zchunk_delta_artifact
+        ;;
+    *)
+        bbfatal "You must set a valid DELTA_UPDATE_TYPE (rdiff/zchunk)"
+        ;;
+    esac
+}
+
+addtask do_image_delta_update before do_image_swu after do_image_wic
diff --git a/conf/layer.conf b/conf/layer.conf
index 6198e1b..c652a61 100644
--- a/conf/layer.conf
+++ b/conf/layer.conf
@@ -23,5 +23,5 @@ LAYERSERIES_COMPAT_cip-core = "next"
 LAYERDIR_cip-core = "${LAYERDIR}"
 LAYERDIR_cip-core[vardepvalue] = "isar-cip-core"
 
-IMAGE_CLASSES += "squashfs verity swupdate"
+IMAGE_CLASSES += "squashfs verity swupdate delta-update"
 
diff --git a/kas/opt/swupdate.yml b/kas/opt/swupdate.yml
index ef61e4e..81dbe95 100644
--- a/kas/opt/swupdate.yml
+++ b/kas/opt/swupdate.yml
@@ -30,3 +30,9 @@ local_conf_header:
     ABROOTFS_PART_UUID_B ?= "fedcba98-7654-3210-cafe-5e0710000002"
     PREFERRED_PROVIDER_swupdate-certificates-key ??= "swupdate-certificates-key-snakeoil"
     PREFERRED_PROVIDER_swupdate-certificates ??= "swupdate-certificates-snakeoil"
+    # delta update specific variables"
+    IMAGE_FSTYPES:append = " delta_update"
+    DELTA_UPDATE_TYPE = "rdiff"
+    DELTA_RDIFF_REF_IMAGE = "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
+    DELTA_ZCK_URL = ""
+
-- 
2.39.2

[isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[Adithya Balakumar]

This allows the creation of a swu file for delta update.

Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
---
 classes/swupdate.bbclass                      | 32 ++++++++++++++-
 .../images/swu/sw-description-delta.tmpl      | 39 +++++++++++++++++++
 2 files changed, 70 insertions(+), 1 deletion(-)
 create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl

diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
index 1a7cd4d..76c7aa5 100644
--- a/classes/swupdate.bbclass
+++ b/classes/swupdate.bbclass
@@ -26,6 +26,9 @@ SWU_EBG_UPDATE ?= ""
 SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA"
 SWU_BOOTLOADER ??= "ebg"
 SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}-${SWU_BOOTLOADER}"
+SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-delta"
+SWU_DELTA_UPDATE_ARTIFACT = "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}"
+SWU_DELTA_UPDATE_ARTIFACT_TYPE = "delta_update${@get_swu_compression_type(d)}"
 
 SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}"
 SWU_DESCRIPTION_FILE ?= "sw-description"
@@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms"
 SWU_BUILDCHROOT_IMAGE_FILE ?= "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}"
 
 IMAGE_TYPEDEP:swu = "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}"
+IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if d.getVar('DELTA_UPDATE_TYPE') else ''}"
 IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
 IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
 IMAGE_INSTALL += "${@'swupdate-certificates' if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
@@ -44,8 +48,10 @@ IMAGE_INSTALL += "${@'swupdate-certificates' if bb.utils.to_boolean(d.getVar('SW
 
 IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl"
 IMAGE_SRC_URI:swu += "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
+IMAGE_SRC_URI:delta_update += "file://${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
 IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl"
 IMAGE_TEMPLATE_FILES:swu += "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
+IMAGE_TEMPLATE_FILES:delta_update += "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
 IMAGE_TEMPLATE_VARS:swu = " \
     SWU_ROOTFS_PARTITION_NAME \
     TARGET_IMAGE_UUID \
@@ -58,6 +64,8 @@ IMAGE_TEMPLATE_VARS:swu = " \
     SWU_FILE_NODES \
     SWU_BOOTLOADER_FILE_NODE \
     SWU_SCRIPTS_NODE \
+    SWU_DELTA_UPDATE_ARTIFACT \
+    SWU_DELTA_UPDATE_PROPERTIES \
     "
 
 # TARGET_IMAGE_UUID needs to be generated before completing the template
@@ -148,6 +156,22 @@ python add_scripts_node() {
     d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node)
 }
 
+SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties"
+python add_swu_delta_update_properties() {
+    delta_type = d.getVar('DELTA_UPDATE_TYPE')
+    swu_delta_update_properties = ""
+    if delta_type == "rdiff":
+        swu_delta_update_properties =  'chainhandler = "rdiff_image";'
+    elif delta_type == "zchunk":
+        zck_url = d.getVar('DELTA_ZCK_URL')
+        swu_delta_update_properties = f"""
+                        chainhandler = "delta";
+                        url = "{zck_url}";
+                        zckloglevel = "error";
+        """
+    d.setVar('SWU_DELTA_UPDATE_PROPERTIES', swu_delta_update_properties)
+}
+
 # convert between swupdate compressor name and imagetype extension
 def get_swu_compression_type(d):
     swu_ct = d.getVar('SWU_COMPRESSION_TYPE')
@@ -166,14 +190,20 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_cip-core}/recipes-core/images/swu"
 
 do_image_swu[depends] += "${PN}:do_transform_template"
 do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-${SWU_BOOTLOADER}"
+do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-${SWU_BOOTLOADER} ${WORKDIR}/swu-delta"
 do_image_swu[prefuncs] = "do_extend_sw_description"
 IMAGE_CMD:swu() {
     rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu
+    if [ -z "${DELTA_UPDATE_TYPE}" ]; then
+        rm -rf ${WORKDIR}/swu-delta
+    fi
     cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}' '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}'
     if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then
         cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}'
     fi
+    if [ -n "${DELTA_UPDATE_TYPE}" ]; then
+        cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}' '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}'
+    fi
 
     for swu_file in "${WORKDIR}"/swu*; do
         swu_file_base=$(basename $swu_file)
diff --git a/recipes-core/images/swu/sw-description-delta.tmpl b/recipes-core/images/swu/sw-description-delta.tmpl
new file mode 100644
index 0000000..7f7d623
--- /dev/null
+++ b/recipes-core/images/swu/sw-description-delta.tmpl
@@ -0,0 +1,39 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens AG, 2020
+#
+# Authors:
+#  Quirin Gylstorff <quirin.gylstorff@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+software =
+{
+    version = "${SWU_VERSION}";
+    name = "${SWU_NAME}";
+    ${SWU_HW_COMPAT_NODE}
+    images: ({
+            filename = "${SWU_DELTA_UPDATE_ARTIFACT}";
+            device = "C:BOOT0:linux.efi->${ABROOTFS_PART_UUID_A},C:BOOT1:linux.efi->${ABROOTFS_PART_UUID_B}";
+            type = "roundrobin";
+            ${SWU_COMPRESSION_NODE}
+            properties: {
+                        subtype = "image";
+                        configfilecheck = "/etc/os-release@not_match@IMAGE_UUID=${TARGET_IMAGE_UUID}";
+			${SWU_DELTA_UPDATE_PROPERTIES}
+            };
+            sha256 = "${SWU_DELTA_UPDATE_ARTIFACT}-sha256";
+    });
+    files: ({
+            filename = "linux.efi";
+            path = "linux.efi";
+            type = "roundrobin";
+            device = "C:BOOT0:linux.efi->BOOT0,C:BOOT1:linux.efi->BOOT1";
+            filesystem = "vfat";
+            properties: {
+                        subtype = "kernel";
+            };
+            sha256 = "linux.efi-sha256";
+    }${SWU_FILE_NODES});
+}
-- 
2.39.2

[isar-cip-core][PATCH v3 3/3] doc/README.swupdate.md: Update steps to test Delta software Update

[Adithya Balakumar]

This change includes steps to verify Delta Software Update with
rdiff_image and delta handler

Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
---
 doc/README.swupdate.md | 59 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/doc/README.swupdate.md b/doc/README.swupdate.md
index b7e13f7..c3bd3b4 100644
--- a/doc/README.swupdate.md
+++ b/doc/README.swupdate.md
@@ -360,6 +360,65 @@ user variables:
 
 ```
 
+# Building and testing the CIP Core image for Delta Software Update
+
+Set up `kas-container` as described in the [top-level README](../README.md), and then proceed with the following steps.
+
+First build an image using the following command:
+```
+host$ ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml:kas/opt/rt.yml
+```
+The above image will be used as the base image to which the update is applied.
+
+## Delta Software Update using rdiff_image handler
+
+Creating an delta update file for rdiff_image handler requires a reference artifact (against which the delta is computed). In this case, the image built in the previous section can be used as the reference artifact. By default the `DELTA_UPDATE_TYPE` is set to `rdiff` and `DELTA_RDIFF_REF_IMAGE` is set to the name `${IMAGE_FULLNAME}.squashfs` (or ${IMAGE_FULLNAME}.verity in the case of Secure boot enabled image). The values of `DELTA_UPDATE_TYPE` and `DELTA_RDIFF_REF_IMAGE` can be changed in the `swupdate.yml` file.
+
+The build system looks for the reference artifact in a directory named `previous-image` in the build directory used for the build process.
+
+Copy the reference artifact to the mentioned directory with the following commands:
+```
+mkdir -p build-v2/previous-image
+cp build/tmp/deploy/images/qemu-amd64/cip-core-image-cip-core-bookworm-qemu-amd64.squashfs build-v2/previous-image
+```
+Build the second image with `build-v2` as the build directory with the following command:
+```
+KAS_BUILD_DIR=build-v2 ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml
+```
+Now start the first image. Run the following commands:
+```
+host$ DISTRO_RELEASE=bookworm SWUPDATE_BOOT=y ./start-qemu.sh amd64
+```
+Copy `cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu` file from `build-v2/tmp/deploy/images/qemu-amd64/` folder into the running system:
+```
+host$ cd build-v2/tmp/deploy/images/qemu-amd64/
+host$ scp -P 22222 ./cip-core-image-cip-core-bookworm-qemu-amd64-delta.swu root@localhost:
+```
+
+## Delta Software Update using delta handler
+
+Currently zchunk based delta updates are supported only in Sid images. Make sure to build the first image with Sid as the distribution (use `sid.yml` file as part of the build command).
+For Delta update with zchunk, set the variable `DELTA_ZCK_URL` with the URL of the zck file that is hosted in a http server and set the `DELTA_UPDATE_TYPE` to `zchunk` in `swupdate.yml` file.
+
+Build the second image with the modification as shown above with the following command:
+```
+KAS_BUILD_DIR=build-v2 ./kas-container build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/ebg-swu.yml:kas/opt/sid.yml
+```
+Now start the first image. Run the following commands:
+```
+host$ DISTRO_RELEASE=sid SWUPDATE_BOOT=y ./start-qemu.sh amd64
+```
+Copy `cip-core-image-cip-core-sid-qemu-amd64-delta.swu` file from `build-v2/tmp/deploy/images/qemu-amd64/` folder into the running system:
+```
+host$ cd build-v2/tmp/deploy/images/qemu-amd64/
+host$ scp -P 22222 ./cip-core-image-cip-core-sid-qemu-amd64-delta.swu root@localhost:
+```
+The `cip-core-image-cip-core-sid-qemu-amd64.zck` file must be hosted in a http server.
+
+### Delta Software Update Verification
+
+Follow the steps mentioned in the section [SWUpdate verification](#swupdate-verification) for verification.
+
 # Building and testing the CIP Core image for BBB
 
 Follow the steps mentioned in the section [Building and testing the CIP Core image](README.swupdate.md#building-and-testing-the-cip-core-image) for creating images and .swu files.
-- 
2.39.2

Re: [isar-cip-core][PATCH v3 1/3] Add Delta update support with rdiff_image and delta handler

[MOESSBAUER, Felix]

On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
> swupdate supports delta updates with rdiff_image and delta(zchunk)
> handler. This change adds support to use either of the handler
> for creating delta update artifacts. zchunk based updates are
> supported
> only for sid.
> 
> DELTA_UPDATE_TYPE, DELTA_RDIFF_REF_IMAGE and DELTA_ZCK_URL variables
> can
> be modified based on the update type in the swupdate.yml file.
> 
> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> ---
>  classes/delta-update.bbclass | 97
> ++++++++++++++++++++++++++++++++++++
>  conf/layer.conf              |  2 +-
>  kas/opt/swupdate.yml         |  6 +++
>  3 files changed, 104 insertions(+), 1 deletion(-)
>  create mode 100644 classes/delta-update.bbclass
> 
> diff --git a/classes/delta-update.bbclass b/classes/delta-
> update.bbclass
> new file mode 100644
> index 0000000..87bf7ff
> --- /dev/null
> +++ b/classes/delta-update.bbclass
> @@ -0,0 +1,97 @@
> +#
> +# CIP Core, generic profile
> +#
> +# Copyright (c) Toshiba Corporation 2024
> +#
> +# Authors:
> +#  Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +IMAGER_INSTALL:delta_update .= "zchunk rdiff"

When appending with .=, prepend a space.

> +
> +FILESEXTRAPATHS:append = ":${TOPDIR}/previous-image"

Please always prepend to the filesextrapath to make sure this dir is
searched first.

Felix

> +do_image_delta_update[cleandirs] +=
> "${WORKDIR}/delta_interim_artifacts"
> +
> +DELTA_UPDATE_TYPE ??= "rdiff"
> +DELTA_RDIFF_REF_IMAGE ??= "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
> +DELTA_ZCK_URL ??= ""
> +DELTA_PREV_IMAGE_PATH ??= "${TOPDIR}/previous-image"
> +
> +def disable_delta_update_tasks (d):
> +    d.appendVarFlag("do_image_delta_update", "noexec", "1")
> +    d.setVar("DELTA_UPDATE_TYPE", "")
> +
> +def delta_update_prereq_check (d):
> +    import os
> +    if d.getVar("DELTA_UPDATE_TYPE") == "rdiff":
> +        path = d.getVar("DELTA_PREV_IMAGE_PATH") + "/" +
> d.getVar("DELTA_RDIFF_REF_IMAGE")
> +        if not os.path.isfile(path):
> +            disable_delta_update_tasks(d)
> +        else:
> +            return "file://" + d.getVar("DELTA_RDIFF_REF_IMAGE")
> +    elif d.getVar("DELTA_UPDATE_TYPE") == "zchunk":
> +        if d.getVar("BASE_DISTRO_CODENAME") != "sid":
> +            disable_delta_update_tasks(d)
> +    else:
> +        disable_delta_update_tasks(d)
> +    return ""
> +
> +SRC_URI += "${@delta_update_prereq_check(d)}"
> +
> +create_rdiff_delta_artifact() {
> +    if [ -z "${DELTA_RDIFF_REF_IMAGE}" ]; then
> +        bbfatal "You must set DELTA_RDIFF_REF_IMAGE and provide the
> required files as artifacts to this recipe"
> +    fi
> +
> +    rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta
> +    # create signature file with rdiff
> +    ${SUDO_CHROOT} /usr/bin/rdiff signature
> ${WORKDIR}/${DELTA_RDIFF_REF_IMAGE} \
> +        ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig
> +
> +    # create delta file with the signature file
> +    ${SUDO_CHROOT} /usr/bin/rdiff delta
> ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig \
> +        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}
> ${PP_DEPLOY}/${IMAGE_FULLNAME}.delta
> +
> +    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.delta
> +
> +    # create a symbolic link as IMAGE_CMD expects a *.delta_update
> file in deploy image directory
> +    ln -sf ${DELTA_ARTIFACT_SWU}
> ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
> +}
> +
> +create_zchunk_delta_artifact() {
> +    # Create .zck file
> +    ${SUDO_CHROOT} /bin/zck \
> +        --output ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck \
> +        -u --chunk-hash-type sha256 \
> +        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}
> +
> +    # Calculate size of zck header
> +    HSIZE="$(${SUDO_CHROOT} /bin/zck_read_header -v
> ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck | grep "Header size" | cut -d ':'
> -f 2)"
> +
> +    # Extract the zck header
> +    ${SUDO_CHROOT} /bin/dd if="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck
> of="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck.header bs=1 count="$HSIZE"
> status=none
> +
> +    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.zck.header
> +
> +    # create a symbolic link as IMAGE_CMD expects a *.delta_update
> file in deploy image directory
> +    ln -sf ${DELTA_ARTIFACT_SWU}
> ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
> +}
> +
> +do_image_delta_update[depends] += "${PN}:do_transform_template"
> +IMAGE_CMD:delta_update() {
> +    case "${DELTA_UPDATE_TYPE}" in
> +    "rdiff")
> +        create_rdiff_delta_artifact
> +        ;;
> +    "zchunk")
> +        create_zchunk_delta_artifact
> +        ;;
> +    *)
> +        bbfatal "You must set a valid DELTA_UPDATE_TYPE
> (rdiff/zchunk)"
> +        ;;
> +    esac
> +}
> +
> +addtask do_image_delta_update before do_image_swu after do_image_wic
> diff --git a/conf/layer.conf b/conf/layer.conf
> index 6198e1b..c652a61 100644
> --- a/conf/layer.conf
> +++ b/conf/layer.conf
> @@ -23,5 +23,5 @@ LAYERSERIES_COMPAT_cip-core = "next"
>  LAYERDIR_cip-core = "${LAYERDIR}"
>  LAYERDIR_cip-core[vardepvalue] = "isar-cip-core"
>  
> -IMAGE_CLASSES += "squashfs verity swupdate"
> +IMAGE_CLASSES += "squashfs verity swupdate delta-update"
>  
> diff --git a/kas/opt/swupdate.yml b/kas/opt/swupdate.yml
> index ef61e4e..81dbe95 100644
> --- a/kas/opt/swupdate.yml
> +++ b/kas/opt/swupdate.yml
> @@ -30,3 +30,9 @@ local_conf_header:
>      ABROOTFS_PART_UUID_B ?= "fedcba98-7654-3210-cafe-5e0710000002"
>      PREFERRED_PROVIDER_swupdate-certificates-key ??= "swupdate-
> certificates-key-snakeoil"
>      PREFERRED_PROVIDER_swupdate-certificates ??= "swupdate-
> certificates-snakeoil"
> +    # delta update specific variables"
> +    IMAGE_FSTYPES:append = " delta_update"
> +    DELTA_UPDATE_TYPE = "rdiff"
> +    DELTA_RDIFF_REF_IMAGE = "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
> +    DELTA_ZCK_URL = ""
> +

-- 
Siemens AG, Technology
Linux Expert Center

Re: [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[MOESSBAUER, Felix]

On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
> This allows the creation of a swu file for delta update.
> 
> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> ---
>  classes/swupdate.bbclass                      | 32 ++++++++++++++-
>  .../images/swu/sw-description-delta.tmpl      | 39
> +++++++++++++++++++
>  2 files changed, 70 insertions(+), 1 deletion(-)
>  create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl
> 
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 1a7cd4d..76c7aa5 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -26,6 +26,9 @@ SWU_EBG_UPDATE ?= ""
>  SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA"
>  SWU_BOOTLOADER ??= "ebg"
>  SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}-
> ${SWU_BOOTLOADER}"
> +SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-
> delta"

Hi, I'm still wondering if we really need a dedicated sw-description
file for that. Can't we just mix in the SWU_DELTA_UPDATE_PROPERTIES
into the normal sw-description? Maybe Quirin can help with the
integration using the new infrastructure.

Felix

> +SWU_DELTA_UPDATE_ARTIFACT =
> "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}"
> +SWU_DELTA_UPDATE_ARTIFACT_TYPE =
> "delta_update${@get_swu_compression_type(d)}"
>  
>  SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}"
>  SWU_DESCRIPTION_FILE ?= "sw-description"
> @@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms"
>  SWU_BUILDCHROOT_IMAGE_FILE ?=
> "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}"
>  
>  IMAGE_TYPEDEP:swu =
> "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}"
> +IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if
> d.getVar('DELTA_UPDATE_TYPE') else ''}"
>  IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key'
> if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGE_INSTALL += "${@'swupdate-certificates' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
> @@ -44,8 +48,10 @@ IMAGE_INSTALL += "${@'swupdate-certificates' if
> bb.utils.to_boolean(d.getVar('SW
>  
>  IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_SRC_URI:swu +=
> "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_SRC_URI:delta_update +=
> "file://${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu +=
> "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_TEMPLATE_FILES:delta_update +=
> "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_ROOTFS_PARTITION_NAME \
>      TARGET_IMAGE_UUID \
> @@ -58,6 +64,8 @@ IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_FILE_NODES \
>      SWU_BOOTLOADER_FILE_NODE \
>      SWU_SCRIPTS_NODE \
> +    SWU_DELTA_UPDATE_ARTIFACT \
> +    SWU_DELTA_UPDATE_PROPERTIES \
>      "
>  
>  # TARGET_IMAGE_UUID needs to be generated before completing the
> template
> @@ -148,6 +156,22 @@ python add_scripts_node() {
>      d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node)
>  }
>  
> +SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties"
> +python add_swu_delta_update_properties() {
> +    delta_type = d.getVar('DELTA_UPDATE_TYPE')
> +    swu_delta_update_properties = ""
> +    if delta_type == "rdiff":
> +        swu_delta_update_properties =  'chainhandler =
> "rdiff_image";'
> +    elif delta_type == "zchunk":
> +        zck_url = d.getVar('DELTA_ZCK_URL')
> +        swu_delta_update_properties = f"""
> +                        chainhandler = "delta";
> +                        url = "{zck_url}";
> +                        zckloglevel = "error";
> +        """
> +    d.setVar('SWU_DELTA_UPDATE_PROPERTIES',
> swu_delta_update_properties)
> +}
> +
>  # convert between swupdate compressor name and imagetype extension
>  def get_swu_compression_type(d):
>      swu_ct = d.getVar('SWU_COMPRESSION_TYPE')
> @@ -166,14 +190,20 @@ FILESEXTRAPATHS:append = ":${LAYERDIR_cip-
> core}/recipes-core/images/swu"
>  
>  do_image_swu[depends] += "${PN}:do_transform_template"
>  do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> -do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
> ${SWU_BOOTLOADER}"
> +do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
> ${SWU_BOOTLOADER} ${WORKDIR}/swu-delta"
>  do_image_swu[prefuncs] = "do_extend_sw_description"
>  IMAGE_CMD:swu() {
>      rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu
> +    if [ -z "${DELTA_UPDATE_TYPE}" ]; then
> +        rm -rf ${WORKDIR}/swu-delta
> +    fi
>      cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}'
> '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}'
>      if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then
>          cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}'
> '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}'
>      fi
> +    if [ -n "${DELTA_UPDATE_TYPE}" ]; then
> +        cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}'
> '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}'
> +    fi
>  
>      for swu_file in "${WORKDIR}"/swu*; do
>          swu_file_base=$(basename $swu_file)
> diff --git a/recipes-core/images/swu/sw-description-delta.tmpl
> b/recipes-core/images/swu/sw-description-delta.tmpl
> new file mode 100644
> index 0000000..7f7d623
> --- /dev/null
> +++ b/recipes-core/images/swu/sw-description-delta.tmpl
> @@ -0,0 +1,39 @@
> +#
> +# CIP Core, generic profile
> +#
> +# Copyright (c) Siemens AG, 2020
> +#
> +# Authors:
> +#  Quirin Gylstorff <quirin.gylstorff@siemens.com>
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +software =
> +{
> +    version = "${SWU_VERSION}";
> +    name = "${SWU_NAME}";
> +    ${SWU_HW_COMPAT_NODE}
> +    images: ({
> +            filename = "${SWU_DELTA_UPDATE_ARTIFACT}";
> +            device = "C:BOOT0:linux.efi-
> >${ABROOTFS_PART_UUID_A},C:BOOT1:linux.efi->${ABROOTFS_PART_UUID_B}";
> +            type = "roundrobin";
> +            ${SWU_COMPRESSION_NODE}
> +            properties: {
> +                        subtype = "image";
> +                        configfilecheck =
> "/etc/os-release@not_match@IMAGE_UUID=${TARGET_IMAGE_UUID}";
> +                       ${SWU_DELTA_UPDATE_PROPERTIES}
> +            };
> +            sha256 = "${SWU_DELTA_UPDATE_ARTIFACT}-sha256";
> +    });
> +    files: ({
> +            filename = "linux.efi";
> +            path = "linux.efi";
> +            type = "roundrobin";
> +            device = "C:BOOT0:linux.efi->BOOT0,C:BOOT1:linux.efi-
> >BOOT1";
> +            filesystem = "vfat";
> +            properties: {
> +                        subtype = "kernel";
> +            };
> +            sha256 = "linux.efi-sha256";
> +    }${SWU_FILE_NODES});
> +}

-- 
Siemens AG, Technology
Linux Expert Center

RE: [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[Adithya.Balakumar]

-----Original Message-----
From: MOESSBAUER, Felix <felix.moessbauer@siemens.com> 
Sent: Tuesday, April 16, 2024 9:46 AM
To: cip-dev@lists.cip-project.org; balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>; Kiszka, Jan <jan.kiszka@siemens.com>
Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
> This allows the creation of a swu file for delta update.
> 
> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> ---
>  classes/swupdate.bbclass                      | 32 ++++++++++++++-
>  .../images/swu/sw-description-delta.tmpl      | 39
> +++++++++++++++++++
>  2 files changed, 70 insertions(+), 1 deletion(-)
>  create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl
> 
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass index 
> 1a7cd4d..76c7aa5 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -26,6 +26,9 @@ SWU_EBG_UPDATE ?= ""
>  SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA"
>  SWU_BOOTLOADER ??= "ebg"
>  SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}- 
> ${SWU_BOOTLOADER}"
> +SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-
> delta"

Hi, I'm still wondering if we really need a dedicated sw-description file for that. Can't we just mix in the SWU_DELTA_UPDATE_PROPERTIES into the normal sw-description? Maybe Quirin can help with the integration using the new infrastructure.

Felix

[Adithya Balakumar] 
Hi Felix,
In a case where I have to generate a delta swu and a complete swu, I would need a sw-description file for each. Hence the approach that I have taken here. Please correct me if my understanding is wrong.

Adithya

> +SWU_DELTA_UPDATE_ARTIFACT =
> "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}"
> +SWU_DELTA_UPDATE_ARTIFACT_TYPE =
> "delta_update${@get_swu_compression_type(d)}"
>  
>  SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}"
>  SWU_DESCRIPTION_FILE ?= "sw-description"
> @@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms"
>  SWU_BUILDCHROOT_IMAGE_FILE ?=
> "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}"
>  
>  IMAGE_TYPEDEP:swu =
> "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}"
> +IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if
> d.getVar('DELTA_UPDATE_TYPE') else ''}"
>  IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key'
> if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGE_INSTALL += "${@'swupdate-certificates' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
> @@ -44,8 +48,10 @@ IMAGE_INSTALL += "${@'swupdate-certificates' if 
> bb.utils.to_boolean(d.getVar('SW
>  
>  IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_SRC_URI:swu +=
> "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_SRC_URI:delta_update +=
> "file://${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu +=
> "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_TEMPLATE_FILES:delta_update +=
> "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_ROOTFS_PARTITION_NAME \
>      TARGET_IMAGE_UUID \
> @@ -58,6 +64,8 @@ IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_FILE_NODES \
>      SWU_BOOTLOADER_FILE_NODE \
>      SWU_SCRIPTS_NODE \
> +    SWU_DELTA_UPDATE_ARTIFACT \
> +    SWU_DELTA_UPDATE_PROPERTIES \
>      "
>  
>  # TARGET_IMAGE_UUID needs to be generated before completing the 
> template @@ -148,6 +156,22 @@ python add_scripts_node() {
>      d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node)
>  }
>  
> +SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties"
> +python add_swu_delta_update_properties() {
> +    delta_type = d.getVar('DELTA_UPDATE_TYPE')
> +    swu_delta_update_properties = ""
> +    if delta_type == "rdiff":
> +        swu_delta_update_properties =  'chainhandler =
> "rdiff_image";'
> +    elif delta_type == "zchunk":
> +        zck_url = d.getVar('DELTA_ZCK_URL')
> +        swu_delta_update_properties = f"""
> +                        chainhandler = "delta";
> +                        url = "{zck_url}";
> +                        zckloglevel = "error";
> +        """
> +    d.setVar('SWU_DELTA_UPDATE_PROPERTIES',
> swu_delta_update_properties)
> +}
> +
>  # convert between swupdate compressor name and imagetype extension
>  def get_swu_compression_type(d):
>      swu_ct = d.getVar('SWU_COMPRESSION_TYPE') @@ -166,14 +190,20 @@ 
> FILESEXTRAPATHS:append = ":${LAYERDIR_cip- 
> core}/recipes-core/images/swu"
>  
>  do_image_swu[depends] += "${PN}:do_transform_template"
>  do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> -do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu- 
> ${SWU_BOOTLOADER}"
> +do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
> ${SWU_BOOTLOADER} ${WORKDIR}/swu-delta"
>  do_image_swu[prefuncs] = "do_extend_sw_description"
>  IMAGE_CMD:swu() {
>      rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu
> +    if [ -z "${DELTA_UPDATE_TYPE}" ]; then
> +        rm -rf ${WORKDIR}/swu-delta
> +    fi
>      cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}'
> '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}'
>      if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then
>          cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}'
> '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}'
>      fi
> +    if [ -n "${DELTA_UPDATE_TYPE}" ]; then
> +        cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}'
> '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}'
> +    fi
>  
>      for swu_file in "${WORKDIR}"/swu*; do
>          swu_file_base=$(basename $swu_file) diff --git 
> a/recipes-core/images/swu/sw-description-delta.tmpl
> b/recipes-core/images/swu/sw-description-delta.tmpl
> new file mode 100644
> index 0000000..7f7d623
> --- /dev/null
> +++ b/recipes-core/images/swu/sw-description-delta.tmpl
> @@ -0,0 +1,39 @@
> +#
> +# CIP Core, generic profile
> +#
> +# Copyright (c) Siemens AG, 2020
> +#
> +# Authors:
> +#  Quirin Gylstorff <quirin.gylstorff@siemens.com> # # 
> +SPDX-License-Identifier: MIT # software = {
> +    version = "${SWU_VERSION}";
> +    name = "${SWU_NAME}";
> +    ${SWU_HW_COMPAT_NODE}
> +    images: ({
> +            filename = "${SWU_DELTA_UPDATE_ARTIFACT}";
> +            device = "C:BOOT0:linux.efi-
> >${ABROOTFS_PART_UUID_A},C:BOOT1:linux.efi->${ABROOTFS_PART_UUID_B}";
> +            type = "roundrobin";
> +            ${SWU_COMPRESSION_NODE}
> +            properties: {
> +                        subtype = "image";
> +                        configfilecheck =
> "/etc/os-release@not_match@IMAGE_UUID=${TARGET_IMAGE_UUID}";
> +                       ${SWU_DELTA_UPDATE_PROPERTIES}
> +            };
> +            sha256 = "${SWU_DELTA_UPDATE_ARTIFACT}-sha256";
> +    });
> +    files: ({
> +            filename = "linux.efi";
> +            path = "linux.efi";
> +            type = "roundrobin";
> +            device = "C:BOOT0:linux.efi->BOOT0,C:BOOT1:linux.efi-
> >BOOT1";
> +            filesystem = "vfat";
> +            properties: {
> +                        subtype = "kernel";
> +            };
> +            sha256 = "linux.efi-sha256";
> +    }${SWU_FILE_NODES});
> +}

--
Siemens AG, Technology
Linux Expert Center

RE: [isar-cip-core][PATCH v3 1/3] Add Delta update support with rdiff_image and delta handler

[Adithya.Balakumar]

-----Original Message-----
From: MOESSBAUER, Felix <felix.moessbauer@siemens.com> 
Sent: Tuesday, April 16, 2024 9:42 AM
To: cip-dev@lists.cip-project.org; balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>; Kiszka, Jan <jan.kiszka@siemens.com>
Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [isar-cip-core][PATCH v3 1/3] Add Delta update support with rdiff_image and delta handler

On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
> swupdate supports delta updates with rdiff_image and delta(zchunk) 
> handler. This change adds support to use either of the handler for 
> creating delta update artifacts. zchunk based updates are supported 
> only for sid.
> 
> DELTA_UPDATE_TYPE, DELTA_RDIFF_REF_IMAGE and DELTA_ZCK_URL variables 
> can be modified based on the update type in the swupdate.yml file.
> 
> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> ---
>  classes/delta-update.bbclass | 97
> ++++++++++++++++++++++++++++++++++++
>  conf/layer.conf              |  2 +-
>  kas/opt/swupdate.yml         |  6 +++
>  3 files changed, 104 insertions(+), 1 deletion(-)
>  create mode 100644 classes/delta-update.bbclass
> 
> diff --git a/classes/delta-update.bbclass b/classes/delta- 
> update.bbclass new file mode 100644 index 0000000..87bf7ff
> --- /dev/null
> +++ b/classes/delta-update.bbclass
> @@ -0,0 +1,97 @@
> +#
> +# CIP Core, generic profile
> +#
> +# Copyright (c) Toshiba Corporation 2024 # # Authors:
> +#  Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +IMAGER_INSTALL:delta_update .= "zchunk rdiff"

When appending with .=, prepend a space.

> +
> +FILESEXTRAPATHS:append = ":${TOPDIR}/previous-image"

Please always prepend to the filesextrapath to make sure this dir is searched first.

Felix

[Adithya Balakumar]
Hi Felix,

Thanks for the feedback. I will fix both your comments in v4.

Adithya
 
> +do_image_delta_update[cleandirs] +=
> "${WORKDIR}/delta_interim_artifacts"
> +
> +DELTA_UPDATE_TYPE ??= "rdiff"
> +DELTA_RDIFF_REF_IMAGE ??= "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
> +DELTA_ZCK_URL ??= ""
> +DELTA_PREV_IMAGE_PATH ??= "${TOPDIR}/previous-image"
> +
> +def disable_delta_update_tasks (d):
> +    d.appendVarFlag("do_image_delta_update", "noexec", "1")
> +    d.setVar("DELTA_UPDATE_TYPE", "")
> +
> +def delta_update_prereq_check (d):
> +    import os
> +    if d.getVar("DELTA_UPDATE_TYPE") == "rdiff":
> +        path = d.getVar("DELTA_PREV_IMAGE_PATH") + "/" +
> d.getVar("DELTA_RDIFF_REF_IMAGE")
> +        if not os.path.isfile(path):
> +            disable_delta_update_tasks(d)
> +        else:
> +            return "file://" + d.getVar("DELTA_RDIFF_REF_IMAGE")
> +    elif d.getVar("DELTA_UPDATE_TYPE") == "zchunk":
> +        if d.getVar("BASE_DISTRO_CODENAME") != "sid":
> +            disable_delta_update_tasks(d)
> +    else:
> +        disable_delta_update_tasks(d)
> +    return ""
> +
> +SRC_URI += "${@delta_update_prereq_check(d)}"
> +
> +create_rdiff_delta_artifact() {
> +    if [ -z "${DELTA_RDIFF_REF_IMAGE}" ]; then
> +        bbfatal "You must set DELTA_RDIFF_REF_IMAGE and provide the
> required files as artifacts to this recipe"
> +    fi
> +
> +    rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta
> +    # create signature file with rdiff
> +    ${SUDO_CHROOT} /usr/bin/rdiff signature
> ${WORKDIR}/${DELTA_RDIFF_REF_IMAGE} \
> +        ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig
> +
> +    # create delta file with the signature file
> +    ${SUDO_CHROOT} /usr/bin/rdiff delta
> ${WORKDIR}/delta_interim_artifacts/old-image-rootfs.sig \
> +        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}
> ${PP_DEPLOY}/${IMAGE_FULLNAME}.delta
> +
> +    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.delta
> +
> +    # create a symbolic link as IMAGE_CMD expects a *.delta_update
> file in deploy image directory
> +    ln -sf ${DELTA_ARTIFACT_SWU}
> ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
> +}
> +
> +create_zchunk_delta_artifact() {
> +    # Create .zck file
> +    ${SUDO_CHROOT} /bin/zck \
> +        --output ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck \
> +        -u --chunk-hash-type sha256 \
> +        ${PP_DEPLOY}/${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}
> +
> +    # Calculate size of zck header
> +    HSIZE="$(${SUDO_CHROOT} /bin/zck_read_header -v
> ${PP_DEPLOY}/${IMAGE_FULLNAME}.zck | grep "Header size" | cut -d ':'
> -f 2)"
> +
> +    # Extract the zck header
> +    ${SUDO_CHROOT} /bin/dd if="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck
> of="${PP_DEPLOY}/${IMAGE_FULLNAME}".zck.header bs=1 count="$HSIZE"
> status=none
> +
> +    DELTA_ARTIFACT_SWU=${IMAGE_FULLNAME}.zck.header
> +
> +    # create a symbolic link as IMAGE_CMD expects a *.delta_update
> file in deploy image directory
> +    ln -sf ${DELTA_ARTIFACT_SWU}
> ${DEPLOY_DIR_IMAGE}/${IMAGE_FULLNAME}.delta_update
> +}
> +
> +do_image_delta_update[depends] += "${PN}:do_transform_template"
> +IMAGE_CMD:delta_update() {
> +    case "${DELTA_UPDATE_TYPE}" in
> +    "rdiff")
> +        create_rdiff_delta_artifact
> +        ;;
> +    "zchunk")
> +        create_zchunk_delta_artifact
> +        ;;
> +    *)
> +        bbfatal "You must set a valid DELTA_UPDATE_TYPE
> (rdiff/zchunk)"
> +        ;;
> +    esac
> +}
> +
> +addtask do_image_delta_update before do_image_swu after do_image_wic
> diff --git a/conf/layer.conf b/conf/layer.conf index 6198e1b..c652a61 
> 100644
> --- a/conf/layer.conf
> +++ b/conf/layer.conf
> @@ -23,5 +23,5 @@ LAYERSERIES_COMPAT_cip-core = "next"
>  LAYERDIR_cip-core = "${LAYERDIR}"
>  LAYERDIR_cip-core[vardepvalue] = "isar-cip-core"
>  
> -IMAGE_CLASSES += "squashfs verity swupdate"
> +IMAGE_CLASSES += "squashfs verity swupdate delta-update"
>  
> diff --git a/kas/opt/swupdate.yml b/kas/opt/swupdate.yml index 
> ef61e4e..81dbe95 100644
> --- a/kas/opt/swupdate.yml
> +++ b/kas/opt/swupdate.yml
> @@ -30,3 +30,9 @@ local_conf_header:
>      ABROOTFS_PART_UUID_B ?= "fedcba98-7654-3210-cafe-5e0710000002"
>      PREFERRED_PROVIDER_swupdate-certificates-key ??= "swupdate- 
> certificates-key-snakeoil"
>      PREFERRED_PROVIDER_swupdate-certificates ??= "swupdate- 
> certificates-snakeoil"
> +    # delta update specific variables"
> +    IMAGE_FSTYPES:append = " delta_update"
> +    DELTA_UPDATE_TYPE = "rdiff"
> +    DELTA_RDIFF_REF_IMAGE = "${IMAGE_FULLNAME}.${SWU_ROOTFS_TYPE}"
> +    DELTA_ZCK_URL = ""
> +

--
Siemens AG, Technology
Linux Expert Center

RE: [cip-dev] [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[Adithya.Balakumar]

-----Original Message-----
From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On Behalf Of Adithya Balakumar
Sent: Tuesday, April 16, 2024 10:00 AM
To: felix.moessbauer@siemens.com; cip-dev@lists.cip-project.org; jan.kiszka@siemens.com
Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [cip-dev] [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates



-----Original Message-----
From: MOESSBAUER, Felix <felix.moessbauer@siemens.com>
Sent: Tuesday, April 16, 2024 9:46 AM
To: cip-dev@lists.cip-project.org; balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>; Kiszka, Jan <jan.kiszka@siemens.com>
Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
Subject: Re: [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
> This allows the creation of a swu file for delta update.
> 
> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
> ---
>  classes/swupdate.bbclass                      | 32 ++++++++++++++-
>  .../images/swu/sw-description-delta.tmpl      | 39
> +++++++++++++++++++
>  2 files changed, 70 insertions(+), 1 deletion(-)
>  create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl
> 
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass index
> 1a7cd4d..76c7aa5 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -26,6 +26,9 @@ SWU_EBG_UPDATE ?= ""
>  SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA"
>  SWU_BOOTLOADER ??= "ebg"
>  SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}- 
> ${SWU_BOOTLOADER}"
> +SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-
> delta"

Hi, I'm still wondering if we really need a dedicated sw-description file for that. Can't we just mix in the SWU_DELTA_UPDATE_PROPERTIES into the normal sw-description? Maybe Quirin can help with the integration using the new infrastructure.

Felix

[Adithya Balakumar]
Hi Felix,
In a case where I have to generate a delta swu and a complete swu, I would need a sw-description file for each. Hence the approach that I have taken here. Please correct me if my understanding is wrong. 

My thought was to have delta update swu as an optional / additional artifact apart from the complete update swu. 
In a case where a reference artifact is not provided (atleast in the case of rdiff), a delta artifact will not be created. But we could still create a complete update swu anyway.

Would like to understand if you are thinking of a different approach ?

Adithya

> +SWU_DELTA_UPDATE_ARTIFACT =
> "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}"
> +SWU_DELTA_UPDATE_ARTIFACT_TYPE =
> "delta_update${@get_swu_compression_type(d)}"
>  
>  SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}"
>  SWU_DESCRIPTION_FILE ?= "sw-description"
> @@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms"
>  SWU_BUILDCHROOT_IMAGE_FILE ?=
> "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}"
>  
>  IMAGE_TYPEDEP:swu =
> "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}"
> +IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if
> d.getVar('DELTA_UPDATE_TYPE') else ''}"
>  IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key'
> if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>  IMAGE_INSTALL += "${@'swupdate-certificates' if
> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
> @@ -44,8 +48,10 @@ IMAGE_INSTALL += "${@'swupdate-certificates' if 
> bb.utils.to_boolean(d.getVar('SW
>  
>  IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_SRC_URI:swu +=
> "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_SRC_URI:delta_update +=
> "file://${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl"
>  IMAGE_TEMPLATE_FILES:swu +=
> "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
> +IMAGE_TEMPLATE_FILES:delta_update +=
> "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>  IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_ROOTFS_PARTITION_NAME \
>      TARGET_IMAGE_UUID \
> @@ -58,6 +64,8 @@ IMAGE_TEMPLATE_VARS:swu = " \
>      SWU_FILE_NODES \
>      SWU_BOOTLOADER_FILE_NODE \
>      SWU_SCRIPTS_NODE \
> +    SWU_DELTA_UPDATE_ARTIFACT \
> +    SWU_DELTA_UPDATE_PROPERTIES \
>      "
>  
>  # TARGET_IMAGE_UUID needs to be generated before completing the 
> template @@ -148,6 +156,22 @@ python add_scripts_node() {
>      d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node)
>  }
>  
> +SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties"
> +python add_swu_delta_update_properties() {
> +    delta_type = d.getVar('DELTA_UPDATE_TYPE')
> +    swu_delta_update_properties = ""
> +    if delta_type == "rdiff":
> +        swu_delta_update_properties =  'chainhandler =
> "rdiff_image";'
> +    elif delta_type == "zchunk":
> +        zck_url = d.getVar('DELTA_ZCK_URL')
> +        swu_delta_update_properties = f"""
> +                        chainhandler = "delta";
> +                        url = "{zck_url}";
> +                        zckloglevel = "error";
> +        """
> +    d.setVar('SWU_DELTA_UPDATE_PROPERTIES',
> swu_delta_update_properties)
> +}
> +
>  # convert between swupdate compressor name and imagetype extension
>  def get_swu_compression_type(d):
>      swu_ct = d.getVar('SWU_COMPRESSION_TYPE') @@ -166,14 +190,20 @@ 
> FILESEXTRAPATHS:append = ":${LAYERDIR_cip- 
> core}/recipes-core/images/swu"
>  
>  do_image_swu[depends] += "${PN}:do_transform_template"
>  do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}"
> -do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu- 
> ${SWU_BOOTLOADER}"
> +do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
> ${SWU_BOOTLOADER} ${WORKDIR}/swu-delta"
>  do_image_swu[prefuncs] = "do_extend_sw_description"
>  IMAGE_CMD:swu() {
>      rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu
> +    if [ -z "${DELTA_UPDATE_TYPE}" ]; then
> +        rm -rf ${WORKDIR}/swu-delta
> +    fi
>      cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}'
> '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}'
>      if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then
>          cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}'
> '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}'
>      fi
> +    if [ -n "${DELTA_UPDATE_TYPE}" ]; then
> +        cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}'
> '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}'
> +    fi
>  
>      for swu_file in "${WORKDIR}"/swu*; do
>          swu_file_base=$(basename $swu_file) diff --git 
> a/recipes-core/images/swu/sw-description-delta.tmpl
> b/recipes-core/images/swu/sw-description-delta.tmpl
> new file mode 100644
> index 0000000..7f7d623
> --- /dev/null
> +++ b/recipes-core/images/swu/sw-description-delta.tmpl
> @@ -0,0 +1,39 @@
> +#
> +# CIP Core, generic profile
> +#
> +# Copyright (c) Siemens AG, 2020
> +#
> +# Authors:
> +#  Quirin Gylstorff <quirin.gylstorff@siemens.com> # #
> +SPDX-License-Identifier: MIT # software = {
> +    version = "${SWU_VERSION}";
> +    name = "${SWU_NAME}";
> +    ${SWU_HW_COMPAT_NODE}
> +    images: ({
> +            filename = "${SWU_DELTA_UPDATE_ARTIFACT}";
> +            device = "C:BOOT0:linux.efi-
> >${ABROOTFS_PART_UUID_A},C:BOOT1:linux.efi->${ABROOTFS_PART_UUID_B}";
> +            type = "roundrobin";
> +            ${SWU_COMPRESSION_NODE}
> +            properties: {
> +                        subtype = "image";
> +                        configfilecheck =
> "/etc/os-release@not_match@IMAGE_UUID=${TARGET_IMAGE_UUID}";
> +                       ${SWU_DELTA_UPDATE_PROPERTIES}
> +            };
> +            sha256 = "${SWU_DELTA_UPDATE_ARTIFACT}-sha256";
> +    });
> +    files: ({
> +            filename = "linux.efi";
> +            path = "linux.efi";
> +            type = "roundrobin";
> +            device = "C:BOOT0:linux.efi->BOOT0,C:BOOT1:linux.efi-
> >BOOT1";
> +            filesystem = "vfat";
> +            properties: {
> +                        subtype = "kernel";
> +            };
> +            sha256 = "linux.efi-sha256";
> +    }${SWU_FILE_NODES});
> +}

--
Siemens AG, Technology
Linux Expert Center

Re: [cip-dev] [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[Gylstorff Quirin]

On 4/17/24 1:42 PM, Adithya.Balakumar@toshiba-tsip.com wrote:
> 
> 
> -----Original Message-----
> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On Behalf Of Adithya Balakumar
> Sent: Tuesday, April 16, 2024 10:00 AM
> To: felix.moessbauer@siemens.com; cip-dev@lists.cip-project.org; jan.kiszka@siemens.com
> Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
> Subject: Re: [cip-dev] [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates
> 
> 
> 
> -----Original Message-----
> From: MOESSBAUER, Felix <felix.moessbauer@siemens.com>
> Sent: Tuesday, April 16, 2024 9:46 AM
> To: cip-dev@lists.cip-project.org; balakumar adithya(ＴＳＩＰ TEUR) <Adithya.Balakumar@toshiba-tsip.com>; Kiszka, Jan <jan.kiszka@siemens.com>
> Cc: kunijadar shivanand(ＴＳＩＰ TMIEC ODG Porting) <Shivanand.Kunijadar@toshiba-tsip.com>; quirin.gylstorff@siemens.com; dinesh kumar(ＴＳＩＰ TMIEC ODG Porting) <dinesh.kumar@toshiba-tsip.com>; ashrith sai(ＴＳＩＰ) <Sai.Sathujoda@toshiba-tsip.com>; hayashi kazuhiro(林 和宏 ＤＭＥ ○ＤＩＧ□ＭＰＳ○ＭＰ４) <kazuhiro3.hayashi@toshiba.co.jp>
> Subject: Re: [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates
> 
> On Mon, 2024-04-15 at 16:16 +0530, Adithya Balakumar wrote:
>> This allows the creation of a swu file for delta update.
>>
>> Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
>> ---
>>   classes/swupdate.bbclass                      | 32 ++++++++++++++-
>>   .../images/swu/sw-description-delta.tmpl      | 39
>> +++++++++++++++++++
>>   2 files changed, 70 insertions(+), 1 deletion(-)
>>   create mode 100644 recipes-core/images/swu/sw-description-delta.tmpl
>>
>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass index
>> 1a7cd4d..76c7aa5 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -26,6 +26,9 @@ SWU_EBG_UPDATE ?= ""
>>   SWU_EFI_BOOT_DEVICE ?= "/dev/disk/by-uuid/4321-DCBA"
>>   SWU_BOOTLOADER ??= "ebg"
>>   SWU_DESCRIPITION_FILE_BOOTLOADER ??= "${SWU_DESCRIPTION_FILE}-
>> ${SWU_BOOTLOADER}"
>> +SWU_DESCRIPITION_FILE_DELTA_UPDATE ??= "${SWU_DESCRIPTION_FILE}-
>> delta"
> 
> Hi, I'm still wondering if we really need a dedicated sw-description file for that. Can't we just mix in the SWU_DELTA_UPDATE_PROPERTIES into the normal sw-description? Maybe Quirin can help with the integration using the new infrastructure.
> 
> Felix
> 
> [Adithya Balakumar]
> Hi Felix,
> In a case where I have to generate a delta swu and a complete swu, I would need a sw-description file for each. Hence the approach that I have taken here. Please correct me if my understanding is wrong.
> 
> My thought was to have delta update swu as an optional / additional artifact apart from the complete update swu.
> In a case where a reference artifact is not provided (atleast in the case of rdiff), a delta artifact will not be created. But we could still create a complete update swu anyway.
> 

First the question no one answered:
Do we need/want to provide both the full update and the delta update and 
why?

- My thoughts for **not** providing a seperate delta.swu:
   Downstream can select during build if the want delta update or not.
   One swu is less confusing as we need to a suffix/prefix to the full or
   delta update.


- My thoughts for providing a seperate delta.swu:
   We can easily support the usecase for version jump updates.
   e.g. 1.0 -> 2.5

If we don't need both update files:
We replace the properties section with a template variable and generate
similar to the ebg use case.

If we require both delta and full update we additionally need to copy 
the original sw-description and do some regex magic on the delta update 
variant before replacing the template variables - this was done in a 
previous iteration of the patch series.

Quirin

> Would like to understand if you are thinking of a different approach ?
> 
> Adithya
> 
>> +SWU_DELTA_UPDATE_ARTIFACT =
>> "${SWU_ROOTFS_NAME}.delta_update${@get_swu_compression_type(d)}"
>> +SWU_DELTA_UPDATE_ARTIFACT_TYPE =
>> "delta_update${@get_swu_compression_type(d)}"
>>   
>>   SWU_IMAGE_FILE ?= "${IMAGE_FULLNAME}"
>>   SWU_DESCRIPTION_FILE ?= "sw-description"
>> @@ -37,6 +40,7 @@ SWU_SIGNATURE_TYPE ?= "cms"
>>   SWU_BUILDCHROOT_IMAGE_FILE ?=
>> "${@os.path.basename(d.getVar('SWU_IMAGE_FILE'))}"
>>   
>>   IMAGE_TYPEDEP:swu =
>> "${SWU_ROOTFS_TYPE}${@get_swu_compression_type(d)}"
>> +IMAGE_TYPEDEP:swu += "${@ '${SWU_DELTA_UPDATE_ARTIFACT_TYPE}' if
>> d.getVar('DELTA_UPDATE_TYPE') else ''}"
>>   IMAGER_BUILD_DEPS:swu += "${@'swupdate-certificates-key' if
>> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>>   IMAGER_INSTALL:swu += "cpio ${@'openssl swupdate-certificates-key'
>> if bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>>   IMAGE_INSTALL += "${@'swupdate-certificates' if
>> bb.utils.to_boolean(d.getVar('SWU_SIGNED')) else ''}"
>> @@ -44,8 +48,10 @@ IMAGE_INSTALL += "${@'swupdate-certificates' if
>> bb.utils.to_boolean(d.getVar('SW
>>   
>>   IMAGE_SRC_URI:swu = "file://${SWU_DESCRIPTION_FILE}.tmpl"
>>   IMAGE_SRC_URI:swu +=
>> "file://${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
>> +IMAGE_SRC_URI:delta_update +=
>> "file://${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>>   IMAGE_TEMPLATE_FILES:swu = "${SWU_DESCRIPTION_FILE}.tmpl"
>>   IMAGE_TEMPLATE_FILES:swu +=
>> "${SWU_DESCRIPITION_FILE_BOOTLOADER}.tmpl"
>> +IMAGE_TEMPLATE_FILES:delta_update +=
>> "${SWU_DESCRIPITION_FILE_DELTA_UPDATE}.tmpl"
>>   IMAGE_TEMPLATE_VARS:swu = " \
>>       SWU_ROOTFS_PARTITION_NAME \
>>       TARGET_IMAGE_UUID \
>> @@ -58,6 +64,8 @@ IMAGE_TEMPLATE_VARS:swu = " \
>>       SWU_FILE_NODES \
>>       SWU_BOOTLOADER_FILE_NODE \
>>       SWU_SCRIPTS_NODE \
>> +    SWU_DELTA_UPDATE_ARTIFACT \
>> +    SWU_DELTA_UPDATE_PROPERTIES \
>>       "
>>   
>>   # TARGET_IMAGE_UUID needs to be generated before completing the
>> template @@ -148,6 +156,22 @@ python add_scripts_node() {
>>       d.appendVar('SWU_SCRIPTS_NODE', swu_scripts_node)
>>   }
>>   
>> +SWU_EXTEND_SW_DESCRIPTION += "add_swu_delta_update_properties"
>> +python add_swu_delta_update_properties() {
>> +    delta_type = d.getVar('DELTA_UPDATE_TYPE')
>> +    swu_delta_update_properties = ""
>> +    if delta_type == "rdiff":
>> +        swu_delta_update_properties =  'chainhandler =
>> "rdiff_image";'
>> +    elif delta_type == "zchunk":
>> +        zck_url = d.getVar('DELTA_ZCK_URL')
>> +        swu_delta_update_properties = f"""
>> +                        chainhandler = "delta";
>> +                        url = "{zck_url}";
>> +                        zckloglevel = "error";
>> +        """
>> +    d.setVar('SWU_DELTA_UPDATE_PROPERTIES',
>> swu_delta_update_properties)
>> +}
>> +
>>   # convert between swupdate compressor name and imagetype extension
>>   def get_swu_compression_type(d):
>>       swu_ct = d.getVar('SWU_COMPRESSION_TYPE') @@ -166,14 +190,20 @@
>> FILESEXTRAPATHS:append = ":${LAYERDIR_cip-
>> core}/recipes-core/images/swu"
>>   
>>   do_image_swu[depends] += "${PN}:do_transform_template"
>>   do_image_swu[stamp-extra-info] = "${DISTRO}-${MACHINE}"
>> -do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
>> ${SWU_BOOTLOADER}"
>> +do_image_swu[cleandirs] += "${WORKDIR}/swu ${WORKDIR}/swu-
>> ${SWU_BOOTLOADER} ${WORKDIR}/swu-delta"
>>   do_image_swu[prefuncs] = "do_extend_sw_description"
>>   IMAGE_CMD:swu() {
>>       rm -f '${DEPLOY_DIR_IMAGE}/${SWU_IMAGE_FILE}'*.swu
>> +    if [ -z "${DELTA_UPDATE_TYPE}" ]; then
>> +        rm -rf ${WORKDIR}/swu-delta
>> +    fi
>>       cp '${WORKDIR}/${SWU_DESCRIPTION_FILE}'
>> '${WORKDIR}/swu/${SWU_DESCRIPTION_FILE}'
>>       if [ -f '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}' ]; then
>>           cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_BOOTLOADER}'
>> '${WORKDIR}/swu-${SWU_BOOTLOADER}/${SWU_DESCRIPTION_FILE}'
>>       fi
>> +    if [ -n "${DELTA_UPDATE_TYPE}" ]; then
>> +        cp '${WORKDIR}/${SWU_DESCRIPITION_FILE_DELTA_UPDATE}'
>> '${WORKDIR}/swu-delta/${SWU_DESCRIPTION_FILE}'
>> +    fi
>>   
>>       for swu_file in "${WORKDIR}"/swu*; do
>>           swu_file_base=$(basename $swu_file) diff --git
>> a/recipes-core/images/swu/sw-description-delta.tmpl
>> b/recipes-core/images/swu/sw-description-delta.tmpl
>> new file mode 100644
>> index 0000000..7f7d623
>> --- /dev/null
>> +++ b/recipes-core/images/swu/sw-description-delta.tmpl
>> @@ -0,0 +1,39 @@
>> +#
>> +# CIP Core, generic profile
>> +#
>> +# Copyright (c) Siemens AG, 2020
>> +#
>> +# Authors:
>> +#  Quirin Gylstorff <quirin.gylstorff@siemens.com> # #
>> +SPDX-License-Identifier: MIT # software = {
>> +    version = "${SWU_VERSION}";
>> +    name = "${SWU_NAME}";
>> +    ${SWU_HW_COMPAT_NODE}
>> +    images: ({
>> +            filename = "${SWU_DELTA_UPDATE_ARTIFACT}";
>> +            device = "C:BOOT0:linux.efi-
>>> ${ABROOTFS_PART_UUID_A},C:BOOT1:linux.efi->${ABROOTFS_PART_UUID_B}";
>> +            type = "roundrobin";
>> +            ${SWU_COMPRESSION_NODE}
>> +            properties: {
>> +                        subtype = "image";
>> +                        configfilecheck =
>> "/etc/os-release@not_match@IMAGE_UUID=${TARGET_IMAGE_UUID}";
>> +                       ${SWU_DELTA_UPDATE_PROPERTIES}
>> +            };
>> +            sha256 = "${SWU_DELTA_UPDATE_ARTIFACT}-sha256";
>> +    });
>> +    files: ({
>> +            filename = "linux.efi";
>> +            path = "linux.efi";
>> +            type = "roundrobin";
>> +            device = "C:BOOT0:linux.efi->BOOT0,C:BOOT1:linux.efi-
>>> BOOT1";
>> +            filesystem = "vfat";
>> +            properties: {
>> +                        subtype = "kernel";
>> +            };
>> +            sha256 = "linux.efi-sha256";
>> +    }${SWU_FILE_NODES});
>> +}
> 
> --
> Siemens AG, Technology
> Linux Expert Center
> 
> 

Re: [cip-dev] [isar-cip-core][PATCH v3 2/3] swupdate.bbclass: Generate swu for delta updates

[MOESSBAUER, Felix]

> > 
> > Hi, I'm still wondering if we really need a dedicated sw-
> > description file for that. Can't we just mix in the
> > SWU_DELTA_UPDATE_PROPERTIES into the normal sw-description? Maybe
> > Quirin can help with the integration using the new infrastructure.
> > 
> > Felix
> > 
> > [Adithya Balakumar]
> > Hi Felix,
> > In a case where I have to generate a delta swu and a complete swu,
> > I would need a sw-description file for each. Hence the approach
> > that I have taken here. Please correct me if my understanding is
> > wrong.
> > 
> > My thought was to have delta update swu as an optional / additional
> > artifact apart from the complete update swu.
> > In a case where a reference artifact is not provided (atleast in
> > the case of rdiff), a delta artifact will not be created. But we
> > could still create a complete update swu anyway.
> > 
> 
> First the question no one answered:
> Do we need/want to provide both the full update and the delta update
> and 
> why?

I don't think we need to generate both .swu at the same time. However,
the user still shall be able to generate a full swu if requested. This
logic is already available via the overrides.

> 
> - My thoughts for **not** providing a seperate delta.swu:
>    Downstream can select during build if the want delta update or
> not.

Yes!

>    One swu is less confusing as we need to a suffix/prefix to the
> full or
>    delta update.

Yes!

> 
> 
> - My thoughts for providing a seperate delta.swu:
>    We can easily support the usecase for version jump updates.
>    e.g. 1.0 -> 2.5

IMHO, this is an independent topic, as the delta .swu is generated from
the image (rootfs) provided in build/previous-image.

> 
> If we don't need both update files:
> We replace the properties section with a template variable and
> generate
> similar to the ebg use case.

That was my idea as well.

Felix

> 
> If we require both delta and full update we additionally need to copy
> the original sw-description and do some regex magic on the delta
> update 
> variant before replacing the template variables - this was done in a 
> previous iteration of the patch series.
> 
> Quirin
> 
> > Would like to understand if you are thinking of a different
> > approach ?
> > 
> > Adithya
> > 
> > 

-- 
Siemens AG, Technology
Linux Expert Center