[PATCH v4 1/7] lib/oe: add generic functions for overlayfs

[PATCH v4 1/7] lib/oe: add generic functions for overlayfs

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/lib/oe/overlayfs.py | 43 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
 create mode 100644 meta/lib/oe/overlayfs.py

diff --git a/meta/lib/oe/overlayfs.py b/meta/lib/oe/overlayfs.py
new file mode 100644
index 0000000000..21ef710509
--- /dev/null
+++ b/meta/lib/oe/overlayfs.py
@@ -0,0 +1,43 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# This file contains common functions for overlayfs and its QA check
+
+# this function is based on https://github.com/systemd/systemd/blob/main/src/basic/unit-name.c
+def escapeSystemdUnitName(path):
+    escapeMap = {
+        '/': '-',
+        '-': "\\x2d",
+        '\\': "\\x5d"
+    }
+    return "".join([escapeMap.get(c, c) for c in path.strip('/')])
+
+def strForBash(s):
+    return s.replace('\\', '\\\\')
+
+def mountUnitName(unit):
+    return escapeSystemdUnitName(unit) + '.mount'
+
+def helperUnitName(unit):
+    return escapeSystemdUnitName(unit) + '-create-upper-dir.service'
+
+def unitFileList(d):
+    fileList = []
+    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+
+    if not overlayMountPoints:
+        bb.fatal("A recipe uses overlayfs class but there is no OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
+
+    # check that we have required mount points set first
+    requiredMountPoints = d.getVarFlags('OVERLAYFS_WRITABLE_PATHS')
+    for mountPoint in requiredMountPoints:
+        if mountPoint not in overlayMountPoints:
+            bb.fatal("Missing required mount point for OVERLAYFS_MOUNT_POINT[%s] in your MACHINE configuration" % mountPoint)
+
+    for mountPoint in overlayMountPoints:
+        for path in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
+            fileList.append(mountUnitName(path))
+            fileList.append(helperUnitName(path))
+
+    return fileList
+
-- 
2.28.0

[PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Vyacheslav Yurkov]

It's often desired in Embedded System design to have a read-only rootfs.
But a lot of different applications might want to have a read-write access
to some parts of a filesystem. It can be especially useful when your update
mechanism overwrites the whole rootfs, but you want your application data
to be preserved between updates. This class provides a way to achieve that
by means of overlayfs and at the same time keeping the base rootfs read-only.

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/classes/overlayfs.bbclass | 111 +++++++++++++++++++++++++++++++++
 1 file changed, 111 insertions(+)
 create mode 100644 meta/classes/overlayfs.bbclass

diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
new file mode 100644
index 0000000000..8d9b59c9bf
--- /dev/null
+++ b/meta/classes/overlayfs.bbclass
@@ -0,0 +1,111 @@
+# Class for generation of overlayfs mount units
+#
+# It's often desired in Embedded System design to have a read-only rootfs.
+# But a lot of different applications might want to have a read-write access to
+# some parts of a filesystem. It can be especially useful when your update mechanism
+# overwrites the whole rootfs, but you want your application data to be preserved
+# between updates. This class provides a way to achieve that by means
+# of overlayfs and at the same time keeping the base rootfs read-only.
+#
+# Usage example.
+#
+# Set a mount point for a partition overlayfs is going to use as upper layer
+# in your machine configuration. Underlying file system can be anything that
+# is supported by overlayfs. This has to be done in your machine configuration.
+# QA check fails to catch file existence if you redefine this variable in your recipe!
+#
+#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
+#
+# The class assumes you have a data.mount systemd unit defined in your
+# systemd-machine-units recipe and installed to the image.
+#
+# Then you can specify writable directories on a recipe base
+#
+#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
+#
+# To support several mount points you can use a different variable flag. Assume we
+# want to have a writable location on the file system, but not interested where the data
+# survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs file system:
+#
+#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
+#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
+#
+# Note: the class does not support /etc directory itself, because systemd depends on it
+
+REQUIRED_DISTRO_FEATURES += "systemd overlayfs"
+
+inherit systemd features_check
+
+python do_create_overlayfs_units() {
+    CreateDirsUnitTemplate = """[Unit]
+Description=Overlayfs directories setup
+Requires={DATA_MOUNT_UNIT}
+After={DATA_MOUNT_UNIT}
+DefaultDependencies=no
+
+[Service]
+Type=oneshot
+ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p {DATA_MOUNT_POINT}/upper{LOWERDIR}
+RemainAfterExit=true
+StandardOutput=journal
+
+[Install]
+WantedBy=multi-user.target
+"""
+    MountUnitTemplate = """[Unit]
+Description=Overlayfs mount unit
+Requires={CREATE_DIRS_SERVICE}
+After={CREATE_DIRS_SERVICE}
+
+[Mount]
+What=overlay
+Where={LOWERDIR}
+Type=overlay
+Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
+
+[Install]
+WantedBy=multi-user.target
+"""
+
+    def prepareUnits(data, lower):
+        from oe.overlayfs import mountUnitName, helperUnitName
+
+        args = {
+            'DATA_MOUNT_POINT': data,
+            'DATA_MOUNT_UNIT': mountUnitName(data),
+            'CREATE_DIRS_SERVICE': helperUnitName(lower),
+            'LOWERDIR': lower,
+        }
+
+        with open(os.path.join(d.getVar('WORKDIR'), mountUnitName(lower)), 'w') as f:
+            f.write(MountUnitTemplate.format(**args))
+
+        with open(os.path.join(d.getVar('WORKDIR'), helperUnitName(lower)), 'w') as f:
+            f.write(CreateDirsUnitTemplate.format(**args))
+
+    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+    for mountPoint in overlayMountPoints:
+        for lower in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
+            prepareUnits(d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint), lower)
+}
+
+# we need to generate file names early during parsing stage
+python () {
+    from oe.overlayfs import strForBash, unitFileList
+
+    unitList = unitFileList(d)
+    for unit in unitList:
+        d.appendVar('SYSTEMD_SERVICE:' + d.getVar('PN'), ' ' + unit);
+        d.appendVar('FILES:' + d.getVar('PN'), ' ' + strForBash(unit))
+
+    d.setVar('OVERLAYFS_UNIT_LIST', ' '.join([strForBash(s) for s in unitList]))
+}
+
+do_install:append() {
+    install -d ${D}${systemd_system_unitdir}
+    for unit in ${OVERLAYFS_UNIT_LIST}; do
+        install -m 0444 ${WORKDIR}/${unit} ${D}${systemd_system_unitdir}
+    done
+}
+
+addtask create_overlayfs_units before do_install
-- 
2.28.0

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Michael Opdenacker]

Slava, Richard,

On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> It's often desired in Embedded System design to have a read-only rootfs.
> But a lot of different applications might want to have a read-write access
> to some parts of a filesystem. It can be especially useful when your update
> mechanism overwrites the whole rootfs, but you want your application data
> to be preserved between updates. This class provides a way to achieve that
> by means of overlayfs and at the same time keeping the base rootfs read-only.
>
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
>  meta/classes/overlayfs.bbclass | 111 +++++++++++++++++++++++++++++++++
>  1 file changed, 111 insertions(+)
>  create mode 100644 meta/classes/overlayfs.bbclass
>
> diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
> new file mode 100644
> index 0000000000..8d9b59c9bf
> --- /dev/null
> +++ b/meta/classes/overlayfs.bbclass
> @@ -0,0 +1,111 @@
> +# Class for generation of overlayfs mount units
> +#
> +# It's often desired in Embedded System design to have a read-only rootfs.
> +# But a lot of different applications might want to have a read-write access to
> +# some parts of a filesystem. It can be especially useful when your update mechanism
> +# overwrites the whole rootfs, but you want your application data to be preserved
> +# between updates. This class provides a way to achieve that by means
> +# of overlayfs and at the same time keeping the base rootfs read-only.
> +#
> +# Usage example.
> +#
> +# Set a mount point for a partition overlayfs is going to use as upper layer
> +# in your machine configuration. Underlying file system can be anything that
> +# is supported by overlayfs. This has to be done in your machine configuration.
> +# QA check fails to catch file existence if you redefine this variable in your recipe!
> +#
> +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> +#
> +# The class assumes you have a data.mount systemd unit defined in your
> +# systemd-machine-units recipe and installed to the image.
> +#
> +# Then you can specify writable directories on a recipe base
> +#
> +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
> +#
> +# To support several mount points you can use a different variable flag. Assume we
> +# want to have a writable location on the file system, but not interested where the data
> +# survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs file system:
> +#
> +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
> +#
> +# Note: the class does not support /etc directory itself, because systemd depends on it


Many thanks for this new class and the corresponding documentation.
Richard, I guess that's something we'll need to document in the manual,
once it's merge, right?

Thanks again,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Richard Purdie]

On Fri, 2021-08-06 at 15:00 +0200, Michael Opdenacker wrote:
> Slava, Richard,
> 
> On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> > It's often desired in Embedded System design to have a read-only rootfs.
> > But a lot of different applications might want to have a read-write access
> > to some parts of a filesystem. It can be especially useful when your update
> > mechanism overwrites the whole rootfs, but you want your application data
> > to be preserved between updates. This class provides a way to achieve that
> > by means of overlayfs and at the same time keeping the base rootfs read-only.
> > 
> > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> > ---
> >  meta/classes/overlayfs.bbclass | 111 +++++++++++++++++++++++++++++++++
> >  1 file changed, 111 insertions(+)
> >  create mode 100644 meta/classes/overlayfs.bbclass
> > 
> > diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
> > new file mode 100644
> > index 0000000000..8d9b59c9bf
> > --- /dev/null
> > +++ b/meta/classes/overlayfs.bbclass
> > @@ -0,0 +1,111 @@
> > +# Class for generation of overlayfs mount units
> > +#
> > +# It's often desired in Embedded System design to have a read-only rootfs.
> > +# But a lot of different applications might want to have a read-write access to
> > +# some parts of a filesystem. It can be especially useful when your update mechanism
> > +# overwrites the whole rootfs, but you want your application data to be preserved
> > +# between updates. This class provides a way to achieve that by means
> > +# of overlayfs and at the same time keeping the base rootfs read-only.
> > +#
> > +# Usage example.
> > +#
> > +# Set a mount point for a partition overlayfs is going to use as upper layer
> > +# in your machine configuration. Underlying file system can be anything that
> > +# is supported by overlayfs. This has to be done in your machine configuration.
> > +# QA check fails to catch file existence if you redefine this variable in your recipe!
> > +#
> > +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> > +#
> > +# The class assumes you have a data.mount systemd unit defined in your
> > +# systemd-machine-units recipe and installed to the image.
> > +#
> > +# Then you can specify writable directories on a recipe base
> > +#
> > +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
> > +#
> > +# To support several mount points you can use a different variable flag. Assume we
> > +# want to have a writable location on the file system, but not interested where the data
> > +# survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs file system:
> > +#
> > +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> > +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
> > +#
> > +# Note: the class does not support /etc directory itself, because systemd depends on it
> 
> 
> Many thanks for this new class and the corresponding documentation.
> Richard, I guess that's something we'll need to document in the manual,
> once it's merge, right?

Ideally, yes, correct.

Cheers,

Richard

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 3593 bytes --]

I've already updated the docs and will submit the patch as soon as this
series is approved and merged.

Vyacheslav

On Fri, Aug 6, 2021, 16:06 Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2021-08-06 at 15:00 +0200, Michael Opdenacker wrote:
> > Slava, Richard,
> >
> > On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> > > It's often desired in Embedded System design to have a read-only
> rootfs.
> > > But a lot of different applications might want to have a read-write
> access
> > > to some parts of a filesystem. It can be especially useful when your
> update
> > > mechanism overwrites the whole rootfs, but you want your application
> data
> > > to be preserved between updates. This class provides a way to achieve
> that
> > > by means of overlayfs and at the same time keeping the base rootfs
> read-only.
> > >
> > > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> > > ---
> > >  meta/classes/overlayfs.bbclass | 111 +++++++++++++++++++++++++++++++++
> > >  1 file changed, 111 insertions(+)
> > >  create mode 100644 meta/classes/overlayfs.bbclass
> > >
> > > diff --git a/meta/classes/overlayfs.bbclass
> b/meta/classes/overlayfs.bbclass
> > > new file mode 100644
> > > index 0000000000..8d9b59c9bf
> > > --- /dev/null
> > > +++ b/meta/classes/overlayfs.bbclass
> > > @@ -0,0 +1,111 @@
> > > +# Class for generation of overlayfs mount units
> > > +#
> > > +# It's often desired in Embedded System design to have a read-only
> rootfs.
> > > +# But a lot of different applications might want to have a read-write
> access to
> > > +# some parts of a filesystem. It can be especially useful when your
> update mechanism
> > > +# overwrites the whole rootfs, but you want your application data to
> be preserved
> > > +# between updates. This class provides a way to achieve that by means
> > > +# of overlayfs and at the same time keeping the base rootfs read-only.
> > > +#
> > > +# Usage example.
> > > +#
> > > +# Set a mount point for a partition overlayfs is going to use as
> upper layer
> > > +# in your machine configuration. Underlying file system can be
> anything that
> > > +# is supported by overlayfs. This has to be done in your machine
> configuration.
> > > +# QA check fails to catch file existence if you redefine this
> variable in your recipe!
> > > +#
> > > +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> > > +#
> > > +# The class assumes you have a data.mount systemd unit defined in your
> > > +# systemd-machine-units recipe and installed to the image.
> > > +#
> > > +# Then you can specify writable directories on a recipe base
> > > +#
> > > +#   OVERLAYFS_WRITABLE_PATHS[data] =
> "/usr/share/my-custom-application"
> > > +#
> > > +# To support several mount points you can use a different variable
> flag. Assume we
> > > +# want to have a writable location on the file system, but not
> interested where the data
> > > +# survive a reboot. Then we could have a mnt-overlay.mount unit for a
> tmpfs file system:
> > > +#
> > > +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> > > +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] =
> "/usr/share/another-application"
> > > +#
> > > +# Note: the class does not support /etc directory itself, because
> systemd depends on it
> >
> >
> > Many thanks for this new class and the corresponding documentation.
> > Richard, I guess that's something we'll need to document in the manual,
> > once it's merge, right?
>
> Ideally, yes, correct.
>
> Cheers,
>
> Richard
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4509 bytes --]

Re: [OE-core] [PATCH v4 2/7] overlayfs.bbclass: generate overlayfs mount units

[Michael Opdenacker]

On 8/6/21 3:13 PM, Vyacheslav Yurkov wrote:
> I've already updated the docs and will submit the patch as soon as
> this series is approved and merged.


Fantastic. Many thanks :)

Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v4 3/7] rootfs-postcommands: add QA check for overlayfs

[Vyacheslav Yurkov]

The check is conditional and only enabled when overlayfs is set in
DISTRO_FEATURES

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/classes/rootfs-postcommands.bbclass | 25 ++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index fbfa63fcb3..c5746eba13 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -39,6 +39,8 @@ ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd"
 
 ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;'
 
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "overlayfs", "overlayfs_qa_check;", "", d)}'
+
 inherit image-artifact-names
 
 # Sort the user and group entries in /etc by ID in order to make the content
@@ -373,3 +375,26 @@ rootfs_reproducible () {
 		fi
 	fi
 }
+
+python overlayfs_qa_check() {
+    from oe.overlayfs import mountUnitName
+
+    # this is a dumb check for unit existence, not its validity
+    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
+    imagepath = d.getVar("IMAGE_ROOTFS")
+    searchpaths = [oe.path.join(imagepath, d.getVar("sysconfdir"), "systemd", "system"),
+                   oe.path.join(imagepath, d.getVar("systemd_system_unitdir"))]
+
+    allUnitExist = True;
+    for mountPoint in overlayMountPoints:
+        path = d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint)
+        unit = mountUnitName(path)
+
+        if not any(os.path.isfile(oe.path.join(dirpath, unit))
+                   for dirpath in searchpaths):
+            bb.warn('Unit name %s not found in systemd unit directories' % unit)
+            allUnitExist = False;
+
+    if not allUnitExist:
+        bb.fatal('Not all mount units are installed by the BSP')
+}
-- 
2.28.0

[PATCH v4 4/7] systemd-machine-units: add bbappend for meta-selftest

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 .../systemd-machine-units/systemd-machine-units_%.bbappend      | 2 ++
 1 file changed, 2 insertions(+)
 create mode 100644 meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend

diff --git a/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend b/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend
new file mode 100644
index 0000000000..205720982c
--- /dev/null
+++ b/meta-selftest/recipes-test/systemd-machine-units/systemd-machine-units_%.bbappend
@@ -0,0 +1,2 @@
+# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
+include test_recipe.inc
-- 
2.28.0

[PATCH v4 5/7] overlayfs: meta-selftest recipe

[Vyacheslav Yurkov]

The recipe demonstrates example usage of overlayfs bbclass

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 .../overlayfs-user/overlayfs-user.bb            | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb

diff --git a/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb b/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb
new file mode 100644
index 0000000000..60405067de
--- /dev/null
+++ b/meta-selftest/recipes-test/overlayfs-user/overlayfs-user.bb
@@ -0,0 +1,17 @@
+SUMMARY = "Overlayfs class unit test"
+DESCRIPTION = "Contains an overlayfs configuration"
+LICENSE = "MIT"
+
+INHIBIT_DEFAULT_DEPS = "1"
+EXCLUDE_FROM_WORLD = "1"
+
+inherit ${@bb.utils.contains("DISTRO_FEATURES", "overlayfs", "overlayfs", "", d)}
+include test_recipe.inc
+
+OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/my-application"
+
+do_install() {
+    install -d ${D}/usr/share/my-application
+}
+
+FILES:${PN} += "/usr"
-- 
2.28.0

[PATCH v4 6/7] oeqa/selftest: overlayfs unit tests

[Vyacheslav Yurkov]

Unit tests for overlayfs.bbclass

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 meta/lib/oeqa/selftest/cases/overlayfs.py | 171 ++++++++++++++++++++++
 1 file changed, 171 insertions(+)
 create mode 100644 meta/lib/oeqa/selftest/cases/overlayfs.py

diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py b/meta/lib/oeqa/selftest/cases/overlayfs.py
new file mode 100644
index 0000000000..0184d52494
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/overlayfs.py
@@ -0,0 +1,171 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
+
+class OverlayFSTests(OESelftestTestCase):
+    """Overlayfs class usage tests"""
+
+    def getline(self, res, line):
+        for l in res.output.split('\n'):
+            if line in l:
+                return l
+
+    def add_overlay_conf_to_machine(self):
+        machine_inc = """
+OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
+"""
+        self.set_machine_config(machine_inc)
+
+    def test_distro_features_missing(self):
+        """
+        Summary:   Check that required DISTRO_FEATURES are set
+        Expected:  Fail when either systemd or overlayfs are not in DISTRO_FEATURES
+        Author:    Vyacheslav Yurkov <uvv.mail@gmail.com>
+        """
+
+        config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+"""
+        overlayfs_recipe_append = """
+inherit overlayfs
+"""
+        self.write_config(config)
+        self.add_overlay_conf_to_machine()
+        self.write_recipeinc('overlayfs-user', overlayfs_recipe_append)
+
+        res = bitbake('core-image-minimal', ignore_status=True)
+        line = self.getline(res, "overlayfs-user was skipped: missing required distro features")
+        self.assertTrue("overlayfs" in res.output, msg=res.output)
+        self.assertTrue("systemd" in res.output, msg=res.output)
+        self.assertTrue("ERROR: Required build target 'core-image-minimal' has no buildable providers." in res.output, msg=res.output)
+
+    def test_not_all_units_installed(self):
+        """
+        Summary:   Test QA check that we have required mount units in the image
+        Expected:  Fail because mount unit for overlay partition is not installed
+        Author:    Vyacheslav Yurkov <uvv.mail@gmail.com>
+        """
+
+        config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+        self.write_config(config)
+        self.add_overlay_conf_to_machine()
+
+        res = bitbake('core-image-minimal', ignore_status=True)
+        line = self.getline(res, "Unit name mnt-overlay.mount not found in systemd unit directories")
+        self.assertTrue(line and line.startswith("WARNING:"), msg=res.output)
+        line = self.getline(res, "Not all mount units are installed by the BSP")
+        self.assertTrue(line and line.startswith("ERROR:"), msg=res.output)
+
+    def test_mount_unit_not_set(self):
+        """
+        Summary:   Test whether mount unit was set properly
+        Expected:  Fail because mount unit was not set
+        Author:    Vyacheslav Yurkov <uvv.mail@gmail.com>
+        """
+
+        config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+        self.write_config(config)
+
+        res = bitbake('core-image-minimal', ignore_status=True)
+        line = self.getline(res, "A recipe uses overlayfs class but there is no OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
+        self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), msg=res.output)
+
+    def test_wrong_mount_unit_set(self):
+        """
+        Summary:   Test whether mount unit was set properly
+        Expected:  Fail because not the correct flag used for mount unit
+        Author:    Vyacheslav Yurkov <uvv.mail@gmail.com>
+        """
+
+        config = """
+IMAGE_INSTALL:append = " overlayfs-user"
+DISTRO_FEATURES += "systemd overlayfs"
+"""
+
+        wrong_machine_config = """
+OVERLAYFS_MOUNT_POINT[usr-share-overlay] = "/usr/share/overlay"
+"""
+
+        self.write_config(config)
+        self.set_machine_config(wrong_machine_config)
+
+        res = bitbake('core-image-minimal', ignore_status=True)
+        line = self.getline(res, "Missing required mount point for OVERLAYFS_MOUNT_POINT[mnt-overlay] in your MACHINE configuration")
+        self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), msg=res.output)
+
+    def test_correct_image(self):
+        """
+        Summary:   Check that we can create an image when all parameters are
+                   set correctly
+        Expected:  Image is created successfully
+        Author:    Vyacheslav Yurkov <uvv.mail@gmail.com>
+        """
+
+        config = """
+IMAGE_INSTALL:append = " overlayfs-user systemd-machine-units"
+DISTRO_FEATURES += "systemd overlayfs"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+"""
+
+        systemd_machine_unit_append = """
+SYSTEMD_SERVICE:${PN} += " \
+    mnt-overlay.mount \
+"
+
+do_install:append() {
+    install -d ${D}${systemd_system_unitdir}
+    cat <<EOT > ${D}${systemd_system_unitdir}/mnt-overlay.mount
+[Unit]
+Description=Tmpfs directory
+DefaultDependencies=no
+
+[Mount]
+What=tmpfs
+Where=/mnt/overlay
+Type=tmpfs
+Options=mode=1777,strictatime,nosuid,nodev
+
+[Install]
+WantedBy=multi-user.target
+EOT
+}
+
+"""
+
+        self.write_config(config)
+        self.add_overlay_conf_to_machine()
+        self.write_recipeinc('systemd-machine-units', systemd_machine_unit_append)
+
+        bitbake('core-image-minimal')
+
+        def getline_qemu(out, line):
+            for l in out.split('\n'):
+                if line in l:
+                    return l
+
+        with runqemu('core-image-minimal') as qemu:
+            # Check that we have /mnt/overlay fs mounted as tmpfs and
+            # /usr/share/my-application as an overlay (see overlayfs-user recipe)
+            status, output = qemu.run_serial("/bin/mount -t tmpfs,overlay")
+
+            line = getline_qemu(output, "on /mnt/overlay")
+            self.assertTrue(line and line.startswith("tmpfs"), msg=output)
+
+            line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/my-application")
+            self.assertTrue(line and line.startswith("overlay"), msg=output)
-- 
2.28.0

[PATCH v4 7/7] MAINTAINERS: add overlayfs maintainer

[Vyacheslav Yurkov]

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
 MAINTAINERS.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/MAINTAINERS.md b/MAINTAINERS.md
index 2ddcde6878..36a9bde90c 100644
--- a/MAINTAINERS.md
+++ b/MAINTAINERS.md
@@ -40,6 +40,7 @@ Component/Subsystem Maintainers
 * opkg: Alex Stewart
 * devtool: Saul Wold
 * eSDK: Saul Wold
+* overlayfs: Vyacheslav Yurkov
 
 Maintainers needed
 ------------------
-- 
2.28.0

Re: [OE-core] [PATCH v4 7/7] MAINTAINERS: add overlayfs maintainer

[Michael Opdenacker]

On 8/6/21 2:06 PM, Vyacheslav Yurkov wrote:
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
>  MAINTAINERS.md | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/MAINTAINERS.md b/MAINTAINERS.md
> index 2ddcde6878..36a9bde90c 100644
> --- a/MAINTAINERS.md
> +++ b/MAINTAINERS.md
> @@ -40,6 +40,7 @@ Component/Subsystem Maintainers
>  * opkg: Alex Stewart
>  * devtool: Saul Wold
>  * eSDK: Saul Wold
> +* overlayfs: Vyacheslav Yurkov


Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>

;-)

Michael.
-- 

Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com