* [PATCH 1/8] files: add preinit.sh.in
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 12:42 ` [OE-core] " Richard Purdie
2021-12-10 11:50 ` [PATCH 2/8] overlayfs-etc: mount etc as overlayfs Vyacheslav Yurkov
` (6 subsequent siblings)
7 siblings, 1 reply; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
A template init script for overlayfs-etc class
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/files/preinit.sh.in | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
create mode 100644 meta/files/preinit.sh.in
diff --git a/meta/files/preinit.sh.in b/meta/files/preinit.sh.in
new file mode 100644
index 0000000000..2ebb6c9224
--- /dev/null
+++ b/meta/files/preinit.sh.in
@@ -0,0 +1,29 @@
+#!/bin/sh
+
+echo "PREINIT: Start"
+
+PATH=/sbin:/bin:/usr/sbin:/usr/bin
+mount -o remount,rw /
+
+mkdir -p /proc
+mkdir -p /sys
+mkdir -p /run
+mkdir -p /var/run
+
+mount -t proc proc /proc
+mount -t sysfs sysfs /sys
+
+[ -z "$CONSOLE" ] && CONSOLE="/dev/console"
+
+mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}
+if mount -n -t {OVERLAYFS_ETC_FSTYPE} -o {OVERLAYFS_ETC_MOUNT_OPTIONS} {OVERLAYFS_ETC_DEVICE} {OVERLAYFS_ETC_MOUNT_POINT}
+then
+ mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper
+ mkdir -p {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/work
+ mount -n -t overlay -o upperdir={OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper,lowerdir=/etc,workdir={OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/work {OVERLAYFS_ETC_MOUNT_POINT}/overlay-etc/upper /etc || echo "PREINIT: Mounting etc-overlay failed!"
+else
+ echo "PREINIT: Mounting </data> failed!"
+fi
+
+echo "PREINIT: done; starting </sbin/init>"
+exec {SBIN_INIT_NAME}
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* Re: [OE-core] [PATCH 1/8] files: add preinit.sh.in
2021-12-10 11:50 ` [PATCH 1/8] files: add preinit.sh.in Vyacheslav Yurkov
@ 2021-12-10 12:42 ` Richard Purdie
0 siblings, 0 replies; 11+ messages in thread
From: Richard Purdie @ 2021-12-10 12:42 UTC (permalink / raw)
To: Vyacheslav Yurkov, openembedded-core
On Fri, 2021-12-10 at 12:50 +0100, Vyacheslav Yurkov wrote:
> A template init script for overlayfs-etc class
>
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
> meta/files/preinit.sh.in | 29 +++++++++++++++++++++++++++++
> 1 file changed, 29 insertions(+)
> create mode 100644 meta/files/preinit.sh.in
Can you call this something like overlayfs-etc-preinit.sh please?
That way we can know roughly what is using it!
Cheers,
Richard
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 2/8] overlayfs-etc: mount etc as overlayfs
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 1/8] files: add preinit.sh.in Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 3/8] wic: image for overlayfs-etc tests Vyacheslav Yurkov
` (5 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
This class provides an image feature that mounts /etc as an overlayfs
file system. This is an extension for existing overlayfs class, which
doesn't support /etc
Signed-off-by: Alfred Schapansky <alfred.schapansky@avantys.de>
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/classes/overlayfs-etc.bbclass | 76 ++++++++++++++++++++++++++++++
1 file changed, 76 insertions(+)
create mode 100644 meta/classes/overlayfs-etc.bbclass
diff --git a/meta/classes/overlayfs-etc.bbclass b/meta/classes/overlayfs-etc.bbclass
new file mode 100644
index 0000000000..5e89339269
--- /dev/null
+++ b/meta/classes/overlayfs-etc.bbclass
@@ -0,0 +1,76 @@
+# Class for setting up /etc in overlayfs
+#
+# In order to have /etc directory in overlayfs a special handling at early boot stage is required
+# The idea is to supply a custom init script that mounts /etc before launching actual init program,
+# because the latter already requires /etc to be mounted
+#
+# The configuration must be machine specific. You should at least set these three variables:
+# OVERLAYFS_ETC_MOUNT_POINT ?= "/data"
+# OVERLAYFS_ETC_FSTYPE ?= "ext4"
+# OVERLAYFS_ETC_DEVICE ?= "/dev/mmcblk0p2"
+#
+# To control more mount options you should consider setting mount options:
+# OVERLAYFS_ETC_MOUNT_OPTIONS ?= "defaults"
+#
+# The class provides two options for /sbin/init generation
+# 1. Default option is to rename original /sbin/init to /sbin/init.orig and place generated init under
+# original name, i.e. /sbin/init. It has an advantage that you won't need to change any kernel
+# parameters in order to make it work, but it poses a restriction that package-management can't
+# be used, becaause updating init manager would remove generated script
+# 2. If you are would like to keep original init as is, you can set
+# OVERLAYFS_ETC_USE_ORIG_INIT_NAME = "0"
+# Then generated init will be named /sbin/preinit and you would need to extend you kernel parameters
+# manually in your bootloader configuration.
+#
+# Regardless which mode you choose, update and migration strategy of configuration files under /etc
+# overlay is out of scope of this class
+
+ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("IMAGE_FEATURES", "overlayfs-etc", "create_overlayfs_etc_preinit;", "", d)}'
+IMAGE_FEATURES_CONFLICTS_overlayfs-etc = "package-management"
+
+OVERLAYFS_ETC_MOUNT_POINT ??= ""
+OVERLAYFS_ETC_FSTYPE ??= ""
+OVERLAYFS_ETC_DEVICE ??= ""
+OVERLAYFS_ETC_USE_ORIG_INIT_NAME ??= "1"
+OVERLAYFS_ETC_MOUNT_OPTIONS ??= "defaults"
+OVERLAYFS_ETC_INIT_TEMPLATE ??= "${COREBASE}/meta/files/preinit.sh.in"
+
+python create_overlayfs_etc_preinit() {
+ overlayEtcMountPoint = d.getVar("OVERLAYFS_ETC_MOUNT_POINT")
+ overlayEtcFsType = d.getVar("OVERLAYFS_ETC_FSTYPE")
+ overlayEtcDevice = d.getVar("OVERLAYFS_ETC_DEVICE")
+
+ if not overlayEtcMountPoint:
+ bb.fatal("OVERLAYFS_ETC_MOUNT_POINT must be set in your MACHINE configuration")
+ if not overlayEtcDevice:
+ bb.fatal("OVERLAYFS_ETC_DEVICE must be set in your MACHINE configuration")
+ if not overlayEtcFsType:
+ bb.fatal("OVERLAYFS_ETC_FSTYPE should contain a valid file system type on {0}".format(overlayEtcDevice))
+
+ with open(d.getVar("OVERLAYFS_ETC_INIT_TEMPLATE"), "r") as f:
+ PreinitTemplate = f.read()
+
+ useOrigInit = oe.types.boolean(d.getVar('OVERLAYFS_ETC_USE_ORIG_INIT_NAME'))
+ preinitPath = oe.path.join(d.getVar("IMAGE_ROOTFS"), d.getVar("base_sbindir"), "preinit")
+ initBaseName = oe.path.join(d.getVar("base_sbindir"), "init")
+ origInitNameSuffix = ".orig"
+
+ args = {
+ 'OVERLAYFS_ETC_MOUNT_POINT': overlayEtcMountPoint,
+ 'OVERLAYFS_ETC_MOUNT_OPTIONS': d.getVar('OVERLAYFS_ETC_MOUNT_OPTIONS'),
+ 'OVERLAYFS_ETC_FSTYPE': overlayEtcFsType,
+ 'OVERLAYFS_ETC_DEVICE': overlayEtcDevice,
+ 'SBIN_INIT_NAME': initBaseName + origInitNameSuffix if useOrigInit else initBaseName
+ }
+
+ if useOrigInit:
+ # rename original /sbin/init
+ origInit = oe.path.join(d.getVar("IMAGE_ROOTFS"), initBaseName)
+ bb.debug(1, "rootfs path %s, init path %s, test %s" % (d.getVar('IMAGE_ROOTFS'), origInit, d.getVar("IMAGE_ROOTFS")))
+ bb.utils.rename(origInit, origInit + origInitNameSuffix)
+ preinitPath = origInit
+
+ with open(preinitPath, 'w') as f:
+ f.write(PreinitTemplate.format(**args))
+ os.chmod(preinitPath, 0o755)
+}
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 3/8] wic: image for overlayfs-etc tests
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 1/8] files: add preinit.sh.in Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 2/8] overlayfs-etc: mount etc as overlayfs Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 4/8] image: add overlayfs-etc image feature Vyacheslav Yurkov
` (4 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Introduce wic image for overlayfs-etc tests with a dedicated /data
partition and configurable kernel parameters
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta-selftest/wic/overlayfs_etc.wks.in | 4 ++++
1 file changed, 4 insertions(+)
create mode 100644 meta-selftest/wic/overlayfs_etc.wks.in
diff --git a/meta-selftest/wic/overlayfs_etc.wks.in b/meta-selftest/wic/overlayfs_etc.wks.in
new file mode 100644
index 0000000000..1e1e5836e7
--- /dev/null
+++ b/meta-selftest/wic/overlayfs_etc.wks.in
@@ -0,0 +1,4 @@
+part /boot --active --source bootimg-biosplusefi --ondisk sda --sourceparams="loader=grub-efi" --align 1024
+part / --source rootfs --ondisk sda --fstype=ext4 --use-uuid --align 1024
+part --ondisk sda --fstype=ext4 --size=5 --align 1024
+bootloader --ptable gpt --timeout=1 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0 ${OVERLAYFS_INIT_OPTION}"
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 4/8] image: add overlayfs-etc image feature
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
` (2 preceding siblings ...)
2021-12-10 11:50 ` [PATCH 3/8] wic: image for overlayfs-etc tests Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 5/8] oeqa/selftest: overlayfs helper function Vyacheslav Yurkov
` (3 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/classes/image.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8a46b4852c..2b0ce4a988 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -15,6 +15,7 @@ IMGCLASSES += "${@bb.utils.contains('IMAGE_FSTYPES', 'container', 'image-contain
IMGCLASSES += "image_types_wic"
IMGCLASSES += "rootfs-postcommands"
IMGCLASSES += "image-postinst-intercepts"
+IMGCLASSES += "overlayfs-etc"
inherit ${IMGCLASSES}
TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
@@ -33,7 +34,7 @@ INHIBIT_DEFAULT_DEPS = "1"
# IMAGE_FEATURES may contain any available package group
IMAGE_FEATURES ?= ""
IMAGE_FEATURES[type] = "list"
-IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging"
+IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password allow-empty-password allow-root-login post-install-logging overlayfs-etc"
# Generate companion debugfs?
IMAGE_GEN_DEBUGFS ?= "0"
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 5/8] oeqa/selftest: overlayfs helper function
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
` (3 preceding siblings ...)
2021-12-10 11:50 ` [PATCH 4/8] image: add overlayfs-etc image feature Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 6/8] oeqa/selftest: unit tests for overlayfs-etc Vyacheslav Yurkov
` (2 subsequent siblings)
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Move helper functions out of class scope so they can be used in other tests
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/lib/oeqa/selftest/cases/overlayfs.py | 32 +++++++++++------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py b/meta/lib/oeqa/selftest/cases/overlayfs.py
index 84242a1605..43415778ce 100644
--- a/meta/lib/oeqa/selftest/cases/overlayfs.py
+++ b/meta/lib/oeqa/selftest/cases/overlayfs.py
@@ -5,16 +5,16 @@
from oeqa.selftest.case import OESelftestTestCase
from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu
-class OverlayFSTests(OESelftestTestCase):
- """Overlayfs class usage tests"""
+def getline_qemu(out, line):
+ for l in out.split('\n'):
+ if line in l:
+ return l
- def getline_qemu(self, out, line):
- for l in out.split('\n'):
- if line in l:
- return l
+def getline(res, line):
+ return getline_qemu(res.output, line)
- def getline(self, res, line):
- return self.getline_qemu(res.output, line)
+class OverlayFSTests(OESelftestTestCase):
+ """Overlayfs class usage tests"""
def add_overlay_conf_to_machine(self):
machine_inc = """
@@ -40,7 +40,7 @@ inherit overlayfs
self.write_recipeinc('overlayfs-user', overlayfs_recipe_append)
res = bitbake('core-image-minimal', ignore_status=True)
- line = self.getline(res, "overlayfs-user was skipped: missing required distro features")
+ line = getline(res, "overlayfs-user was skipped: missing required distro features")
self.assertTrue("overlayfs" in res.output, msg=res.output)
self.assertTrue("systemd" in res.output, msg=res.output)
self.assertTrue("ERROR: Required build target 'core-image-minimal' has no buildable providers." in res.output, msg=res.output)
@@ -61,9 +61,9 @@ DISTRO_FEATURES += "systemd overlayfs"
self.add_overlay_conf_to_machine()
res = bitbake('core-image-minimal', ignore_status=True)
- line = self.getline(res, "Unit name mnt-overlay.mount not found in systemd unit directories")
+ line = getline(res, "Unit name mnt-overlay.mount not found in systemd unit directories")
self.assertTrue(line and line.startswith("WARNING:"), msg=res.output)
- line = self.getline(res, "Not all mount units are installed by the BSP")
+ line = getline(res, "Not all mount units are installed by the BSP")
self.assertTrue(line and line.startswith("ERROR:"), msg=res.output)
def test_mount_unit_not_set(self):
@@ -81,7 +81,7 @@ DISTRO_FEATURES += "systemd overlayfs"
self.write_config(config)
res = bitbake('core-image-minimal', ignore_status=True)
- line = self.getline(res, "A recipe uses overlayfs class but there is no OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
+ line = getline(res, "A recipe uses overlayfs class but there is no OVERLAYFS_MOUNT_POINT set in your MACHINE configuration")
self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), msg=res.output)
def test_wrong_mount_unit_set(self):
@@ -104,7 +104,7 @@ OVERLAYFS_MOUNT_POINT[usr-share-overlay] = "/usr/share/overlay"
self.set_machine_config(wrong_machine_config)
res = bitbake('core-image-minimal', ignore_status=True)
- line = self.getline(res, "Missing required mount point for OVERLAYFS_MOUNT_POINT[mnt-overlay] in your MACHINE configuration")
+ line = getline(res, "Missing required mount point for OVERLAYFS_MOUNT_POINT[mnt-overlay] in your MACHINE configuration")
self.assertTrue(line and line.startswith("Parsing recipes...ERROR:"), msg=res.output)
def test_correct_image(self):
@@ -201,11 +201,11 @@ EOT
# /usr/share/my-application as an overlay (see overlayfs-user recipe)
status, output = qemu.run_serial("/bin/mount -t tmpfs,overlay")
- line = self.getline_qemu(output, "on /mnt/overlay")
+ line = getline_qemu(output, "on /mnt/overlay")
self.assertTrue(line and line.startswith("tmpfs"), msg=output)
- line = self.getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/my-application")
+ line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/my-application")
self.assertTrue(line and line.startswith("overlay"), msg=output)
- line = self.getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/another-overlay-mount")
+ line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/another-overlay-mount")
self.assertTrue(line and line.startswith("overlay"), msg=output)
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 6/8] oeqa/selftest: unit tests for overlayfs-etc
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
` (4 preceding siblings ...)
2021-12-10 11:50 ` [PATCH 5/8] oeqa/selftest: overlayfs helper function Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 7/8] overlayfs: update notes on /etc Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 8/8] overlayfs: move templates to files directory Vyacheslav Yurkov
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/lib/oeqa/selftest/cases/overlayfs.py | 184 ++++++++++++++++++++++
1 file changed, 184 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py b/meta/lib/oeqa/selftest/cases/overlayfs.py
index 43415778ce..82007fade7 100644
--- a/meta/lib/oeqa/selftest/cases/overlayfs.py
+++ b/meta/lib/oeqa/selftest/cases/overlayfs.py
@@ -209,3 +209,187 @@ EOT
line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/another-overlay-mount")
self.assertTrue(line and line.startswith("overlay"), msg=output)
+
+class OverlayFSEtcRunTimeTests(OESelftestTestCase):
+ """overlayfs-etc class tests"""
+
+ def test_all_required_variables_set(self):
+ """
+ Summary: Check that required variables are set
+ Expected: Fail when any of required variables is missing
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ configBase = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+# Image configuration for overlayfs-etc
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+IMAGE_FEATURES:remove = "package-management"
+"""
+ configMountPoint = """
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+"""
+ configDevice = """
+OVERLAYFS_ETC_DEVICE = "/dev/mmcblk0p1"
+"""
+
+ self.write_config(configBase)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_MOUNT_POINT must be set in your MACHINE configuration")
+ self.assertTrue(line, msg=res.output)
+
+ self.append_config(configMountPoint)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_DEVICE must be set in your MACHINE configuration")
+ self.assertTrue(line, msg=res.output)
+
+ self.append_config(configDevice)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_FSTYPE should contain a valid file system type on /dev/mmcblk0p1")
+ self.assertTrue(line, msg=res.output)
+
+ def test_image_feature_conflict(self):
+ """
+ Summary: Overlayfs-etc is not allowed to be used with package-management
+ Expected: Feature conflict
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+EXTRA_IMAGE_FEATURES += "package-management"
+"""
+
+ self.write_config(config)
+
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "contains conflicting IMAGE_FEATURES")
+ self.assertTrue("overlayfs-etc" in res.output, msg=res.output)
+ self.assertTrue("package-management" in res.output, msg=res.output)
+
+ def test_image_feature_is_missing_class_included(self):
+ configAppend = """
+INHERIT += "overlayfs-etc"
+"""
+ self.run_check_image_feature(configAppend)
+
+ def test_image_feature_is_missing(self):
+ self.run_check_image_feature()
+
+ def run_check_image_feature(self, appendToConfig=""):
+ """
+ Summary: Overlayfs-etc class is not applied when image feature is not set
+ even if we inherit it directly,
+ Expected: Image is created successfully but /etc is not an overlay
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = f"""
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+IMAGE_FSTYPES += "wic"
+WKS_FILE = "overlayfs_etc.wks.in"
+
+EXTRA_IMAGE_FEATURES += "read-only-rootfs"
+# Image configuration for overlayfs-etc
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+OVERLAYFS_ETC_DEVICE = "/dev/sda3"
+{appendToConfig}
+"""
+
+ self.write_config(config)
+
+ bitbake('core-image-minimal')
+
+ with runqemu('core-image-minimal', image_fstype='wic') as qemu:
+ status, output = qemu.run_serial("/bin/mount")
+
+ line = getline_qemu(output, "upperdir=/data/overlay-etc/upper")
+ self.assertFalse(line, msg=output)
+
+ def test_sbin_init_preinit(self):
+ self.run_sbin_init(False)
+
+ def test_sbin_init_original(self):
+ self.run_sbin_init(True)
+
+ def run_sbin_init(self, origInit):
+ """
+ Summary: Confirm we can replace original init and mount overlay on top of /etc
+ Expected: Image is created successfully and /etc is mounted as an overlay
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+IMAGE_FSTYPES += "wic"
+OVERLAYFS_INIT_OPTION = "{OVERLAYFS_INIT_OPTION}"
+WKS_FILE = "overlayfs_etc.wks.in"
+
+EXTRA_IMAGE_FEATURES += "read-only-rootfs"
+# Image configuration for overlayfs-etc
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+IMAGE_FEATURES:remove = "package-management"
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+OVERLAYFS_ETC_FSTYPE = "ext4"
+OVERLAYFS_ETC_DEVICE = "/dev/sda3"
+OVERLAYFS_ETC_USE_ORIG_INIT_NAME = "{OVERLAYFS_ETC_USE_ORIG_INIT_NAME}"
+"""
+
+ args = {
+ 'OVERLAYFS_INIT_OPTION': "" if origInit else "init=/sbin/preinit",
+ 'OVERLAYFS_ETC_USE_ORIG_INIT_NAME': int(origInit == True)
+ }
+
+ self.write_config(config.format(**args))
+
+ bitbake('core-image-minimal')
+ testFile = "/etc/my-test-data"
+
+ with runqemu('core-image-minimal', image_fstype='wic', discard_writes=False) as qemu:
+ status, output = qemu.run_serial("/bin/mount")
+
+ line = getline_qemu(output, "/dev/sda3")
+ self.assertTrue("/data" in output, msg=output)
+
+ line = getline_qemu(output, "upperdir=/data/overlay-etc/upper")
+ self.assertTrue(line and line.startswith("/data/overlay-etc/upper on /etc type overlay"), msg=output)
+
+ status, output = qemu.run_serial("touch " + testFile)
+ status, output = qemu.run_serial("sync")
+ status, output = qemu.run_serial("ls -1 " + testFile)
+ line = getline_qemu(output, testFile)
+ self.assertTrue(line and line.startswith(testFile), msg=output)
+
+ # Check that file exists in /etc after reboot
+ with runqemu('core-image-minimal', image_fstype='wic') as qemu:
+ status, output = qemu.run_serial("ls -1 " + testFile)
+ line = getline_qemu(output, testFile)
+ self.assertTrue(line and line.startswith(testFile), msg=output)
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 7/8] overlayfs: update notes on /etc
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
` (5 preceding siblings ...)
2021-12-10 11:50 ` [PATCH 6/8] oeqa/selftest: unit tests for overlayfs-etc Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
2021-12-10 11:50 ` [PATCH 8/8] overlayfs: move templates to files directory Vyacheslav Yurkov
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/classes/overlayfs.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
index 3c0f4dc882..f1b8086ea8 100644
--- a/meta/classes/overlayfs.bbclass
+++ b/meta/classes/overlayfs.bbclass
@@ -31,6 +31,7 @@
# OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
#
# Note: the class does not support /etc directory itself, because systemd depends on it
+# For /etc directory use overlayfs-etc class
REQUIRED_DISTRO_FEATURES += "systemd overlayfs"
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread* [PATCH 8/8] overlayfs: move templates to files directory
2021-12-10 11:50 [PATCH v2 0/8] Pull request (cover letter only) Vyacheslav Yurkov
` (6 preceding siblings ...)
2021-12-10 11:50 ` [PATCH 7/8] overlayfs: update notes on /etc Vyacheslav Yurkov
@ 2021-12-10 11:50 ` Vyacheslav Yurkov
7 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 11:50 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/classes/overlayfs.bbclass | 52 ++++----------------
meta/files/overlayfs-all-overlays.service.in | 12 +++++
meta/files/overlayfs-create-dirs.service.in | 14 ++++++
meta/files/overlayfs-unit.mount.in | 13 +++++
4 files changed, 49 insertions(+), 42 deletions(-)
create mode 100644 meta/files/overlayfs-all-overlays.service.in
create mode 100644 meta/files/overlayfs-create-dirs.service.in
create mode 100644 meta/files/overlayfs-unit.mount.in
diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
index f1b8086ea8..4a860f7308 100644
--- a/meta/classes/overlayfs.bbclass
+++ b/meta/classes/overlayfs.bbclass
@@ -37,51 +37,19 @@ REQUIRED_DISTRO_FEATURES += "systemd overlayfs"
inherit systemd features_check
+OVERLAYFS_CREATE_DIRS_TEMPLATE ??= "${COREBASE}/meta/files/overlayfs-create-dirs.service.in"
+OVERLAYFS_MOUNT_UNIT_TEMPLATE ??= "${COREBASE}/meta/files/overlayfs-unit.mount.in"
+OVERLAYFS_ALL_OVERLAYS_TEMPLATE ??= "${COREBASE}/meta/files/overlayfs-all-overlays.service.in"
+
python do_create_overlayfs_units() {
from oe.overlayfs import mountUnitName
- CreateDirsUnitTemplate = """[Unit]
-Description=Overlayfs directories setup
-Requires={DATA_MOUNT_UNIT}
-After={DATA_MOUNT_UNIT}
-DefaultDependencies=no
-
-[Service]
-Type=oneshot
-ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p {DATA_MOUNT_POINT}/upper{LOWERDIR}
-RemainAfterExit=true
-StandardOutput=journal
-
-[Install]
-WantedBy=multi-user.target
-"""
- MountUnitTemplate = """[Unit]
-Description=Overlayfs mount unit
-Requires={CREATE_DIRS_SERVICE}
-After={CREATE_DIRS_SERVICE}
-
-[Mount]
-What=overlay
-Where={LOWERDIR}
-Type=overlay
-Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
-
-[Install]
-WantedBy=multi-user.target
-"""
- AllOverlaysTemplate = """[Unit]
-Description=Groups all overlays required by {PN} in one unit
-After={ALL_OVERLAYFS_UNITS}
-Requires={ALL_OVERLAYFS_UNITS}
-
-[Service]
-Type=oneshot
-ExecStart=/bin/true
-RemainAfterExit=true
-
-[Install]
-WantedBy=local-fs.target
-"""
+ with open(d.getVar("OVERLAYFS_CREATE_DIRS_TEMPLATE"), "r") as f:
+ CreateDirsUnitTemplate = f.read()
+ with open(d.getVar("OVERLAYFS_MOUNT_UNIT_TEMPLATE"), "r") as f:
+ MountUnitTemplate = f.read()
+ with open(d.getVar("OVERLAYFS_ALL_OVERLAYS_TEMPLATE"), "r") as f:
+ AllOverlaysTemplate = f.read()
def prepareUnits(data, lower):
from oe.overlayfs import helperUnitName
diff --git a/meta/files/overlayfs-all-overlays.service.in b/meta/files/overlayfs-all-overlays.service.in
new file mode 100644
index 0000000000..74ee4e90ae
--- /dev/null
+++ b/meta/files/overlayfs-all-overlays.service.in
@@ -0,0 +1,12 @@
+[Unit]
+Description=Groups all overlays required by {PN} in one unit
+After={ALL_OVERLAYFS_UNITS}
+Requires={ALL_OVERLAYFS_UNITS}
+
+[Service]
+Type=oneshot
+ExecStart=/bin/true
+RemainAfterExit=true
+
+[Install]
+WantedBy=local-fs.target
diff --git a/meta/files/overlayfs-create-dirs.service.in b/meta/files/overlayfs-create-dirs.service.in
new file mode 100644
index 0000000000..17204145f2
--- /dev/null
+++ b/meta/files/overlayfs-create-dirs.service.in
@@ -0,0 +1,14 @@
+[Unit]
+Description=Overlayfs directories setup
+Requires={DATA_MOUNT_UNIT}
+After={DATA_MOUNT_UNIT}
+DefaultDependencies=no
+
+[Service]
+Type=oneshot
+ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p {DATA_MOUNT_POINT}/upper{LOWERDIR}
+RemainAfterExit=true
+StandardOutput=journal
+
+[Install]
+WantedBy=multi-user.target
diff --git a/meta/files/overlayfs-unit.mount.in b/meta/files/overlayfs-unit.mount.in
new file mode 100644
index 0000000000..1d33b7e39c
--- /dev/null
+++ b/meta/files/overlayfs-unit.mount.in
@@ -0,0 +1,13 @@
+[Unit]
+Description=Overlayfs mount unit
+Requires={CREATE_DIRS_SERVICE}
+After={CREATE_DIRS_SERVICE}
+
+[Mount]
+What=overlay
+Where={LOWERDIR}
+Type=overlay
+Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
+
+[Install]
+WantedBy=multi-user.target
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 6/8] oeqa/selftest: unit tests for overlayfs-etc
2021-12-10 13:01 [PATCH v3 0/8] Pull request (cover letter only) Vyacheslav Yurkov
@ 2021-12-10 13:01 ` Vyacheslav Yurkov
0 siblings, 0 replies; 11+ messages in thread
From: Vyacheslav Yurkov @ 2021-12-10 13:01 UTC (permalink / raw)
To: openembedded-core
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
---
meta/lib/oeqa/selftest/cases/overlayfs.py | 184 ++++++++++++++++++++++
1 file changed, 184 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py b/meta/lib/oeqa/selftest/cases/overlayfs.py
index 43415778ce..82007fade7 100644
--- a/meta/lib/oeqa/selftest/cases/overlayfs.py
+++ b/meta/lib/oeqa/selftest/cases/overlayfs.py
@@ -209,3 +209,187 @@ EOT
line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/another-overlay-mount")
self.assertTrue(line and line.startswith("overlay"), msg=output)
+
+class OverlayFSEtcRunTimeTests(OESelftestTestCase):
+ """overlayfs-etc class tests"""
+
+ def test_all_required_variables_set(self):
+ """
+ Summary: Check that required variables are set
+ Expected: Fail when any of required variables is missing
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ configBase = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+# Image configuration for overlayfs-etc
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+IMAGE_FEATURES:remove = "package-management"
+"""
+ configMountPoint = """
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+"""
+ configDevice = """
+OVERLAYFS_ETC_DEVICE = "/dev/mmcblk0p1"
+"""
+
+ self.write_config(configBase)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_MOUNT_POINT must be set in your MACHINE configuration")
+ self.assertTrue(line, msg=res.output)
+
+ self.append_config(configMountPoint)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_DEVICE must be set in your MACHINE configuration")
+ self.assertTrue(line, msg=res.output)
+
+ self.append_config(configDevice)
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "OVERLAYFS_ETC_FSTYPE should contain a valid file system type on /dev/mmcblk0p1")
+ self.assertTrue(line, msg=res.output)
+
+ def test_image_feature_conflict(self):
+ """
+ Summary: Overlayfs-etc is not allowed to be used with package-management
+ Expected: Feature conflict
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+EXTRA_IMAGE_FEATURES += "package-management"
+"""
+
+ self.write_config(config)
+
+ res = bitbake('core-image-minimal', ignore_status=True)
+ line = getline(res, "contains conflicting IMAGE_FEATURES")
+ self.assertTrue("overlayfs-etc" in res.output, msg=res.output)
+ self.assertTrue("package-management" in res.output, msg=res.output)
+
+ def test_image_feature_is_missing_class_included(self):
+ configAppend = """
+INHERIT += "overlayfs-etc"
+"""
+ self.run_check_image_feature(configAppend)
+
+ def test_image_feature_is_missing(self):
+ self.run_check_image_feature()
+
+ def run_check_image_feature(self, appendToConfig=""):
+ """
+ Summary: Overlayfs-etc class is not applied when image feature is not set
+ even if we inherit it directly,
+ Expected: Image is created successfully but /etc is not an overlay
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = f"""
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+IMAGE_FSTYPES += "wic"
+WKS_FILE = "overlayfs_etc.wks.in"
+
+EXTRA_IMAGE_FEATURES += "read-only-rootfs"
+# Image configuration for overlayfs-etc
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+OVERLAYFS_ETC_DEVICE = "/dev/sda3"
+{appendToConfig}
+"""
+
+ self.write_config(config)
+
+ bitbake('core-image-minimal')
+
+ with runqemu('core-image-minimal', image_fstype='wic') as qemu:
+ status, output = qemu.run_serial("/bin/mount")
+
+ line = getline_qemu(output, "upperdir=/data/overlay-etc/upper")
+ self.assertFalse(line, msg=output)
+
+ def test_sbin_init_preinit(self):
+ self.run_sbin_init(False)
+
+ def test_sbin_init_original(self):
+ self.run_sbin_init(True)
+
+ def run_sbin_init(self, origInit):
+ """
+ Summary: Confirm we can replace original init and mount overlay on top of /etc
+ Expected: Image is created successfully and /etc is mounted as an overlay
+ Author: Vyacheslav Yurkov <uvv.mail@gmail.com>
+ """
+
+ config = """
+DISTRO_FEATURES += "systemd"
+
+# Use systemd as init manager
+VIRTUAL-RUNTIME_init_manager = "systemd"
+
+# enable overlayfs in the kernel
+KERNEL_EXTRA_FEATURES:append = " features/overlayfs/overlayfs.scc"
+
+IMAGE_FSTYPES += "wic"
+OVERLAYFS_INIT_OPTION = "{OVERLAYFS_INIT_OPTION}"
+WKS_FILE = "overlayfs_etc.wks.in"
+
+EXTRA_IMAGE_FEATURES += "read-only-rootfs"
+# Image configuration for overlayfs-etc
+EXTRA_IMAGE_FEATURES += "overlayfs-etc"
+IMAGE_FEATURES:remove = "package-management"
+OVERLAYFS_ETC_MOUNT_POINT = "/data"
+OVERLAYFS_ETC_FSTYPE = "ext4"
+OVERLAYFS_ETC_DEVICE = "/dev/sda3"
+OVERLAYFS_ETC_USE_ORIG_INIT_NAME = "{OVERLAYFS_ETC_USE_ORIG_INIT_NAME}"
+"""
+
+ args = {
+ 'OVERLAYFS_INIT_OPTION': "" if origInit else "init=/sbin/preinit",
+ 'OVERLAYFS_ETC_USE_ORIG_INIT_NAME': int(origInit == True)
+ }
+
+ self.write_config(config.format(**args))
+
+ bitbake('core-image-minimal')
+ testFile = "/etc/my-test-data"
+
+ with runqemu('core-image-minimal', image_fstype='wic', discard_writes=False) as qemu:
+ status, output = qemu.run_serial("/bin/mount")
+
+ line = getline_qemu(output, "/dev/sda3")
+ self.assertTrue("/data" in output, msg=output)
+
+ line = getline_qemu(output, "upperdir=/data/overlay-etc/upper")
+ self.assertTrue(line and line.startswith("/data/overlay-etc/upper on /etc type overlay"), msg=output)
+
+ status, output = qemu.run_serial("touch " + testFile)
+ status, output = qemu.run_serial("sync")
+ status, output = qemu.run_serial("ls -1 " + testFile)
+ line = getline_qemu(output, testFile)
+ self.assertTrue(line and line.startswith(testFile), msg=output)
+
+ # Check that file exists in /etc after reboot
+ with runqemu('core-image-minimal', image_fstype='wic') as qemu:
+ status, output = qemu.run_serial("ls -1 " + testFile)
+ line = getline_qemu(output, testFile)
+ self.assertTrue(line and line.startswith(testFile), msg=output)
--
2.28.0
^ permalink raw reply related [flat|nested] 11+ messages in thread