All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Adrian Freihofer" <adrian.freihofer@gmail.com>
To: Vyacheslav Yurkov <uvv.mail@gmail.com>,
	 Openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH] overlayfs.bbclass: generate overlayfs mount units
Date: Sat, 05 Jun 2021 09:37:23 +0200	[thread overview]
Message-ID: <64e51c69669c3e08da50789f22560f3f7857492b.camel@gmail.com> (raw)
In-Reply-To: <20210603142106.115932-1-uvv.mail@gmail.com>

Hi Vyacheslav

It would be very nice to have such a feature in Yocto.

If I remember correctly I tried to do that once with an overlay on
/etc. However, that did not work with systemd. Did you ever try to add
/etc as a mount point?

On Thu, 2021-06-03 at 16:21 +0200, Vyacheslav Yurkov wrote:
> It's often desired in Embedded System design to have a read-only rootfs.
> But a lot of different applications might want to have a read-write access
> to some parts of a filesystem. It can be especially useful when your update
> mechanism overwrites the whole rootfs, but you want your application data
> to be preserved between updates. This class provides a way to achieve that
> by means of overlayfs and at the same time keeping the base rootfs read-only.
> 
> Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
> ---
>  meta/classes/overlayfs.bbclass | 132 +++++++++++++++++++++++++++++++++
>  1 file changed, 132 insertions(+)
>  create mode 100644 meta/classes/overlayfs.bbclass
> 
> diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
> new file mode 100644
> index 0000000000..7fac3e696e
> --- /dev/null
> +++ b/meta/classes/overlayfs.bbclass
> @@ -0,0 +1,132 @@
> +# Class for generation of overlayfs mount units
> +#
> +# It's often desired in Embedded System design to have a read-only rootfs.
> +# But a lot of different applications might want to have a read-write access to
> +# some parts of a filesystem. It can be especially useful when your update mechanism
> +# overwrites the whole rootfs, but you want your application data to be preserved
> +# between updates. This class provides a way to achieve that by means
> +# of overlayfs and at the same time keeping the base rootfs read-only.
> +#
> +# Usage example.
> +#
> +# Set a mount point for a partition overlayfs is going to use as upper layer
> +# in your machine configuration. Underlying file system can be anything that
> +# is supported by overlayfs
> +#
> +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
> +#
> +# The class assumes you have a data.mount systemd unit defined elsewhere in your
> +# BSP and installed to the image.
> +#
> +# Then you can specify writable directories on a recipe base
> +#
> +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
> +#
> +# To support several mount points you can use a different variable flag. Assume we
> +# what to have a writable location on the file system, but not interested where the data
> +# survive a reboot. The we could have a mnt-overlay.mount unit for a tmpfs file system:
> +#
> +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
> +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
> +
> +OVERLAYFS_WRITABLE_PATHS[data] ?= ""
> +
> +inherit systemd
> +
> +def strForBash(s):
> +    return s.replace('\\', '\\\\')
> +
> +def unitFileList(d):
> +    fileList = []
> +    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
> +    for mountPoint in overlayMountPoints:
> +        for path in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
> +            fileList.append(mountUnitName(path))
> +            fileList.append(helperUnitName(path))
> +
> +    return fileList
> +
> +# this function is based on https://github.com/systemd/systemd/blob/main/src/basic/unit-name.c
> +def escapeSystemdUnitName(path):
> +    escapeMap = {
> +        '/': '-',
> +        '-': "\\x2d",
> +        '\\': "\\x5d"
> +    }
> +    return "".join([escapeMap.get(c, c) for c in path.strip('/')])
> +
> +def mountUnitName(unit):
> +    return escapeSystemdUnitName(unit) + '.mount'
> +
> +def helperUnitName(unit):
> +    return escapeSystemdUnitName(unit) + '-create-upper-dir.service'
> +
> +python do_create_overlayfs_units() {
> +    CreateDirsUnitTemplate = """[Unit]
> +Description=Overlayfs directories setup
> +Requires={DATA_MOUNT_UNIT}
> +After={DATA_MOUNT_UNIT}
> +DefaultDependencies=no
> +
> +[Service]
> +Type=oneshot
> +ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p {DATA_MOUNT_POINT}/upper{LOWERDIR}
> +RemainAfterExit=true
> +StandardOutput=journal
> +
> +[Install]
> +WantedBy=multi-user.target
The mounting of overlays should probably happen very early at boot,
before services start using the desired folders.

Regards,
Adrian

> +"""
> +    MountUnitTemplate = """[Unit]
> +Description=Overlayfs mount unit
> +Requires={CREATE_DIRS_SERVICE}
> +After={CREATE_DIRS_SERVICE}
> +
> +[Mount]
> +What=overlay
> +Where={LOWERDIR}
> +Type=overlay
> +Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
> +
> +[Install]
> +WantedBy=multi-user.target
> +"""
> +
> +    def prepareUnits(data, lower):
> +        args = {
> +            'DATA_MOUNT_POINT': data,
> +            'DATA_MOUNT_UNIT': mountUnitName(data),
> +            'CREATE_DIRS_SERVICE': helperUnitName(lower),
> +            'LOWERDIR': lower,
> +        }
> +
> +        with open(os.path.join(d.getVar('WORKDIR'), mountUnitName(lower)), 'w') as f:
> +            f.write(MountUnitTemplate.format(**args))
> +
> +        with open(os.path.join(d.getVar('WORKDIR'), helperUnitName(lower)), 'w') as f:
> +            f.write(CreateDirsUnitTemplate.format(**args))
> +
> +    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
> +    for mountPoint in overlayMountPoints:
> +        for lower in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
> +            prepareUnits(d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint), lower)
> +}
> +
> +# we need to generate file names early during parsing stage
> +python () {
> +    unitList = unitFileList(d)
> +    for unit in unitList:
> +        d.appendVar('SYSTEMD_SERVICE_' + d.getVar('PN'), ' ' + unit);
> +        d.appendVar('FILES_' + d.getVar('PN'), strForBash(unit))
> +
> +    d.setVar('OVERLAYFS_UNIT_LIST', ' '.join([strForBash(s) for s in unitList]))
> +}
> +
> +do_install_append() {
> +    install -d ${D}${systemd_system_unitdir}
> +    for unit in ${OVERLAYFS_UNIT_LIST}; do
> +        install -m 0444 ${WORKDIR}/${unit} ${D}${systemd_system_unitdir}
> +    done
> +}
> +
> +addtask create_overlayfs_units before do_install
> 
> 



  parent reply	other threads:[~2021-06-05  7:37 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-03 14:21 [PATCH] overlayfs.bbclass: generate overlayfs mount units Vyacheslav Yurkov
2021-06-03 14:35 ` [OE-core] " Bruce Ashfield
2021-06-04 17:22   ` Vyacheslav Yurkov
2021-06-04 18:24     ` Bruce Ashfield
2021-06-04 20:20 ` Ayoub Zaki
2021-06-05  6:31   ` Vyacheslav Yurkov
2021-06-05  7:37 ` Adrian Freihofer [this message]
2021-06-05  9:28   ` Vyacheslav Yurkov
2021-06-09 13:56 ` Andrei Gherzan
2021-06-09 19:04   ` Vyacheslav Yurkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=64e51c69669c3e08da50789f22560f3f7857492b.camel@gmail.com \
    --to=adrian.freihofer@gmail.com \
    --cc=Openembedded-core@lists.openembedded.org \
    --cc=uvv.mail@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.