Re: [OE-core] [PATCH] overlayfs.bbclass: generate overlayfs mount units

["Vyacheslav Yurkov" <uvv.mail@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 7357 bytes --]

Hi Ayoub,
Thanks for the hint. I knew the recipe exists, but I didn't know it uses 
overlay.

I did have a closer look at the mount-copybind script, and it uses 
overlay indeed, but it seems it's used for the other purpose. Originally 
the recipe provided only bind-mount functionality, they added overlayfs 
support later to save on space.

I tried to use the recipe with my use case, but it turns out that 
reverses the options of mount command in mount-copybind. I'm also not 
sure how the dependencies should be injected into the mount unit.

Vyacheslav

On 04.06.2021 22:20, Ayoub Zaki wrote:
>
> Hi,
>
>
> volatile-binds (maybe the *volatile* naming is misleading) recipe in 
> yocto is already doing that, just to be appended .ie:
>
> VOLATILE_BINDS += "\
> /data/lib/systemd/network /lib/systemd/network\n\
> /data/home/root /home/root\n\
> "
>
> It will generate for each overlay a mount point service.
>
> BR
>
> On 6/3/21 4:21 PM, Vyacheslav Yurkov wrote:
>> It's often desired in Embedded System design to have a read-only rootfs.
>> But a lot of different applications might want to have a read-write access
>> to some parts of a filesystem. It can be especially useful when your update
>> mechanism overwrites the whole rootfs, but you want your application data
>> to be preserved between updates. This class provides a way to achieve that
>> by means of overlayfs and at the same time keeping the base rootfs read-only.
>>
>> Signed-off-by: Vyacheslav Yurkov<uvv.mail@gmail.com>
>> ---
>>   meta/classes/overlayfs.bbclass | 132 +++++++++++++++++++++++++++++++++
>>   1 file changed, 132 insertions(+)
>>   create mode 100644 meta/classes/overlayfs.bbclass
>>
>> diff --git a/meta/classes/overlayfs.bbclass b/meta/classes/overlayfs.bbclass
>> new file mode 100644
>> index 0000000000..7fac3e696e
>> --- /dev/null
>> +++ b/meta/classes/overlayfs.bbclass
>> @@ -0,0 +1,132 @@
>> +# Class for generation of overlayfs mount units
>> +#
>> +# It's often desired in Embedded System design to have a read-only rootfs.
>> +# But a lot of different applications might want to have a read-write access to
>> +# some parts of a filesystem. It can be especially useful when your update mechanism
>> +# overwrites the whole rootfs, but you want your application data to be preserved
>> +# between updates. This class provides a way to achieve that by means
>> +# of overlayfs and at the same time keeping the base rootfs read-only.
>> +#
>> +# Usage example.
>> +#
>> +# Set a mount point for a partition overlayfs is going to use as upper layer
>> +# in your machine configuration. Underlying file system can be anything that
>> +# is supported by overlayfs
>> +#
>> +#   OVERLAYFS_MOUNT_POINT[data] ?= "/data"
>> +#
>> +# The class assumes you have a data.mount systemd unit defined elsewhere in your
>> +# BSP and installed to the image.
>> +#
>> +# Then you can specify writable directories on a recipe base
>> +#
>> +#   OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application"
>> +#
>> +# To support several mount points you can use a different variable flag. Assume we
>> +# what to have a writable location on the file system, but not interested where the data
>> +# survive a reboot. The we could have a mnt-overlay.mount unit for a tmpfs file system:
>> +#
>> +#   OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay"
>> +#   OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application"
>> +
>> +OVERLAYFS_WRITABLE_PATHS[data] ?= ""
>> +
>> +inherit systemd
>> +
>> +def strForBash(s):
>> +    return s.replace('\\', '\\\\')
>> +
>> +def unitFileList(d):
>> +    fileList = []
>> +    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
>> +    for mountPoint in overlayMountPoints:
>> +        for path in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
>> +            fileList.append(mountUnitName(path))
>> +            fileList.append(helperUnitName(path))
>> +
>> +    return fileList
>> +
>> +# this function is based onhttps://github.com/systemd/systemd/blob/main/src/basic/unit-name.c
>> +def escapeSystemdUnitName(path):
>> +    escapeMap = {
>> +        '/': '-',
>> +        '-': "\\x2d",
>> +        '\\': "\\x5d"
>> +    }
>> +    return "".join([escapeMap.get(c, c) for c in path.strip('/')])
>> +
>> +def mountUnitName(unit):
>> +    return escapeSystemdUnitName(unit) + '.mount'
>> +
>> +def helperUnitName(unit):
>> +    return escapeSystemdUnitName(unit) + '-create-upper-dir.service'
>> +
>> +python do_create_overlayfs_units() {
>> +    CreateDirsUnitTemplate = """[Unit]
>> +Description=Overlayfs directories setup
>> +Requires={DATA_MOUNT_UNIT}
>> +After={DATA_MOUNT_UNIT}
>> +DefaultDependencies=no
>> +
>> +[Service]
>> +Type=oneshot
>> +ExecStart=mkdir -p {DATA_MOUNT_POINT}/workdir{LOWERDIR} && mkdir -p {DATA_MOUNT_POINT}/upper{LOWERDIR}
>> +RemainAfterExit=true
>> +StandardOutput=journal
>> +
>> +[Install]
>> +WantedBy=multi-user.target
>> +"""
>> +    MountUnitTemplate = """[Unit]
>> +Description=Overlayfs mount unit
>> +Requires={CREATE_DIRS_SERVICE}
>> +After={CREATE_DIRS_SERVICE}
>> +
>> +[Mount]
>> +What=overlay
>> +Where={LOWERDIR}
>> +Type=overlay
>> +Options=lowerdir={LOWERDIR},upperdir={DATA_MOUNT_POINT}/upper{LOWERDIR},workdir={DATA_MOUNT_POINT}/workdir{LOWERDIR}
>> +
>> +[Install]
>> +WantedBy=multi-user.target
>> +"""
>> +
>> +    def prepareUnits(data, lower):
>> +        args = {
>> +            'DATA_MOUNT_POINT': data,
>> +            'DATA_MOUNT_UNIT': mountUnitName(data),
>> +            'CREATE_DIRS_SERVICE': helperUnitName(lower),
>> +            'LOWERDIR': lower,
>> +        }
>> +
>> +        with open(os.path.join(d.getVar('WORKDIR'), mountUnitName(lower)), 'w') as f:
>> +            f.write(MountUnitTemplate.format(**args))
>> +
>> +        with open(os.path.join(d.getVar('WORKDIR'), helperUnitName(lower)), 'w') as f:
>> +            f.write(CreateDirsUnitTemplate.format(**args))
>> +
>> +    overlayMountPoints = d.getVarFlags("OVERLAYFS_MOUNT_POINT")
>> +    for mountPoint in overlayMountPoints:
>> +        for lower in d.getVarFlag('OVERLAYFS_WRITABLE_PATHS', mountPoint).split():
>> +            prepareUnits(d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint), lower)
>> +}
>> +
>> +# we need to generate file names early during parsing stage
>> +python () {
>> +    unitList = unitFileList(d)
>> +    for unit in unitList:
>> +        d.appendVar('SYSTEMD_SERVICE_' + d.getVar('PN'), ' ' + unit);
>> +        d.appendVar('FILES_' + d.getVar('PN'), strForBash(unit))
>> +
>> +    d.setVar('OVERLAYFS_UNIT_LIST', ' '.join([strForBash(s) for s in unitList]))
>> +}
>> +
>> +do_install_append() {
>> +    install -d ${D}${systemd_system_unitdir}
>> +    for unit in ${OVERLAYFS_UNIT_LIST}; do
>> +        install -m 0444 ${WORKDIR}/${unit} ${D}${systemd_system_unitdir}
>> +    done
>> +}
>> +
>> +addtask create_overlayfs_units before do_install
>>
> Mit freundlichen Grüßen / Kind regards
>
> -- 
> Ayoub Zaki
> Embedded Systems Consultant
>
> Vaihinger Straße 2/1
> D-71634 Ludwigsburg
>
> Mobile   : +4917662901545
> Email    :ayoub.zaki@embexus.com
> Homepage :https://embexus.com
> VAT No.  : DE313902634
>
> 
>


[-- Attachment #2: Type: text/html, Size: 8686 bytes --]