* How handle files needing updates in read-only filesystem @ 2018-06-09 9:50 Ulf Samuelsson 2018-06-09 23:36 ` Peter Kjellerstedt 0 siblings, 1 reply; 7+ messages in thread From: Ulf Samuelsson @ 2018-06-09 9:50 UTC (permalink / raw) To: yocto We have a rootfs which is read-only. For files that needs to be updated, like "/etc/localtime" we will create a symbolic link to a partition (/persistent) which is retained between rootfs image updates. /etc/localtime -> /persistent/etc/localtime The original recipe will generate the real /etc/localtime of course. To support the file beeing updated, the following needs to be done in a rootfs postprocess. mv /etc/localtime /update/etc/localtime ln -sf /persistent/etc/localtime /etc/localtime install -m 0644 /update/etc/localtime /persistent/etc/localtime chown <user>:<group> /persistent/etc/localtime The last two commands needs to be executed on the running system, since /persistent is not part of the rootfs. To ensure that the file can be updated is an ad-hoc activity for each affected file. It would be better if there was a class where you declare a file to be writeable, and then a post process would move the file to /update (or similar) and create a symbolic link SETTINGS ?= "/persistent" ============================= inherit writeable WRITEABLE = "/etc/localtime" ============================= This would generate the first two commands for each listed file. mv /etc/localtime /update/etc/localtime ln -sf ${SETTINGS}/etc/localtime /etc/localtime The file would also be added to a file indicating that it may need to be copied to the ${SETTINGS} Is it a good solution to modify "populate_volatiles.sh" to support a second file parameter for the f (file) command in the /etc/default/volatiles file? Today the file create command looks like: f <mode> <uid> <gid> <filename> <ignored> An idea would be to have the following syntax: f <mode> <uid> <gid> <filename> <source> If <source> is a valid filepath, then copy this file to <filename> If <source> is not a valid filepath, create <filename> using "touch" This would break any build which has a "funny" volatiles file. Otherwise a script called populate_persistent.sh could be created with such an extension. Comments? BR Ulf Samuelsson ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-09 9:50 How handle files needing updates in read-only filesystem Ulf Samuelsson @ 2018-06-09 23:36 ` Peter Kjellerstedt 2018-06-10 3:30 ` Ulf Samuelsson 0 siblings, 1 reply; 7+ messages in thread From: Peter Kjellerstedt @ 2018-06-09 23:36 UTC (permalink / raw) To: Ulf Samuelsson, yocto > -----Original Message----- > From: yocto-bounces@yoctoproject.org [mailto:yocto- > bounces@yoctoproject.org] On Behalf Of Ulf Samuelsson > Sent: den 9 juni 2018 11:51 > To: yocto@yoctoproject.org > Subject: [yocto] How handle files needing updates in read-only > filesystem > > We have a rootfs which is read-only. > For files that needs to be updated, like "/etc/localtime" > we will create a symbolic link to a partition (/persistent) which > is retained between rootfs image updates. > > /etc/localtime -> /persistent/etc/localtime > > The original recipe will generate the real /etc/localtime of course. > > To support the file beeing updated, the following needs to be done in a > rootfs postprocess. > > mv /etc/localtime /update/etc/localtime > ln -sf /persistent/etc/localtime /etc/localtime > install -m 0644 /update/etc/localtime /persistent/etc/localtime > chown <user>:<group> /persistent/etc/localtime > > The last two commands needs to be executed on the running system, > since /persistent is not part of the rootfs. > > To ensure that the file can be updated is an ad-hoc activity for each > affected file. It would be better if there was a class where you > declare > a file to be writeable, and then a post process > would move the file to /update (or similar) and create a symbolic link > > SETTINGS ?= "/persistent" > ============================= > inherit writeable > > WRITEABLE = "/etc/localtime" > ============================= > > This would generate the first two commands for each listed file. > > mv /etc/localtime /update/etc/localtime > ln -sf ${SETTINGS}/etc/localtime /etc/localtime > > The file would also be added to a file indicating that it may need > to be copied to the ${SETTINGS} > > Is it a good solution to modify "populate_volatiles.sh" > to support a second file parameter for the f (file) command > in the /etc/default/volatiles file? > > Today the file create command looks like: > f <mode> <uid> <gid> <filename> <ignored> > > An idea would be to have the following syntax: > f <mode> <uid> <gid> <filename> <source> > > If <source> is a valid filepath, then copy this file to <filename> > If <source> is not a valid filepath, create <filename> using "touch" > > This would break any build which has a "funny" volatiles file. > Otherwise a script called populate_persistent.sh could be > created with such an extension. > > Comments? > > BR > Ulf Samuelsson Since a lot of files in /etc typically need to be writable, one way to handle this is by using an overlayfs for /etc. That is what we do and it works very well. An alternative, more along your suggestion, is to use bind mounts. The advantage of using a bind mount instead of a symbolic link is that it will look as a normal file. You should look into the volatile-binds recipe and the VOLATILE_BINDS variable. It is used to handle directories that need to be writable. It will create systemd service files that copies the non-volatile directory to the volatile directory (if it does not already exists), and the bind mounts it back. I am not sure volatile-binds works out-of-the-box for files, but I believe it does. If not, it should not be too hard to modify it so that it can handle files as well. //Peter ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-09 23:36 ` Peter Kjellerstedt @ 2018-06-10 3:30 ` Ulf Samuelsson 2018-06-13 13:20 ` Anders Darander 0 siblings, 1 reply; 7+ messages in thread From: Ulf Samuelsson @ 2018-06-10 3:30 UTC (permalink / raw) To: Peter Kjellerstedt; +Cc: yocto Thanks, We want most of /etc to be read-only for security reasons, and the overlayfs will make the whole of /etc writeable. I tried mount —bind /etc/timezone /persistent/etc/timezone, and it complained that they were not directories. Bind mounting /etc again will make all of /etc writeable. Symlinking to /persistent is fine, so the question is what an acceptable method is to have a simple way of ensuring that a certain file is converted to that symlink. Best Regards, Ulf Samuelsson +46 722 427 437 10 juni 2018 kl. 01:36 skrev Peter Kjellerstedt <peter.kjellerstedt@axis.com>: >> -----Original Message----- >> From: yocto-bounces@yoctoproject.org [mailto:yocto- >> bounces@yoctoproject.org] On Behalf Of Ulf Samuelsson >> Sent: den 9 juni 2018 11:51 >> To: yocto@yoctoproject.org >> Subject: [yocto] How handle files needing updates in read-only >> filesystem >> >> We have a rootfs which is read-only. >> For files that needs to be updated, like "/etc/localtime" >> we will create a symbolic link to a partition (/persistent) which >> is retained between rootfs image updates. >> >> /etc/localtime -> /persistent/etc/localtime >> >> The original recipe will generate the real /etc/localtime of course. >> >> To support the file beeing updated, the following needs to be done in a >> rootfs postprocess. >> >> mv /etc/localtime /update/etc/localtime >> ln -sf /persistent/etc/localtime /etc/localtime >> install -m 0644 /update/etc/localtime /persistent/etc/localtime >> chown <user>:<group> /persistent/etc/localtime >> >> The last two commands needs to be executed on the running system, >> since /persistent is not part of the rootfs. >> >> To ensure that the file can be updated is an ad-hoc activity for each >> affected file. It would be better if there was a class where you >> declare >> a file to be writeable, and then a post process >> would move the file to /update (or similar) and create a symbolic link >> >> SETTINGS ?= "/persistent" >> ============================= >> inherit writeable >> >> WRITEABLE = "/etc/localtime" >> ============================= >> >> This would generate the first two commands for each listed file. >> >> mv /etc/localtime /update/etc/localtime >> ln -sf ${SETTINGS}/etc/localtime /etc/localtime >> >> The file would also be added to a file indicating that it may need >> to be copied to the ${SETTINGS} >> >> Is it a good solution to modify "populate_volatiles.sh" >> to support a second file parameter for the f (file) command >> in the /etc/default/volatiles file? >> >> Today the file create command looks like: >> f <mode> <uid> <gid> <filename> <ignored> >> >> An idea would be to have the following syntax: >> f <mode> <uid> <gid> <filename> <source> >> >> If <source> is a valid filepath, then copy this file to <filename> >> If <source> is not a valid filepath, create <filename> using "touch" >> >> This would break any build which has a "funny" volatiles file. >> Otherwise a script called populate_persistent.sh could be >> created with such an extension. >> >> Comments? >> >> BR >> Ulf Samuelsson > > Since a lot of files in /etc typically need to be writable, one way to > handle this is by using an overlayfs for /etc. That is what we do and > it works very well. > > An alternative, more along your suggestion, is to use bind mounts. The > advantage of using a bind mount instead of a symbolic link is that it > will look as a normal file. > > You should look into the volatile-binds recipe and the VOLATILE_BINDS > variable. It is used to handle directories that need to be writable. > It will create systemd service files that copies the non-volatile > directory to the volatile directory (if it does not already exists), > and the bind mounts it back. > > I am not sure volatile-binds works out-of-the-box for files, but I > believe it does. If not, it should not be too hard to modify it so > that it can handle files as well. > > //Peter > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-10 3:30 ` Ulf Samuelsson @ 2018-06-13 13:20 ` Anders Darander 2018-06-13 16:28 ` Ulf Samuelsson 0 siblings, 1 reply; 7+ messages in thread From: Anders Darander @ 2018-06-13 13:20 UTC (permalink / raw) To: Ulf Samuelsson; +Cc: yocto * Ulf Samuelsson <yocto@emagii.com> [180612 22:01]: > We want most of /etc to be read-only for security reasons, > and the overlayfs will make the whole of /etc writeable. > I tried mount —bind /etc/timezone /persistent/etc/timezone, and it > complained that they were not directories. Bind mounting /etc again > will make all of /etc writeable. Try to use: mount —o bind /etc/timezone /persistent/etc/timezone I'm using that heavily, either manually or by the volatile-binds recipe. It works perfectly fine with files. > Symlinking to /persistent is fine, so the question is what an > acceptable method is to have a simple way of ensuring that a certain > file is converted to that symlink. This is normally done by a manual inspection / addition of bbappend file. Cheers, Anders -- Anders Darander, Senior System Architect ChargeStorm AB / eStorm AB ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-13 13:20 ` Anders Darander @ 2018-06-13 16:28 ` Ulf Samuelsson 2018-06-13 23:02 ` Andre McCurdy 0 siblings, 1 reply; 7+ messages in thread From: Ulf Samuelsson @ 2018-06-13 16:28 UTC (permalink / raw) To: Anders Darander; +Cc: yocto Thanks, is it more efficient than symlinking? Best Regards, Ulf Samuelsson > 13 juni 2018 kl. 15:20 skrev Anders Darander <anders@chargestorm.se>: > > * Ulf Samuelsson <yocto@emagii.com> [180612 22:01]: > >> We want most of /etc to be read-only for security reasons, >> and the overlayfs will make the whole of /etc writeable. > >> I tried mount —bind /etc/timezone /persistent/etc/timezone, and it >> complained that they were not directories. Bind mounting /etc again >> will make all of /etc writeable. > > Try to use: mount —o bind /etc/timezone /persistent/etc/timezone > > I'm using that heavily, either manually or by the volatile-binds recipe. > It works perfectly fine with files. > >> Symlinking to /persistent is fine, so the question is what an >> acceptable method is to have a simple way of ensuring that a certain >> file is converted to that symlink. > > This is normally done by a manual inspection / addition of bbappend > file. > > Cheers, > Anders > -- > Anders Darander, Senior System Architect > ChargeStorm AB / eStorm AB ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-13 16:28 ` Ulf Samuelsson @ 2018-06-13 23:02 ` Andre McCurdy 2018-06-14 21:02 ` Ulf Samuelsson 0 siblings, 1 reply; 7+ messages in thread From: Andre McCurdy @ 2018-06-13 23:02 UTC (permalink / raw) To: Ulf Samuelsson; +Cc: Anders Darander, yocto On Wed, Jun 13, 2018 at 9:28 AM, Ulf Samuelsson <yocto@emagii.com> wrote: > Thanks, is it more efficient than symlinking? Efficient in what way? Have you looked at the volatile-binds recipe in oe-core? Its job is to setup enough bind mounts to enable systemd to run from a readonly rootfs. Although it's currently systemd specific (it only provides a systemd service file, no init script) it might give you some clues about how to setup bind mounts at boot time. > Best Regards, > Ulf Samuelsson > >> 13 juni 2018 kl. 15:20 skrev Anders Darander <anders@chargestorm.se>: >> >> * Ulf Samuelsson <yocto@emagii.com> [180612 22:01]: >> >>> We want most of /etc to be read-only for security reasons, >>> and the overlayfs will make the whole of /etc writeable. >> >>> I tried mount —bind /etc/timezone /persistent/etc/timezone, and it >>> complained that they were not directories. Bind mounting /etc again >>> will make all of /etc writeable. >> >> Try to use: mount —o bind /etc/timezone /persistent/etc/timezone >> >> I'm using that heavily, either manually or by the volatile-binds recipe. >> It works perfectly fine with files. >> >>> Symlinking to /persistent is fine, so the question is what an >>> acceptable method is to have a simple way of ensuring that a certain >>> file is converted to that symlink. >> >> This is normally done by a manual inspection / addition of bbappend >> file. >> >> Cheers, >> Anders >> -- >> Anders Darander, Senior System Architect >> ChargeStorm AB / eStorm AB > > -- > _______________________________________________ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: How handle files needing updates in read-only filesystem 2018-06-13 23:02 ` Andre McCurdy @ 2018-06-14 21:02 ` Ulf Samuelsson 0 siblings, 0 replies; 7+ messages in thread From: Ulf Samuelsson @ 2018-06-14 21:02 UTC (permalink / raw) To: Andre McCurdy; +Cc: Anders Darander, yocto I looked at the populate-volatile.sh, and this seemed to almost do the job, if I solve the problem using symlinks. What it needs is a copy file function. As a temporary solution, I created a derivative: populate-settings.sh which will check /etc/default/settings in the same way populate-volatile.sh checks /etc/default/volatiles. It also support copying a file, if the copy target does not exist. What I have right now is a ”writeable.bbclass” To make a file located in a read-only location, I just inherit writeable and declare it writeable in a bbappend. inherit writeable WRITEABLE = ”/etc/localtime” At build time, the ”/etc/localtime” is moved to ”/etc/update/localtime”, and ”/etc/localtime” becomes a symlink to ”/persistent/localtime” (a leading ”/etc” is shaved off) An entry to copy ”/etc/update/localtime” to “/persistent/localtime” is created in “/etc/default/settings/99_tzdata” When “/etc/init.d/populate-settings.sh” is run, “/persistent/localtime” is created. This works. Efficiency is on several levels. I am looking for a solution, where I, like above, only need to declare the name of the file. I want to avoid solutions, where I manually have to add symlinks etc. Once the symlink/bind mount is accessed, it should not eat up the CPU cycles, Kno Best Regards, Ulf Samuelsson > 14 juni 2018 kl. 01:02 skrev Andre McCurdy <armccurdy@gmail.com>: > >> On Wed, Jun 13, 2018 at 9:28 AM, Ulf Samuelsson <yocto@emagii.com> wrote: >> Thanks, is it more efficient than symlinking? > > Efficient in what way? > > Have you looked at the volatile-binds recipe in oe-core? Its job is to > setup enough bind mounts to enable systemd to run from a readonly > rootfs. Although it's currently systemd specific (it only provides a > systemd service file, no init script) it might give you some clues > about how to setup bind mounts at boot time. > >> Best Regards, >> Ulf Samuelsson >> >>> 13 juni 2018 kl. 15:20 skrev Anders Darander <anders@chargestorm.se>: >>> >>> * Ulf Samuelsson <yocto@emagii.com> [180612 22:01]: >>> >>>> We want most of /etc to be read-only for security reasons, >>>> and the overlayfs will make the whole of /etc writeable. >>> >>>> I tried mount —bind /etc/timezone /persistent/etc/timezone, and it >>>> complained that they were not directories. Bind mounting /etc again >>>> will make all of /etc writeable. >>> >>> Try to use: mount —o bind /etc/timezone /persistent/etc/timezone >>> >>> I'm using that heavily, either manually or by the volatile-binds recipe. >>> It works perfectly fine with files. >>> >>>> Symlinking to /persistent is fine, so the question is what an >>>> acceptable method is to have a simple way of ensuring that a certain >>>> file is converted to that symlink. >>> >>> This is normally done by a manual inspection / addition of bbappend >>> file. >>> >>> Cheers, >>> Anders >>> -- >>> Anders Darander, Senior System Architect >>> ChargeStorm AB / eStorm AB >> >> -- >> _______________________________________________ >> yocto mailing list >> yocto@yoctoproject.org >> https://lists.yoctoproject.org/listinfo/yocto ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-06-14 21:02 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-06-09 9:50 How handle files needing updates in read-only filesystem Ulf Samuelsson 2018-06-09 23:36 ` Peter Kjellerstedt 2018-06-10 3:30 ` Ulf Samuelsson 2018-06-13 13:20 ` Anders Darander 2018-06-13 16:28 ` Ulf Samuelsson 2018-06-13 23:02 ` Andre McCurdy 2018-06-14 21:02 ` Ulf Samuelsson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.