* [PATCH] ref-manual: add overlayfs class @ 2021-08-14 4:28 Vyacheslav Yurkov 2021-08-16 9:14 ` [docs] " Quentin Schulz 0 siblings, 1 reply; 5+ messages in thread From: Vyacheslav Yurkov @ 2021-08-14 4:28 UTC (permalink / raw) To: docs; +Cc: Vyacheslav Yurkov Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> --- documentation/ref-manual/classes.rst | 46 ++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index a98a64c43..5d33e81a1 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -1710,6 +1710,52 @@ one such example. However, being aware of this class can reduce the proliferation of different versions of similar classes across multiple layers. +.. _ref-classes-overlayfs: + +``overlayfs.bbclass`` +======================= + + +It's often desired in Embedded System design to have a read-only rootfs. +But a lot of different applications might want to have a read-write access to +some parts of a filesystem. It can be especially useful when your update mechanism +overwrites the whole rootfs, but you want your application data to be preserved +between updates. The ``overlayfs`` class provides a way to achieve that by means +of overlayfs and at the same time keeping the base rootfs read-only. + +To use this class, set a mount point for a partition overlayfs is going to use as upper +layer in your machine configuration. Underlying file system can be anything that +is supported by overlayfs. This has to be done in your machine configuration:: + + OVERLAYFS_MOUNT_POINT[data] = "/data" + +.. note:: + + QA check fails to catch file existence if you redefine this variable in your recipe! + +The class assumes you have a data.mount systemd unit defined elsewhere in your BSP +(e.g. in systemd-machine-units recipe) and it's installed to the image. + +Then you can specify writable directories on a recipe basis:: + + OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application" + +To support several mount points you can use a different variable flag. Assume we +want to have a writable location on the file system, but not interested where the data +survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs file system. + +In your machine configuration:: + + OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay" + +and then in your recipe:: + + OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application" + +.. note:: + + The class does not support /etc directory itself, because systemd depends on it + .. _ref-classes-own-mirrors: ``own-mirrors.bbclass`` -- 2.28.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [docs] [PATCH] ref-manual: add overlayfs class 2021-08-14 4:28 [PATCH] ref-manual: add overlayfs class Vyacheslav Yurkov @ 2021-08-16 9:14 ` Quentin Schulz 2021-08-17 3:29 ` Vyacheslav Yurkov 0 siblings, 1 reply; 5+ messages in thread From: Quentin Schulz @ 2021-08-16 9:14 UTC (permalink / raw) To: Vyacheslav Yurkov; +Cc: docs Hi Vyacheslav, Thanks for the patch! On Sat, Aug 14, 2021 at 06:28:25AM +0200, Vyacheslav Yurkov wrote: > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> > --- > documentation/ref-manual/classes.rst | 46 ++++++++++++++++++++++++++++ > 1 file changed, 46 insertions(+) > > diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst > index a98a64c43..5d33e81a1 100644 > --- a/documentation/ref-manual/classes.rst > +++ b/documentation/ref-manual/classes.rst > @@ -1710,6 +1710,52 @@ one such example. However, being aware of this class can reduce the > proliferation of different versions of similar classes across multiple > layers. > > +.. _ref-classes-overlayfs: > + > +``overlayfs.bbclass`` > +======================= > + > + One newline is enough :) > +It's often desired in Embedded System design to have a read-only rootfs. > +But a lot of different applications might want to have a read-write access to > +some parts of a filesystem. It can be especially useful when your update mechanism > +overwrites the whole rootfs, but you want your application data to be preserved > +between updates. The ``overlayfs`` class provides a way to achieve that by means s/``overlayfs``/:ref:`overlayfs <ref-classes-overlayfs>`/ > +of overlayfs and at the same time keeping the base rootfs read-only. > + I would highlight overlayfs with tick quotes (``overlayfs``). > +To use this class, set a mount point for a partition overlayfs is going to use as upper > +layer in your machine configuration. Underlying file system can be anything that > +is supported by overlayfs. This has to be done in your machine configuration:: > + How do I know what's supported by overlayfs? Where can I find this important piece of information? > + OVERLAYFS_MOUNT_POINT[data] = "/data" > + > +.. note:: > + > + QA check fails to catch file existence if you redefine this variable in your recipe! > + What about: This variable must only be set from a configuration file as QA check will fail to catch file existence if this variable is also set in a recipe. Also, I'm not sure to understand what "catch file existence" means? Could you explain to me please :) > +The class assumes you have a data.mount systemd unit defined elsewhere in your BSP s/data.mount/``data.mount``/ > +(e.g. in systemd-machine-units recipe) and it's installed to the image. > + s/systemd-machine-units/``systemd-machine-units``/ > +Then you can specify writable directories on a recipe basis:: > + If package recipes are supported, does it actually make sense to have such variable in there instead of in image recipes? Just trying to know if we should highlight whether it's a bad idea for users to do so. > + OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application" > + > +To support several mount points you can use a different variable flag. Assume we s/Assume/Assuming/ ? > +want to have a writable location on the file system, but not interested where the data > +survive a reboot. Then we could have a mnt-overlay.mount unit for a tmpfs file system. > + s/mnt-overlay.mount/``mnt-overlay.mount``/ s/tmpfs/``tmpfs``/ ? > +In your machine configuration:: > + > + OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay" > + > +and then in your recipe:: > + > + OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = "/usr/share/another-application" > + > +.. note:: > + > + The class does not support /etc directory itself, because systemd depends on it > + s/\/etc/``\/etc``/ s/systemd/``systemd``/ My nitpicks excluded, looks good :) ~ Cheers, Quentin ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [docs] [PATCH] ref-manual: add overlayfs class 2021-08-16 9:14 ` [docs] " Quentin Schulz @ 2021-08-17 3:29 ` Vyacheslav Yurkov 2021-08-17 8:37 ` Quentin Schulz 0 siblings, 1 reply; 5+ messages in thread From: Vyacheslav Yurkov @ 2021-08-17 3:29 UTC (permalink / raw) To: Quentin Schulz; +Cc: docs [-- Attachment #1: Type: text/plain, Size: 4936 bytes --] Hi Quentin, Thanks for the review! I'll address the notes in v2. A few answers to your questions are inlined below. On Mon, Aug 16, 2021, 13:14 Quentin Schulz < quentin.schulz@theobroma-systems.com> wrote: > Hi Vyacheslav, > > Thanks for the patch! > > On Sat, Aug 14, 2021 at 06:28:25AM +0200, Vyacheslav Yurkov wrote: > > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> > > --- > > documentation/ref-manual/classes.rst | 46 ++++++++++++++++++++++++++++ > > 1 file changed, 46 insertions(+) > > > > diff --git a/documentation/ref-manual/classes.rst > b/documentation/ref-manual/classes.rst > > index a98a64c43..5d33e81a1 100644 > > --- a/documentation/ref-manual/classes.rst > > +++ b/documentation/ref-manual/classes.rst > > @@ -1710,6 +1710,52 @@ one such example. However, being aware of this > class can reduce the > > proliferation of different versions of similar classes across multiple > > layers. > > > > +.. _ref-classes-overlayfs: > > + > > +``overlayfs.bbclass`` > > +======================= > > + > > + > > One newline is enough :) > > > +It's often desired in Embedded System design to have a read-only rootfs. > > +But a lot of different applications might want to have a read-write > access to > > +some parts of a filesystem. It can be especially useful when your > update mechanism > > +overwrites the whole rootfs, but you want your application data to be > preserved > > +between updates. The ``overlayfs`` class provides a way to achieve that > by means > > s/``overlayfs``/:ref:`overlayfs <ref-classes-overlayfs>`/ > > > +of overlayfs and at the same time keeping the base rootfs read-only. > > + > > I would highlight overlayfs with tick quotes (``overlayfs``). > > > +To use this class, set a mount point for a partition overlayfs is going > to use as upper > > +layer in your machine configuration. Underlying file system can be > anything that > > +is supported by overlayfs. This has to be done in your machine > configuration:: > > + > > How do I know what's supported by overlayfs? Where can I find this > important piece of information? > > > + OVERLAYFS_MOUNT_POINT[data] = "/data" > > + > > +.. note:: > > + > > + QA check fails to catch file existence if you redefine this variable > in your recipe! > > + > > What about: > This variable must only be set from a configuration file as QA check will > fail > to catch file existence if this variable is also set in a recipe. > > Also, I'm not sure to understand what "catch file existence" means? > Could you explain to me please :) > That means I check only that file exists in QA check, I do not check whether it's a valid systemd mount unit (to avoid additional dependency on systemd-native recipe). > > +The class assumes you have a data.mount systemd unit defined elsewhere > in your BSP > > s/data.mount/``data.mount``/ > > > +(e.g. in systemd-machine-units recipe) and it's installed to the image. > > + > > s/systemd-machine-units/``systemd-machine-units``/ > > > +Then you can specify writable directories on a recipe basis:: > > + > > If package recipes are supported, does it actually make sense to have > such variable in there instead of in image recipes? > > Just trying to know if we should highlight whether it's a bad idea for > users to do so. > There are actually two pitfalls here. Semantically directory belongs to a recipe it's used in. If you redefine OVERLAYFS_WRITABLE_PATHS in another recipe, do_rootfs would fail when two recipes install the same file. Another trade-off I had to deal with is if you redefine OVERLAYFS_MOUNT_POINT in a recipe, then I can't catch it in my QA check. This is due to a bitbake limitation, in particular I can't forbid further variable overrides. That's why I mentioned above that this variable has to be set in machine configuration. I hope that makes sense :) Regards, Vyacheslav > > + OVERLAYFS_WRITABLE_PATHS[data] = "/usr/share/my-custom-application" > > + > > +To support several mount points you can use a different variable flag. > Assume we > > s/Assume/Assuming/ ? > > > +want to have a writable location on the file system, but not interested > where the data > > +survive a reboot. Then we could have a mnt-overlay.mount unit for a > tmpfs file system. > > + > > s/mnt-overlay.mount/``mnt-overlay.mount``/ > > s/tmpfs/``tmpfs``/ ? > > > +In your machine configuration:: > > + > > + OVERLAYFS_MOUNT_POINT[mnt-overlay] = "/mnt/overlay" > > + > > +and then in your recipe:: > > + > > + OVERLAYFS_WRITABLE_PATHS[mnt-overlay] = > "/usr/share/another-application" > > + > > +.. note:: > > + > > + The class does not support /etc directory itself, because systemd > depends on it > > + > > s/\/etc/``\/etc``/ > > s/systemd/``systemd``/ > > My nitpicks excluded, looks good :) ~ > > Cheers, > Quentin [-- Attachment #2: Type: text/html, Size: 6616 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [docs] [PATCH] ref-manual: add overlayfs class 2021-08-17 3:29 ` Vyacheslav Yurkov @ 2021-08-17 8:37 ` Quentin Schulz 2021-08-17 13:47 ` Vyacheslav Yurkov 0 siblings, 1 reply; 5+ messages in thread From: Quentin Schulz @ 2021-08-17 8:37 UTC (permalink / raw) To: Vyacheslav Yurkov; +Cc: docs Hi Vyacheslav, On Tue, Aug 17, 2021 at 07:29:35AM +0400, Vyacheslav Yurkov wrote: > Hi Quentin, > Thanks for the review! I'll address the notes in v2. A few answers to your > questions are inlined below. > > On Mon, Aug 16, 2021, 13:14 Quentin Schulz < > quentin.schulz@theobroma-systems.com> wrote: > > > Hi Vyacheslav, > > > > Thanks for the patch! > > > > On Sat, Aug 14, 2021 at 06:28:25AM +0200, Vyacheslav Yurkov wrote: > > > Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> > > > --- > > > documentation/ref-manual/classes.rst | 46 ++++++++++++++++++++++++++++ > > > 1 file changed, 46 insertions(+) > > > > > > diff --git a/documentation/ref-manual/classes.rst > > b/documentation/ref-manual/classes.rst > > > index a98a64c43..5d33e81a1 100644 > > > --- a/documentation/ref-manual/classes.rst > > > +++ b/documentation/ref-manual/classes.rst > > > @@ -1710,6 +1710,52 @@ one such example. However, being aware of this > > class can reduce the > > > proliferation of different versions of similar classes across multiple > > > layers. > > > > > > +.. _ref-classes-overlayfs: > > > + > > > +``overlayfs.bbclass`` > > > +======================= > > > + > > > + > > > > One newline is enough :) > > > > > +It's often desired in Embedded System design to have a read-only rootfs. > > > +But a lot of different applications might want to have a read-write > > access to > > > +some parts of a filesystem. It can be especially useful when your > > update mechanism > > > +overwrites the whole rootfs, but you want your application data to be > > preserved > > > +between updates. The ``overlayfs`` class provides a way to achieve that > > by means > > > > s/``overlayfs``/:ref:`overlayfs <ref-classes-overlayfs>`/ > > > > > +of overlayfs and at the same time keeping the base rootfs read-only. > > > + > > > > I would highlight overlayfs with tick quotes (``overlayfs``). > > > > > +To use this class, set a mount point for a partition overlayfs is going > > to use as upper > > > +layer in your machine configuration. Underlying file system can be > > anything that > > > +is supported by overlayfs. This has to be done in your machine > > configuration:: > > > + > > > > How do I know what's supported by overlayfs? Where can I find this > > important piece of information? > > > > > + OVERLAYFS_MOUNT_POINT[data] = "/data" > > > + > > > +.. note:: > > > + > > > + QA check fails to catch file existence if you redefine this variable > > in your recipe! > > > + > > > > What about: > > This variable must only be set from a configuration file as QA check will > > fail > > to catch file existence if this variable is also set in a recipe. > > > > Also, I'm not sure to understand what "catch file existence" means? > > Could you explain to me please :) > > > > That means I check only that file exists in QA check, I do not check > whether it's a valid systemd mount unit (to avoid additional dependency on > systemd-native recipe). > I think what I was missing is that the QA check is actually part of the ROOTFS_POSTPROCESS_COMMAND variable. Therefore, only the image recipe will do the QA check. I think we should highlight that this variable should only be set in configuration files OR image recipes. (or basically, say that it shouldn't be set in package recipes). So I'd actually add two notes: 1) Variables to be defined in conf files or image recipes (or say the opposite: not in package recipes) 2) Only existence of the systemd mount unit file is checked, not its content. > > > > +The class assumes you have a data.mount systemd unit defined elsewhere > > in your BSP > > > > s/data.mount/``data.mount``/ > > > > > +(e.g. in systemd-machine-units recipe) and it's installed to the image. > > > + > > > > s/systemd-machine-units/``systemd-machine-units``/ > > > > > +Then you can specify writable directories on a recipe basis:: > > > + > > > > If package recipes are supported, does it actually make sense to have > > such variable in there instead of in image recipes? > > > > Just trying to know if we should highlight whether it's a bad idea for > > users to do so. > > > > There are actually two pitfalls here. > > Semantically directory belongs to a recipe it's used in. If you redefine > OVERLAYFS_WRITABLE_PATHS in another recipe, do_rootfs would fail when two > recipes install the same file. > > Another trade-off I had to deal with is if you redefine > OVERLAYFS_MOUNT_POINT in a recipe, then I can't catch it in my QA check. > This is due to a bitbake limitation, in particular I can't forbid further > variable overrides. That's why I mentioned above that this variable has to > be set in machine configuration. > I feel like OVERLAYFS configuration makes sense only when defined globally (conf files) or in image recipes. Just thinking out loud right now but couldn't we add the following in overlayfs.bbclass anonymous python code? if not bb.data.inherits_class('image', d): bb.fatal('Only image recipes can inherit overlayfs.bbclass. %s is a package recipe.' % d.getVar('${PN}')) or something like that? What are your thoughts on my remarks? Lemme know! Regards, Quentin ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [docs] [PATCH] ref-manual: add overlayfs class 2021-08-17 8:37 ` Quentin Schulz @ 2021-08-17 13:47 ` Vyacheslav Yurkov 0 siblings, 0 replies; 5+ messages in thread From: Vyacheslav Yurkov @ 2021-08-17 13:47 UTC (permalink / raw) To: Quentin Schulz; +Cc: docs Hi Quentin >> That means I check only that file exists in QA check, I do not check >> whether it's a valid systemd mount unit (to avoid additional dependency on >> systemd-native recipe). >> > I think what I was missing is that the QA check is actually part of the > ROOTFS_POSTPROCESS_COMMAND variable. Therefore, only the image recipe > will do the QA check. > > I think we should highlight that this variable should only be set in > configuration files OR image recipes. (or basically, say that it > shouldn't be set in package recipes). > > So I'd actually add two notes: > > 1) Variables to be defined in conf files or image recipes (or say the > opposite: not in package recipes) Not entirely correct, see my other answer below. > 2) Only existence of the systemd mount unit file is checked, not its > content. Agreed. > I feel like OVERLAYFS configuration makes sense only when defined > globally (conf files) or in image recipes. > > Just thinking out loud right now but couldn't we add the following in > overlayfs.bbclass anonymous python code? > > if not bb.data.inherits_class('image', d): > bb.fatal('Only image recipes can inherit overlayfs.bbclass. %s > is a package recipe.' % d.getVar('${PN}')) > > or something like that? > > What are your thoughts on my remarks? To sum it up: - OVERLAYFS_MOUNT_POINT should be defined globally and _not_ redefined afterwards. - OVERLAYFS_WRITABLE_PATHS can be used in any recipe (I guess not a package group though). It doesn't have to be image recipe, because a directory specified there can be application / recipe specific. For example, if we have a recipe my-app.bb and an application there wants to have a writable path in /usr/share/my-app, then it makes sense to set it in the my-app recipe, not in the image recipe, right? - QA check is part of ROOTFS_POSTPROCESS_COMMAND as you noted before and is done as one of the last step during image creation. > Lemme know! > > Regards, > Quentin Cheers, Vyacheslav ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-08-17 13:47 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-14 4:28 [PATCH] ref-manual: add overlayfs class Vyacheslav Yurkov 2021-08-16 9:14 ` [docs] " Quentin Schulz 2021-08-17 3:29 ` Vyacheslav Yurkov 2021-08-17 8:37 ` Quentin Schulz 2021-08-17 13:47 ` Vyacheslav Yurkov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.