[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Vadim Kochan]

Add ability to generate image by custom package, and one of the
way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
called after rootfs image is ready. This allows to have easier
alternative to post-image script in form of package, which might be
just selected by the user (or this package might be automatically
selected already already).

Signed-off-by: Vadim Kochan <vadim4j@gmail.com>
---
 fs/common.mk           | 1 +
 package/pkg-generic.mk | 2 ++
 2 files changed, 3 insertions(+)

diff --git a/fs/common.mk b/fs/common.mk
index 4ad51fdd0a..3288406af6 100644
--- a/fs/common.mk
+++ b/fs/common.mk
@@ -152,6 +152,7 @@ ifneq ($$(ROOTFS_$(2)_COMPRESS_CMD),)
 	PATH=$$(BR_PATH) $$(ROOTFS_$(2)_COMPRESS_CMD) $$@ > $$@$$(ROOTFS_$(2)_COMPRESS_EXT)
 endif
 	$$(foreach hook,$$(ROOTFS_$(2)_POST_GEN_HOOKS),$$(call $$(hook))$$(sep))
+	$$(foreach hook,$$(ROOTFS_POST_GEN_HOOKS),$$(call $$(hook))$$(sep))
 
 rootfs-$(1)-show-depends:
 	@echo $$(ROOTFS_$(2)_DEPENDENCIES)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index a83813e28d..332e3c3cc1 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -724,6 +724,7 @@ $(2)_PRE_LEGAL_INFO_HOOKS       ?=
 $(2)_POST_LEGAL_INFO_HOOKS      ?=
 $(2)_TARGET_FINALIZE_HOOKS      ?=
 $(2)_ROOTFS_PRE_CMD_HOOKS       ?=
+$(2)_ROOTFS_POST_GEN_HOOKS      ?=
 
 ifeq ($$($(2)_TYPE),target)
 ifneq ($$(HOST_$(2)_KCONFIG_VAR),)
@@ -1052,6 +1053,7 @@ PACKAGES_USERS += $$($(2)_USERS)$$(sep)
 endif
 TARGET_FINALIZE_HOOKS += $$($(2)_TARGET_FINALIZE_HOOKS)
 ROOTFS_PRE_CMD_HOOKS += $$($(2)_ROOTFS_PRE_CMD_HOOKS)
+ROOTFS_POST_GEN_HOOKS += $$($(2)_ROOTFS_POST_GEN_HOOKS)
 
 ifeq ($$($(2)_SITE_METHOD),svn)
 DL_TOOLS_DEPENDENCIES += svn
-- 
2.14.1

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Thomas Petazzoni]

Hello Vadim,

On Mon, 15 Apr 2019 03:26:35 +0300
Vadim Kochan <vadim4j@gmail.com> wrote:

> Add ability to generate image by custom package, and one of the
> way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> called after rootfs image is ready. This allows to have easier
> alternative to post-image script in form of package, which might be
> just selected by the user (or this package might be automatically
> selected already already).

Hum, this description is a bit vague/fuzzy to me. Do you have a
specific example that requires this rootfs post gen hook ?

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Vadim Kochan]

Hi Thomas,

On Mon, Apr 15, 2019 at 08:58:19AM +0200, Thomas Petazzoni wrote:
> Hello Vadim,
> 
> On Mon, 15 Apr 2019 03:26:35 +0300
> Vadim Kochan <vadim4j@gmail.com> wrote:
> 
> > Add ability to generate image by custom package, and one of the
> > way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> > called after rootfs image is ready. This allows to have easier
> > alternative to post-image script in form of package, which might be
> > just selected by the user (or this package might be automatically
> > selected already already).
> 
> Hum, this description is a bit vague/fuzzy to me. Do you have a
> specific example that requires this rootfs post gen hook ?
> 

The use case is simple - allow to generate firmware image by custom
package instead of post-image script. I did not find some dependencies
or other hook to make it installed after rootfs image is ready, so only
option I found is to add yet-another-hook which will be called after
rootfs image is generated.

Regards,
Vadim Kochan

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[yann.morin at orange.com]

Vadim, All,

On 2019-04-15 10:33 +0300, Vadim Kochan spake thusly:
> On Mon, Apr 15, 2019 at 08:58:19AM +0200, Thomas Petazzoni wrote:
> > Hello Vadim,
> > 
> > On Mon, 15 Apr 2019 03:26:35 +0300
> > Vadim Kochan <vadim4j@gmail.com> wrote:
> > 
> > > Add ability to generate image by custom package, and one of the
> > > way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> > > called after rootfs image is ready. This allows to have easier
> > > alternative to post-image script in form of package, which might be
> > > just selected by the user (or this package might be automatically
> > > selected already already).
> > 
> > Hum, this description is a bit vague/fuzzy to me. Do you have a
> > specific example that requires this rootfs post gen hook ?
> > 
> 
> The use case is simple - allow to generate firmware image by custom
> package instead of post-image script. I did not find some dependencies
> or other hook to make it installed after rootfs image is ready, so only
> option I found is to add yet-another-hook which will be called after
> rootfs image is generated.

If you have a custom tool to generate a custom image, why don't you also
provide a custom filesystem implementation of your own in fs/your-fs/
like any other filesystem ?

Note that a filesystem can also be defined in a br2-external tree.

Regards,
Yann E. MORIN.

-- 
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
| +33 534.541.179 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
|      yann.morin (at) orange.com      |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Vadym Kochan]

Hi Yann,

On Mon, Apr 15, 2019 at 09:58:22AM +0200, yann.morin at orange.com wrote:
> Vadim, All,
> 
> On 2019-04-15 10:33 +0300, Vadim Kochan spake thusly:
> > On Mon, Apr 15, 2019 at 08:58:19AM +0200, Thomas Petazzoni wrote:
> > > Hello Vadim,
> > > 
> > > On Mon, 15 Apr 2019 03:26:35 +0300
> > > Vadim Kochan <vadim4j@gmail.com> wrote:
> > > 
> > > > Add ability to generate image by custom package, and one of the
> > > > way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> > > > called after rootfs image is ready. This allows to have easier
> > > > alternative to post-image script in form of package, which might be
> > > > just selected by the user (or this package might be automatically
> > > > selected already already).
> > > 
> > > Hum, this description is a bit vague/fuzzy to me. Do you have a
> > > specific example that requires this rootfs post gen hook ?
> > > 
> > 
> > The use case is simple - allow to generate firmware image by custom
> > package instead of post-image script. I did not find some dependencies
> > or other hook to make it installed after rootfs image is ready, so only
> > option I found is to add yet-another-hook which will be called after
> > rootfs image is generated.
> 
> If you have a custom tool to generate a custom image, why don't you also
> provide a custom filesystem implementation of your own in fs/your-fs/
> like any other filesystem ?
> 
> Note that a filesystem can also be defined in a br2-external tree.
> 

Yes, I understand but this is not for fs image generation but for the
final firmware image (for flashing, e.g. sdcard image), for example:

	https://github.com/vkochan/buildroot-external-rpi/blob/master/package/rpi-image/rpi-image.mk

Regards,
Vadim Kochan

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[yann.morin at orange.com]

Vadym, All,

On 2019-04-15 11:42 +0300, Vadym Kochan spake thusly:
> On Mon, Apr 15, 2019 at 09:58:22AM +0200, yann.morin at orange.com wrote:
> > Vadim, All,
> > 
> > On 2019-04-15 10:33 +0300, Vadim Kochan spake thusly:
> > > On Mon, Apr 15, 2019 at 08:58:19AM +0200, Thomas Petazzoni wrote:
> > > > Hello Vadim,
> > > > 
> > > > On Mon, 15 Apr 2019 03:26:35 +0300
> > > > Vadim Kochan <vadim4j@gmail.com> wrote:
> > > > 
> > > > > Add ability to generate image by custom package, and one of the
> > > > > way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> > > > > called after rootfs image is ready. This allows to have easier
> > > > > alternative to post-image script in form of package, which might be
> > > > > just selected by the user (or this package might be automatically
> > > > > selected already already).
> > > > 
> > > > Hum, this description is a bit vague/fuzzy to me. Do you have a
> > > > specific example that requires this rootfs post gen hook ?
> > > > 
> > > 
> > > The use case is simple - allow to generate firmware image by custom
> > > package instead of post-image script. I did not find some dependencies
> > > or other hook to make it installed after rootfs image is ready, so only
> > > option I found is to add yet-another-hook which will be called after
> > > rootfs image is generated.
> > 
> > If you have a custom tool to generate a custom image, why don't you also
> > provide a custom filesystem implementation of your own in fs/your-fs/
> > like any other filesystem ?
> > 
> > Note that a filesystem can also be defined in a br2-external tree.
> > 
> 
> Yes, I understand but this is not for fs image generation but for the
> final firmware image (for flashing, e.g. sdcard image), for example:
> 
> 	https://github.com/vkochan/buildroot-external-rpi/blob/master/package/rpi-image/rpi-image.mk

I perfectly understand what this is about. But now, say that there are
two (or more) packages that must provide a set of tools to run in
sequence on the generated filesystem?

For example, the first is an incantation of genimage or some such, and
the second one does a signature, and so on?

How would your proposal cover this?

This is exactly what post-image is for: to be able to provide whatever
project-specific, board-specific (etc..) tooling to run after filesystems
have been generated.

Besides, your rpi-image stuff does not even solve your (non-)problem,
because your project-specific defconfig still has to enable that package
anyway, so it is not different than having a post-image script listed
in the defconfig.

Regards,
Yann E. MORIN.

-- 
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
| +33 534.541.179 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
|      yann.morin (at) orange.com      |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'


_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

[Buildroot] [RFC 1/1] infra/pkg-generic: Add rootfs post gen hook

[Vadym Kochan]

Hi Yann,

On Mon, Apr 15, 2019 at 10:53:42AM +0200, yann.morin at orange.com wrote:
> Vadym, All,
> 
> On 2019-04-15 11:42 +0300, Vadym Kochan spake thusly:
> > On Mon, Apr 15, 2019 at 09:58:22AM +0200, yann.morin at orange.com wrote:
> > > Vadim, All,
> > > 
> > > On 2019-04-15 10:33 +0300, Vadim Kochan spake thusly:
> > > > On Mon, Apr 15, 2019 at 08:58:19AM +0200, Thomas Petazzoni wrote:
> > > > > Hello Vadim,
> > > > > 
> > > > > On Mon, 15 Apr 2019 03:26:35 +0300
> > > > > Vadim Kochan <vadim4j@gmail.com> wrote:
> > > > > 
> > > > > > Add ability to generate image by custom package, and one of the
> > > > > > way to do it - is to add new $(PKG)_ROOTFS_POST_GEN_HOOK which will be
> > > > > > called after rootfs image is ready. This allows to have easier
> > > > > > alternative to post-image script in form of package, which might be
> > > > > > just selected by the user (or this package might be automatically
> > > > > > selected already already).
> > > > > 
> > > > > Hum, this description is a bit vague/fuzzy to me. Do you have a
> > > > > specific example that requires this rootfs post gen hook ?
> > > > > 
> > > > 
> > > > The use case is simple - allow to generate firmware image by custom
> > > > package instead of post-image script. I did not find some dependencies
> > > > or other hook to make it installed after rootfs image is ready, so only
> > > > option I found is to add yet-another-hook which will be called after
> > > > rootfs image is generated.
> > > 
> > > If you have a custom tool to generate a custom image, why don't you also
> > > provide a custom filesystem implementation of your own in fs/your-fs/
> > > like any other filesystem ?
> > > 
> > > Note that a filesystem can also be defined in a br2-external tree.
> > > 
> > 
> > Yes, I understand but this is not for fs image generation but for the
> > final firmware image (for flashing, e.g. sdcard image), for example:
> > 
> > 	https://github.com/vkochan/buildroot-external-rpi/blob/master/package/rpi-image/rpi-image.mk
> 
> I perfectly understand what this is about. But now, say that there are
> two (or more) packages that must provide a set of tools to run in
> sequence on the generated filesystem?
> 
> For example, the first is an incantation of genimage or some such, and
> the second one does a signature, and so on?
> 
> How would your proposal cover this?
Good point! Actually the package will cover genimage invocation, the
rest is still possible to handle in user's post-image (which will be called
after $(TARGETS_ROOTFS).

> 
> This is exactly what post-image is for: to be able to provide whatever
> project-specific, board-specific (etc..) tooling to run after filesystems
> have been generated.
> 
> Besides, your rpi-image stuff does not even solve your (non-)problem,
> because your project-specific defconfig still has to enable that package
> anyway, so it is not different than having a post-image script listed
> in the defconfig.
> 
Well the package is automatically selected by default if RPI's top
option is enabled (in ${EXTERNAL}/Config.in, actually it is 'implied',
so it can be easy disabled via menuconfig).

Thanks,
Vadim Kochan