All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] Adding post image hook
@ 2020-05-11 14:35 lpdev at cordier.org
  2020-05-11 15:50 ` Thomas Petazzoni
  0 siblings, 1 reply; 10+ messages in thread
From: lpdev at cordier.org @ 2020-05-11 14:35 UTC (permalink / raw)
  To: buildroot

Hello,

I'm currently trying to improve my custom buildroot tree (that makes use of BR2_EXTERNAL), especially the final steps. For instance my external tree adds an extra steps through the BR2_ROOTFS_POST_IMAGE_SCRIPT to generate the firmware file. This step also retrieve the version of different softwares present in the final image.

However the post image script comes with a big limitation: we cannot take the advantage of reading variables defined in the Makefile. As stated in the doc, only few environment variables are available. The only way to do so is to use the BR2_ROOTFS_POST_SCRIPT_ARGS, but it is also cumbersome to create a getopt parser in the .sh script.

I would like to know if adding a post image hook in the makefile could help to have a cleaner tree with less .sh files, less getopt parsing, etc.

This would be like:

define GENIMAGE_HOOK
 # Call genimage and other stuf here
endef 

define SWUPDATE_HOOK
 # Call genimage and other stuf here
endef 


BR2_POST_IMAGE_TARGET_HOOKS += GENIMAGE_HOOK
BR2_POST_IMAGE_TARGET_HOOKS += SWUPDATE_HOOK

Those could be defined in the external.mk for instance. I can do a PR if this feature makes sense.



This is how I decided to handle the firmware image generation currently. I'm not sure it's the best way for handling such case, because I'm overriding the default "world" target. I'm opened to any suggestions.

____________________________________________________
# Export some useful variable that can be used in other scripts
export BR2_CUSTOM_BOARDNAME := $(call qstrip,$(BR2_CUSTOM_BOARDNAME))
export BR2_CUSTOM_BOARDVERSION := $(shell git -C $(BR2_EXTERNAL_CUSTOM_PATH) describe --tags --dirty=-dev)

# Make an Firmware Update package.
ifeq ($(BR2_CUSTOM_GENERATE_FIRMWARE),y)
genfirmware: target-post-image
	@$(call MESSAGE,"Compressing the filesystem...")
	pigz -9 -c -n $(BINARIES_DIR)/rootfs.ext4 > $(BINARIES_DIR)/rootfs.ext4.gz

	@$(call MESSAGE,"Generating UEFI partition for the bootloader...")
	$(EXTRA_ENV) ./support/scripts/genimage.sh -c $(BR2_EXTERNAL_CUSTOM_PATH)/board/cstick/genimage_uefi.cfg

	@$(call MESSAGE,"Generating firmware version $(BR2_CUSTOM_BOARDVERSION) for board $(BR2_CUSTOM_BOARDNAME) ")
	(We can use bash script here with any makefile variables defined at this point, such as BR2_OPENCV_VERSION for instance)

# Override the default world target.
.PHONY: world
world: genfirmware
endif

...
____________________________________________________


Thank you for your time,

Louis-Paul CORDIER

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-11 14:35 [Buildroot] Adding post image hook lpdev at cordier.org
@ 2020-05-11 15:50 ` Thomas Petazzoni
  2020-05-13 20:05   ` lpdev at cordier.org
  0 siblings, 1 reply; 10+ messages in thread
From: Thomas Petazzoni @ 2020-05-11 15:50 UTC (permalink / raw)
  To: buildroot

Hello,

On Mon, 11 May 2020 16:35:10 +0200 (CEST)
lpdev at cordier.org wrote:

> I'm currently trying to improve my custom buildroot tree (that makes
> use of BR2_EXTERNAL), especially the final steps. For instance my
> external tree adds an extra steps through the
> BR2_ROOTFS_POST_IMAGE_SCRIPT to generate the firmware file. This step
> also retrieve the version of different softwares present in the final
> image.
> 
> However the post image script comes with a big limitation: we cannot
> take the advantage of reading variables defined in the Makefile.

You can actually do that, by running:

	make VARS=<some-variable-name> printvars

from your post-image script.

If what you need to retrieve is the version of the different packages,
you can also run:

	make show-info

from your post-image script, and parse the JSON output.

> # Export some useful variable that can be used in other scripts
> export BR2_CUSTOM_BOARDNAME := $(call qstrip,$(BR2_CUSTOM_BOARDNAME))
> export BR2_CUSTOM_BOARDVERSION := $(shell git -C $(BR2_EXTERNAL_CUSTOM_PATH) describe --tags --dirty=-dev)
> 
> # Make an Firmware Update package.
> ifeq ($(BR2_CUSTOM_GENERATE_FIRMWARE),y)
> genfirmware: target-post-image
> 	@$(call MESSAGE,"Compressing the filesystem...")
> 	pigz -9 -c -n $(BINARIES_DIR)/rootfs.ext4 > $(BINARIES_DIR)/rootfs.ext4.gz
> 
> 	@$(call MESSAGE,"Generating UEFI partition for the bootloader...")
> 	$(EXTRA_ENV) ./support/scripts/genimage.sh -c $(BR2_EXTERNAL_CUSTOM_PATH)/board/cstick/genimage_uefi.cfg
> 
> 	@$(call MESSAGE,"Generating firmware version $(BR2_CUSTOM_BOARDVERSION) for board $(BR2_CUSTOM_BOARDNAME) ")
> 	(We can use bash script here with any makefile variables defined at this point, such as BR2_OPENCV_VERSION for instance)
> 
> # Override the default world target.
> .PHONY: world
> world: genfirmware
> endif

Otherwise, what you did here looks OK to me. It is a hack of course,
but BR2_EXTERNAL is designed to allow hacks :-)

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-11 15:50 ` Thomas Petazzoni
@ 2020-05-13 20:05   ` lpdev at cordier.org
  2020-05-13 20:16     ` Thomas Petazzoni
  0 siblings, 1 reply; 10+ messages in thread
From: lpdev at cordier.org @ 2020-05-13 20:05 UTC (permalink / raw)
  To: buildroot



De : Thomas Petazzoni <thomas.petazzoni@bootlin.com>
? : lpdev at cordier.org
Sujet : Re: [Buildroot] Adding post image hook
Date : 11/05/2020 17:50:27 Europe/Paris
Copie ? : buildroot at buildroot.org

Hello,

On Mon, 11 May 2020 16:35:10 +0200 (CEST)
lpdev at cordier.org wrote:

> I'm currently trying to improve my custom buildroot tree (that makes
> use of BR2_EXTERNAL), especially the final steps. For instance my
> external tree adds an extra steps through the
> BR2_ROOTFS_POST_IMAGE_SCRIPT to generate the firmware file. This step
> also retrieve the version of different softwares present in the final
> image.
> 
> However the post image script comes with a big limitation: we cannot
> take the advantage of reading variables defined in the Makefile.

You can actually do that, by running:

make VARS=<some-variable-name> printvars

from your post-image script.

If what you need to retrieve is the version of the different packages,
you can also run:

make show-info

from your post-image script, and parse the JSON output.



Thank you for the tip. However this method seems to be very slow because of the makefile parsing (takes between 5 to 10sec on my PC). I would not recommend that for retrieving values of a lot of variables.





> # Export some useful variable that can be used in other scripts
> export BR2_CUSTOM_BOARDNAME := $(call qstrip,$(BR2_CUSTOM_BOARDNAME))
> export BR2_CUSTOM_BOARDVERSION := $(shell git -C $(BR2_EXTERNAL_CUSTOM_PATH) describe --tags --dirty=-dev)
> 
> # Make an Firmware Update package.
> ifeq ($(BR2_CUSTOM_GENERATE_FIRMWARE),y)
> genfirmware: target-post-image
> @$(call MESSAGE,"Compressing the filesystem...")
> pigz -9 -c -n $(BINARIES_DIR)/rootfs.ext4 > $(BINARIES_DIR)/rootfs.ext4.gz
> 
> @$(call MESSAGE,"Generating UEFI partition for the bootloader...")
> $(EXTRA_ENV) ./support/scripts/genimage.sh -c $(BR2_EXTERNAL_CUSTOM_PATH)/board/cstick/genimage_uefi.cfg
> 
> @$(call MESSAGE,"Generating firmware version $(BR2_CUSTOM_BOARDVERSION) for board $(BR2_CUSTOM_BOARDNAME) ")
> (We can use bash script here with any makefile variables defined at this point, such as BR2_OPENCV_VERSION for instance)
> 
> # Override the default world target.
> .PHONY: world
> world: genfirmware
> endif

Otherwise, what you did here looks OK to me. It is a hack of course,
but BR2_EXTERNAL is designed to allow hacks :-)



Soooo cool, I just became an hacker then! :D Thank you for your time Thomas




Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200513/1a266df7/attachment.html>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 20:05   ` lpdev at cordier.org
@ 2020-05-13 20:16     ` Thomas Petazzoni
  2020-05-13 20:35       ` Patrick Mochel
  2020-05-13 21:09       ` Patrick Mochel
  0 siblings, 2 replies; 10+ messages in thread
From: Thomas Petazzoni @ 2020-05-13 20:16 UTC (permalink / raw)
  To: buildroot

Hello Louis-Paul,

You should consider changing your e-mail client, or configure it
better. Indeed, the plain text version of your e-mail is not properly
quoted, i.e the text from me you are replying to was not quoted with
">" signs.

On Wed, 13 May 2020 22:05:08 +0200 (CEST)
lpdev at cordier.org wrote:

> You can actually do that, by running:
> 
> make VARS=<some-variable-name> printvars
> 
> from your post-image script.
> 
> If what you need to retrieve is the version of the different packages,
> you can also run:
> 
> make show-info
> 
> from your post-image script, and parse the JSON output.
> 
> 
> 

> Thank you for the tip. However this method seems to be very slow
> because of the makefile parsing (takes between 5 to 10sec on my PC).
> I would not recommend that for retrieving values of a lot of
> variables.

You can do:

  make VARS=FOO% printvars

if the different variables you want to get match a wildcard.

Or alternatively, you can do just:

  make printvars

pay the one-time price of dumping all variables, and then grab what you
need from the output.

But otherwise, I agree that indeed it is quite slow.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 20:16     ` Thomas Petazzoni
@ 2020-05-13 20:35       ` Patrick Mochel
  2020-05-13 21:09       ` Patrick Mochel
  1 sibling, 0 replies; 10+ messages in thread
From: Patrick Mochel @ 2020-05-13 20:35 UTC (permalink / raw)
  To: buildroot

>
> Hi there,


On Wed, May 13, 2020 at 1:05 PM <lpdev@cordier.org> wrote:

>
>
> De : Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> ? : lpdev at cordier.org
> Sujet : Re: [Buildroot] Adding post image hook
> Date : 11/05/2020 17:50:27 Europe/Paris
> Copie ? : buildroot at buildroot.org
>
> Hello,
>
> On Mon, 11 May 2020 16:35:10 +0200 (CEST)
> lpdev at cordier.org wrote:
>
> > I'm currently trying to improve my custom buildroot tree (that makes
> > use of BR2_EXTERNAL), especially the final steps. For instance my
> > external tree adds an extra steps through the
> > BR2_ROOTFS_POST_IMAGE_SCRIPT to generate the firmware file. This step
> > also retrieve the version of different softwares present in the final
> > image.
> >
> > However the post image script comes with a big limitation: we cannot
> > take the advantage of reading variables defined in the Makefile.
>
> You can actually do that, by running:
>
> make VARS=<some-variable-name> printvars
>
> from your post-image script.
>
> If what you need to retrieve is the version of the different packages,
> you can also run:
>
> make show-info
>
> from your post-image script, and parse the JSON output.
>
>
> Thank you for the tip. However this method seems to be very slow because
> of the makefile parsing (takes between 5 to 10sec on my PC). I would not
> recommend that for retrieving values of a lot of variables.
>

We are currently doing something analogous to what you initially described
and have run into the same behavior with 'make printvars' -- it takes a
long time to re-parse the Makefiles, no matter how many or few variables
you want.

You can use '%' to glob the names of variables; e.g.

$ make VARS='%_VERSION  # For every package's version

or

$ make VARS='EUDEV_%' # For every eudev variable

.. but you can only use it once, so you can't do this (purely theoretical)
example:

$ make VARS='%_CPE_%' # to get CPE_VENDOR, CVE_PRODUCT and CPE_VERSION for
every package

And, I've found that:

$ make VARS='%'

.. causes some amount of confusion internally and errors on the command
line.

In order to reduce the output generated -- and the amount of filtering
needed -- we are parsing the .config to get the packages that are enabled,
then generating the list of variable names to pass to 'make printvars'.
However, once that package list gets reasonably long, Bash gets upset at
the length of the command-line you're trying to execute. For now, we've
settled on a hybrid approach until we can revisit it and look into
rectifying it for good (and for everyone).

.. which is an excellent segue into stating that it's hard to say what an
ideal approach would be. It seems wasteful/unnecessary to export all make
variables to external helpers via the environment. And, it seems extraneous
to generate a new .hidden_file somewhere with a laundry-list of all make
variables, especially since they're changing over time (especially with
external components).

Getting back to the original poster -- I recommend looking into how
'support/scripts/pkg-stats' handles this WRT python's string methods
'.startswith()' and '.endswith()'. It ain't the prettiest nor most
efficient, but it works reasonably well and can be adapted to construct a
mapping of whatever it is you're looking for, for any component.

Cheers,


Patrick

>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200513/bc658c5a/attachment.html>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 20:16     ` Thomas Petazzoni
  2020-05-13 20:35       ` Patrick Mochel
@ 2020-05-13 21:09       ` Patrick Mochel
  2020-05-13 21:29         ` Yann E. MORIN
  1 sibling, 1 reply; 10+ messages in thread
From: Patrick Mochel @ 2020-05-13 21:09 UTC (permalink / raw)
  To: buildroot

Hi Thomas,

On Wed, May 13, 2020 at 1:16 PM Thomas Petazzoni <
thomas.petazzoni@bootlin.com> wrote:


> Or alternatively, you can do just:
>
>   make printvars
>

FWIW, This doesn't generate any output.

As I mentioned in my previous email, you can do e.g.:

$ make printvars VARS="%" > printvars-plain.mk

.. which does generate +/- the desired output of _everything_, but it also
results in this in the shell:

make: support/dependencies/check-host-.sh: Command not found


.. with the result (only as an aside) of this:

$ wc -l printvars-plain.mk && ls -sh printvars-plain.mk
307705 printvars-plain.mk
31M printvars-plain.mk

All things considered, it's not that big of a deal in terms of size, but it
does also include all of the expanded macros; e.g. 'define
inner-kconfig-package', of which I'll spare the list the full text and
leave it as an exercise to the reader. ;-)

Those are all appended to the list of package (and other component)
variables; i.e. right after this last package variable:

ZZIPLIB_VERSION=0.13.69

.. and this makes parsing the bulk slightly problematic -- there are ~30k
lines of generated make functions with interspersed blank lines that are
irrelevant to package (or other component) variables.

In the end, this is just FYI. If you have suggestions of how one might
improve on this, I'm all ears and might even look into rectifying it..

Cheers,


Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200513/e126c432/attachment.html>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 21:09       ` Patrick Mochel
@ 2020-05-13 21:29         ` Yann E. MORIN
  2020-05-13 21:32           ` Yann E. MORIN
  2020-05-13 21:46           ` Patrick Mochel
  0 siblings, 2 replies; 10+ messages in thread
From: Yann E. MORIN @ 2020-05-13 21:29 UTC (permalink / raw)
  To: buildroot

Patrick, All,

On 2020-05-13 14:09 -0700, Patrick Mochel spake thusly:
> Hi Thomas,
> On Wed, May 13, 2020 at 1:16 PM Thomas Petazzoni < [1]thomas.petazzoni@bootlin.com> wrote:
>   Or alternatively, you can do just:
>   ? make printvars

This only works for in-tree builds. For out-of-tree builds, you would
want to run (this also works for in-tree builds):

    make -C $(BASE_DIR) printvars

> FWIW, This doesn't generate any output.

Probably you were doing an out-of-tree build?

> As I mentioned in my previous email, you can do e.g.:
> $ make printvars VARS="%" > [2]printvars-plain.mk
> .. which does generate?+/- the desired output of _everything_, but it also results in this in the shell:
>   make: support/dependencies/check-host-.sh: Command not found

Yes, this is a side-effect of not actually runing the commands: some
variables are not set in this context, when they are macros that expect
arguemnts. For example, this very case is caused by:

    support/dependencies/dependencies.mk;

        define suitable-host-package
        $(shell support/dependencies/check-host-$(1).sh $(2))
        endef

So suitable-host-package is a vriable like the others. But it's also a
macro, which is expect to be called with $(call ...). However, in the
context of printvars, it is expanded as a simple variable. In this
context, $(1) and $(2) are empty.

> .. with the result (only as an aside) of this:
> $ wc -l [3]printvars-plain.mk && ls -sh [4]printvars-plain.mk
> 307705 [5]printvars-plain.mk
> 31M [6]printvars-plain.mk
> All things considered, it's not that big of a deal in terms of size, but it does also include all of the expanded macros; e.g. '
> define inner-kconfig-package', of which I'll spare the list the full text and leave it as an exercise to the reader. ;-)?
> Those are all appended to the list of package (and other component) variables; i.e. right after this last package variable:
> ZZIPLIB_VERSION=0.13.69
> .. and this makes parsing the bulk slightly problematic -- there are ~30k lines of generated make functions with interspersed blank
> lines that are irrelevant to package (or other component) variables.
> In the end, this is just FYI. If you have suggestions of how one might improve on this, I'm all ears and might even look into
> rectifying it..

Nowadays, the cannonical way to extract information out of the internals
of Buildroot is geared toward json-formatted output.

If you want information from packages, you should use (and extend if
needed) from your post-scripts:

    make --no-print-directory -C $(BASE_DIR) show-info

Or try in your favourite interactive bourne shell:

    $ make show-info | jq .
    $ make show-info | jq keys

This contains information about packages, indexed by package name, with
version, dependencies, legal info, download URLs, etc, etc, etc...

Other internal stuff is not exposed, and I am not sure if we want to
make it easy/easier to get at... But *if* we were to do that, I'd leave
printvars aside as-is for legacy scripts, and introduce a new show-vars
(or whatever) that outputs json-formatted content. But again, I'm not
sure we'd want to provide that...

Regards,
Yann E. MORIN.

> Cheers,
> Patrick
> 
> Links:
> 1. mailto:thomas.petazzoni at bootlin.com
> 2. http://printvars-plain.mk
> 3. http://printvars-plain.mk
> 4. http://printvars-plain.mk
> 5. http://printvars-plain.mk
> 6. http://printvars-plain.mk

> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot


-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 21:29         ` Yann E. MORIN
@ 2020-05-13 21:32           ` Yann E. MORIN
  2020-05-13 21:46           ` Patrick Mochel
  1 sibling, 0 replies; 10+ messages in thread
From: Yann E. MORIN @ 2020-05-13 21:32 UTC (permalink / raw)
  To: buildroot

All,

On 2020-05-13 23:29 +0200, Yann E. MORIN spake thusly:
> On 2020-05-13 14:09 -0700, Patrick Mochel spake thusly:
> > Hi Thomas,
> > On Wed, May 13, 2020 at 1:16 PM Thomas Petazzoni < [1]thomas.petazzoni@bootlin.com> wrote:
> >   Or alternatively, you can do just:
> >   ? make printvars
> This only works for in-tree builds. For out-of-tree builds, you would
> want to run (this also works for in-tree builds):
>     make -C $(BASE_DIR) printvars

Of course, since this is in a shell script, this should be ${BASE_DIR}
not $(BASE_DIR).

[--SNIP--]
> If you want information from packages, you should use (and extend if
> needed) from your post-scripts:
>     make --no-print-directory -C $(BASE_DIR) show-info

Ditto.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 21:29         ` Yann E. MORIN
  2020-05-13 21:32           ` Yann E. MORIN
@ 2020-05-13 21:46           ` Patrick Mochel
  2020-05-13 23:27             ` LP C
  1 sibling, 1 reply; 10+ messages in thread
From: Patrick Mochel @ 2020-05-13 21:46 UTC (permalink / raw)
  To: buildroot

Hi Yann,

On Wed, May 13, 2020 at 2:30 PM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:


> > FWIW, This doesn't generate any output.
>
> Probably you were doing an out-of-tree build?
>

Nope, it's an in-tree build, configured and already built. And it's this
recent:

 $ git describe
2020.02-1207-ga8da12f0f9


...

> > .. which does generate +/- the desired output of _everything_, but it
> also results in this in the shell:
> >   make: support/dependencies/check-host-.sh: Command not found
>
> Yes, this is a side-effect of not actually runing the commands: some
>
...

ACK -- not a personal concern; just a minor wart.

...

> Nowadays, the cannonical way to extract information out of the internals
> of Buildroot is geared toward json-formatted output.
>
> If you want information from packages, you should use (and extend if
> needed) from your post-scripts:
>
>     make --no-print-directory -C $(BASE_DIR) show-info
>

That's a good point and I shall look into it again as to whether that'll
provide all the metadata that we need, since we need the similar concerning
the firmware and kernel components, which are classified differently than
the packages.

If you're curious, here's the current (fresh!) code:

https://github.com/TimesysGit/vigiles-buildroot

I apologize for the shameless promotion of a semi-commercial product. And,
I must say it ain't my best work. That said, the point of it is to gather
specific metadata about the various components of an built system, generate
a json manifest and submit for a CVE check of known vulnerabilities (that's
our money-maker). YMMV.

Thanks for the reply.

Cheers,


Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20200513/4506c349/attachment.html>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [Buildroot] Adding post image hook
  2020-05-13 21:46           ` Patrick Mochel
@ 2020-05-13 23:27             ` LP C
  0 siblings, 0 replies; 10+ messages in thread
From: LP C @ 2020-05-13 23:27 UTC (permalink / raw)
  To: buildroot

On May 13 2020, at 11:46 pm, Patrick Mochel <patrick.mochel@timesys.com> wrote:

> Hi Yann,
>  
>  
>> On Wed, May 13, 2020 at 2:30 PM Yann E. MORIN
>> <yann.morin.1998@free.fr> wrote:
>> ?
>>  
>>>> FWIW, This doesn't generate any output.
>>>  
>>> Probably you were doing an out-of-tree build?
>>  
>> Nope, it's an in-tree build, configured and already built. And it's
>> this recent:
>>  
>> ?$ git describe
>> 2020.02-1207-ga8da12f0f9
>>  
>>  
>> ...
>>  
>>>> .. which does generate?+/- the desired output of _everything_, but
>>>> it also results in this in the shell:
>>>> ? ?make: support/dependencies/check-host-.sh: Command not found
>>>  
>>> Yes, this is a side-effect of not actually runing the commands: some
>> ...
>>  
>> ACK -- not a personal concern; just a minor wart.
>>  
>> ...
>>  
>>> Nowadays, the cannonical way to extract information out of the internals
>>> of Buildroot is geared toward json-formatted output.
>>>  
>>> If you want information from packages, you should use (and extend if
>>> needed) from your post-scripts:
>>>  
>>> ? ? make --no-print-directory -C $(BASE_DIR) show-info
>>  
>> That's a good point and I shall look into it again as to whether
>> that'll provide all the metadata that we need, since we need the
>> similar concerning the firmware and kernel components, which are
>> classified differently than the packages.
>>  
>> If you're curious, here's the current (fresh!) code:
>>  
>> https://github.com/TimesysGit/vigiles-buildroot
>>  
>> I apologize for the shameless promotion of a semi-commercial product.
>> And, I must say it ain't my best work. That said, the point of it is
>> to gather specific metadata about the various components of an built
>> system, generate a json manifest and submit for a CVE check of known
>> vulnerabilities (that's our money-maker). YMMV.?
>>  
>> Thanks for the reply.?
>>  
>> Cheers,
>>  
>>  
>> Patrick


Hi again,

I would like to jump back to the original topic of this email: adding
post image hook. I would like to have your thoughts about my
proposition. Following are additional considerations on it:

AFAIK, post-build and post-image scripts are the only part of buildroot
that is executing user-provided scripts and the only use case of using
those .sh scripts is when you are using vanilla buildroot (without
BR2_EXTERNAL, without fork), and/or when layering is required (see doc
section: "9.1.1. Implementing layered customizations"). I also
understand it's the only way at the moment for people with only a
defconfig file to launch some extra tasks at the end of their build.

That said we can also have something like BR2_EXTERNAL_MAKEFILE with
those proposed hooks defined inside. It remains to have an external.mk
file, but lighter. Anyway, I see less and less advantages of not using
BR2_EXTERNAL to be honest.

Pros of makefile:  

- full access to all BR2_* variables to customize image name, get
version of packages, generate a manifest...
- does not break the layering customization order, as hooks will be
added and executed in the same order they are defined.
- BR2_ROOTFS_POST_SCRIPT_ARGS: remove the "All the scripts will be
passed the exact same set of arguments, it is not possible to pass
different sets of arguments to each script." limitation stated in the
doc section "9.5. Customizing the generated target filesystem". It is
also the only config that can't take advantage of layering AFAIK. So
scripts will get the arguments they are ONLY needing.

Cons:

- The dev has to define a makefile + .sh script if he/she absolutely
wants to keep legacy code. Some (basic) knowledge of makefile will be required.
- break legacy?

To be honest I already have a solution that works for my use case. My
motivation comes from other devs stuck that would love to have an
"official" and clean way to do achieve those actions (if there is any?).
I would be interested to know if it can makes Patrick's life easier with
this design (sorry I don't have the courage of digging into your
project's code)

Thanks all for your time

Louis-Paul CORDIER  

PS: sorry for the email indentation, my email webclient (mailo) is
barely supporting plain text editing :(

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2020-05-13 23:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-11 14:35 [Buildroot] Adding post image hook lpdev at cordier.org
2020-05-11 15:50 ` Thomas Petazzoni
2020-05-13 20:05   ` lpdev at cordier.org
2020-05-13 20:16     ` Thomas Petazzoni
2020-05-13 20:35       ` Patrick Mochel
2020-05-13 21:09       ` Patrick Mochel
2020-05-13 21:29         ` Yann E. MORIN
2020-05-13 21:32           ` Yann E. MORIN
2020-05-13 21:46           ` Patrick Mochel
2020-05-13 23:27             ` LP C

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.