Prescribed way to make global variables in recipes?

Prescribed way to make global variables in recipes?

[Andy Gikling]

[-- Attachment #1: Type: text/plain, Size: 3096 bytes --]

Dear Yocto,

First of all, I love this project.  Thanks for all your hard work.

Question:  What is the prescribed way to allow image recipes to make global configuration variables that other recipes can use?

After extensive research and educating myself on how bitbake works, it seems that what I'm asking for is technically not possible.  This issue is discussed previously without a clear conclusion in the mailing list:

https://lists.yoctoproject.org/pipermail/yocto/2012-February/004552.html

Another work around is to setup a BB_ENV_EXTRAWHITE variable as described here:

http://stackoverflow.com/questions/17366984/is-it-possible-to-pass-in-command-line-variables-to-a-bitbake-build

However, this doesn't seem like the right approach either.

If there's no way to make global variables in an image recipe it seems one of two things are true:  Either bitbake lacks this powerful feature for some reason or there's some other "prescribed" way to accomplish what I need and I just don't know what that is.  I suspect the latter is true.

Let me give a simple example of why I need this feature.  We are deploying an application with Yocto called "faux-app."  Once this product is live, I would like to use our continuous integration build mechanism to build images for faux-app that contain the latest version of the user space application code.

We have a meta-faux-app layer with a hand full of recipes that are gracefully customizing our image.  One of the recipes is an "image recipe" that pulls in core-image and builds the output deploy binaries called faux-app_0.1.bb.

Another recipe is essentially a bbappend that is in charge of applying a patch to u-boot.  This patch configures u-boot to behave in a "debug" fashion vs a "release" fashion (ie, defaulting to network boot for debug and sdcard boot for release).

Ideally, for CI purposes we would like to allow our CI server to simply call "bitbake faux-app" to make a release build and then "bitbake faux-app-debug" to make a debug build.  The thought process is I could have a variable defined in both "faux-app_0.1.bb" and "faux-app-debug_0.1.bb" called FAUX_RELEASE_MODE.  In faux-app_0.1.bb this variable would be set to "release" and in faux-app-debug_0.1.bb it would be set to "debug."

Then, in the "u-boot_2014.04.bbappend" file found in our layer, we could append the correct patch to the SRC_URI variable using inline python logic for example:

SRC_URI += "${@base_contains('FAUX_RELEASE_MODE', 'release', 'file://u-boot-mode-release.patch', ' file://u-boot-mode-debug.patch ', d)}"

So in this way I have my image recipe setting a value that tells other recipes how to behave.  This seems like a perfectly reasonable use case so I'm at a loss as to why bitbake doesn't support this (I understand the environment variables is "scrubbed" between recipes - it makes sense).  Is there another mechanism I'm not aware of?  Can someone give me an example of a different way to accomplish my use case described above?

Any guidance would be much appreciated!  Thanks,

~Andy Gikling



[-- Attachment #2: Type: text/html, Size: 6607 bytes --]

Re: Prescribed way to make global variables in recipes?

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1429 bytes --]

On 8 April 2016 at 15:30, Andy Gikling <agikling@minnetronix.com> wrote:

> Dear Yocto,
>
>
>
> First of all, I love this project.  Thanks for all your hard work.
>
>
>
> Question:  What is the prescribed way to allow image recipes to make
> global configuration variables that other recipes can use?
>
>
You can't.  The basic reason being that images are built from packages, and
the packages have already been built when the image is built.


> Ideally, for CI purposes we would like to allow our CI server to simply
> call “bitbake faux-app” to make a release build and then “bitbake
> faux-app-debug” to make a debug build.  The thought process is I could have
> a variable defined in both “faux-app_0.1.bb” and “faux-app-debug_0.1.bb”
> called FAUX_RELEASE_MODE.  In faux-app_0.1.bb this variable would be set
> to “release” and in faux-app-debug_0.1.bb it would be set to “debug.”
>

In your view, what would happen if you did "bitbake faux-app
faux-app-debug"?

There are several ways to fix this that don't involve breaking how OE
works.  Either have a rootfs-time postprocess command in the debug image
recipe that manipulates the installed file system, or have two recipes for
the thing that changes (in this case u-boot) where one includes the other
and makes the changes required, and the image install the relevant package
(u-boot vs u-boot-debug).

Ross

[-- Attachment #2: Type: text/html, Size: 2556 bytes --]

Re: Prescribed way to make global variables in recipes?

[Andy Gikling]

[-- Attachment #1: Type: text/plain, Size: 2419 bytes --]

Ross,

Perfect thank you so much.  That makes total sense now.

I like the idea of just having two different recipes and then use IMAGE_INSTALL.  However, in the u-boot example I’m actually just using a bbappend to the u-boot recipe provided with our target SOM’s bsp.  Can I select different bbappend files in my image recipe with IMAGE_INSTALL?  I don’t think so but I’ll try though.

So what do I do in that case? Just make two of my own versions of the u-boot_2014.04.bb file, and give them different names?  For example “u-boot-faux_2014.04.bb” and “u-boot-faux-debug_2014.04.bb” ?  That should work but does that make sense to do?

Thanks again,

~Andy Gikling

From: Burton, Ross [mailto:ross.burton@intel.com]
Sent: Friday, April 08, 2016 9:44 AM
To: Andy Gikling
Cc: yocto@yoctoproject.org
Subject: Re: [yocto] Prescribed way to make global variables in recipes?

On 8 April 2016 at 15:30, Andy Gikling <agikling@minnetronix.com<mailto:agikling@minnetronix.com>> wrote:
Dear Yocto,

First of all, I love this project.  Thanks for all your hard work.

Question:  What is the prescribed way to allow image recipes to make global configuration variables that other recipes can use?

You can't.  The basic reason being that images are built from packages, and the packages have already been built when the image is built.

Ideally, for CI purposes we would like to allow our CI server to simply call “bitbake faux-app” to make a release build and then “bitbake faux-app-debug” to make a debug build.  The thought process is I could have a variable defined in both “faux-app_0.1.bb<http://faux-app_0.1.bb>” and “faux-app-debug_0.1.bb<http://faux-app-debug_0.1.bb>” called FAUX_RELEASE_MODE.  In faux-app_0.1.bb<http://faux-app_0.1.bb> this variable would be set to “release” and in faux-app-debug_0.1.bb<http://faux-app-debug_0.1.bb> it would be set to “debug.”

In your view, what would happen if you did "bitbake faux-app faux-app-debug"?

There are several ways to fix this that don't involve breaking how OE works.  Either have a rootfs-time postprocess command in the debug image recipe that manipulates the installed file system, or have two recipes for the thing that changes (in this case u-boot) where one includes the other and makes the changes required, and the image install the relevant package (u-boot vs u-boot-debug).

Ross

[-- Attachment #2: Type: text/html, Size: 7659 bytes --]

Re: Prescribed way to make global variables in recipes?

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 670 bytes --]

On 8 April 2016 at 16:05, Andy Gikling <agikling@minnetronix.com> wrote:

> So what do I do in that case? Just make two of my own versions of the
> u-boot_2014.04.bb file, and give them different names?  For example “
> u-boot-faux_2014.04.bb” and “u-boot-faux-debug_2014.04.bb” ?  That should
> work but does that make sense to do?


As the stock u-boot appears to be sufficient I'd say use that and then add
a u-boot-faux-debug_2014.04.bb recipe that just does "include
u-boot-${PV}.bb" and then whatever SRC_URI changes you want in that version.

Hopefully you don't need to make any fiddly changes to the recipe as PN is
now different.

Ross

[-- Attachment #2: Type: text/html, Size: 1451 bytes --]

Re: Prescribed way to make global variables in recipes?

[Andy Gikling]

[-- Attachment #1: Type: text/plain, Size: 2630 bytes --]

This doesn’t seem to be a viable option.  I’m still missing something I think.

We are adding our own layer on top of our soc vendor’s layer.  In their layer there is a specialized u-boot recipe for the imx6 we are using.

So I try to make a recipe for uboot (u-boot-faux_2014.04.bb<http://u-boot-faux_2014.04.bb>) that includes the vendor’s uboot recipe with “include recipes-bsp/u-boot/u-boot_2014.04.bb” as suggested and try to build an image that is going to IMAGE_INSTALL my customized version of u-boot instead of the soc vendor’s.  However, I get an error at bitbake parse time saying:

ERROR: Multiple .bb files are due to be built which each provide u-boot

This makes sense because bitbake sees that yes, in fact now there are two recipes that provide u-boot.  I suppose I can do something like use “PREFERRED_PROVIDER_u-boot” in my local.conf, but now I need to change my local.conf any time I want to build a different image (ie bitbake faux-app and bitbake faux-app-debug).  I really want a workflow that doesn’t require me to change configuration files, instead just bitbake different image recipes.

In this project I also need to conditionally patch the kernel and I’m going to have this same problem with multiple kernel providers as well.  Also, I don’t want to remove our soc vendor’s layer to get around this error.  Their layer sets up the machine and all sorts of other things.  If I got rid of it I would need to build all that functionality into my project’s layer…  or is this what I’m going to need to do?

Thoughts?

~Andy

From: Burton, Ross [mailto:ross.burton@intel.com]
Sent: Friday, April 08, 2016 10:10 AM
To: Andy Gikling
Cc: yocto@yoctoproject.org
Subject: Re: [yocto] Prescribed way to make global variables in recipes?


On 8 April 2016 at 16:05, Andy Gikling <agikling@minnetronix.com<mailto:agikling@minnetronix.com>> wrote:
So what do I do in that case? Just make two of my own versions of the u-boot_2014.04.bb<http://u-boot_2014.04.bb> file, and give them different names?  For example “u-boot-faux_2014.04.bb<http://u-boot-faux_2014.04.bb>” and “u-boot-faux-debug_2014.04.bb<http://u-boot-faux-debug_2014.04.bb>” ?  That should work but does that make sense to do?

As the stock u-boot appears to be sufficient I'd say use that and then add a u-boot-faux-debug_2014.04.bb<http://u-boot-faux-debug_2014.04.bb> recipe that just does "include u-boot-${PV}.bb" and then whatever SRC_URI changes you want in that version.

Hopefully you don't need to make any fiddly changes to the recipe as PN is now different.

Ross


[-- Attachment #2: Type: text/html, Size: 7544 bytes --]

Re: Prescribed way to make global variables in recipes?

[Andre McCurdy]

On Fri, Apr 8, 2016 at 10:22 AM, Andy Gikling <agikling@minnetronix.com> wrote:
> This doesn’t seem to be a viable option.  I’m still missing something I
> think.
>
> We are adding our own layer on top of our soc vendor’s layer.  In their
> layer there is a specialized u-boot recipe for the imx6 we are using.
>
> So I try to make a recipe for uboot (u-boot-faux_2014.04.bb) that includes
> the vendor’s uboot recipe with “include
> recipes-bsp/u-boot/u-boot_2014.04.bb” as suggested and try to build an image
> that is going to IMAGE_INSTALL my customized version of u-boot instead of
> the soc vendor’s.  However, I get an error at bitbake parse time saying:
>
> ERROR: Multiple .bb files are due to be built which each provide u-boot
>
> This makes sense because bitbake sees that yes, in fact now there are two
> recipes that provide u-boot.

I guess the problem here is not that you have two versions of u-boot
but that you are trying to include both of them at once - e.g. one you
added explicitly via IMAGE_INSTALL and one that the machine config
probably added via MACHINE_EXTRA_RDEPENDS or similar. To fix that,
you'll need to find a way to disable the dependency being added via
the machine config (it's not a perfect solution though because if you
then try to build an image to which you have not explicitly added a
u-boot dependency too you'll end up with a build which doesn't contain
any u-boot image at all...).

> I suppose I can do something like use
> “PREFERRED_PROVIDER_u-boot” in my local.conf, but now I need to change my
> local.conf any time I want to build a different image (ie bitbake faux-app
> and bitbake faux-app-debug).  I really want a workflow that doesn’t require
> me to change configuration files, instead just bitbake different image
> recipes.
>
> In this project I also need to conditionally patch the kernel and I’m going
> to have this same problem with multiple kernel providers as well.

If you want to change the kernel too then maybe your best option is
just to define a new machine config (you can use a similar approach as
you used for the u-boot recipe of including the original and then
over-riding just the options you want to change).

Since the machine config is a global config file which affects all
recipes, you can also use it to set the global variables etc you were
planning to use to conditionally apply u-boot patches, so you would no
longer need multiple u-boot recipes or multiple images.

> Also, I
> don’t want to remove our soc vendor’s layer to get around this error.  Their
> layer sets up the machine and all sorts of other things.  If I got rid of it
> I would need to build all that functionality into my project’s layer…  or is
> this what I’m going to need to do?
>
>
>
> Thoughts?
>
>
> ~Andy
>
>
>
> From: Burton, Ross [mailto:ross.burton@intel.com]
> Sent: Friday, April 08, 2016 10:10 AM
> To: Andy Gikling
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] Prescribed way to make global variables in recipes?
>
>
>
>
>
> On 8 April 2016 at 16:05, Andy Gikling <agikling@minnetronix.com> wrote:
>
> So what do I do in that case? Just make two of my own versions of the
> u-boot_2014.04.bb file, and give them different names?  For example
> “u-boot-faux_2014.04.bb” and “u-boot-faux-debug_2014.04.bb” ?  That should
> work but does that make sense to do?
>
>
> As the stock u-boot appears to be sufficient I'd say use that and then add a
> u-boot-faux-debug_2014.04.bb recipe that just does "include u-boot-${PV}.bb"
> and then whatever SRC_URI changes you want in that version.
>
>
>
> Hopefully you don't need to make any fiddly changes to the recipe as PN is
> now different.
>
>
>
> Ross
>
>
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>