[yocto][bitbake] variable expansion in files

[yocto][bitbake] variable expansion in files

[Yann CARDAILLAC Pro]

[-- Attachment #1: Type: text/plain, Size: 1902 bytes --]

Hi,

There's something that has always been missing in my experience of Yocto,
it's the ability to simply configure files with yocto variables.

Basically the idea is to have a class that will automatically sed all
variables from yocto inside a file using a delimiter, or to reuse the way
it is done with .wks.in with the ${var} syntax, even if I can already see
some limitations with shell scripts.

What I intend to do is to provide some variables to a recipe and then to
use them to complete a template.

For instance I'd like to put some stuff inside my machine.conf and want
those variables to be filled inside my templated script.

Is there any official way to do this?

As far as I can see there's not, the only examples I've found are using
hand made seds. But I might not be looking in the right place.

If I'm right, I'm wondering why this is not generalized inside a poky
class. I've found this meta :
https://github.com/coreycothrum/meta-bitbake-variable-substitution doing
exactly what I needed, but since it's not mainline I assume that as always
there's a good reason why it's not something you want to have in mainline.

If that's only because there wasn't any submission and If you're up for it
going mainline, I can work (with coreycothrum if he wants to?) it through
the patch submission process.

Best regards,

Yann CARDAILLAC,
--------------------------------------------------------------------------------------------------------------------
Embedded Linux Engineer
Embedded Systems, Computer Science & Electronics, @Polytech Sorbone
--------------------------------------------------------------------------------------------------------------------
Phone : +33669909858
--------------------------------------------------------------------------------------------------------------------
Linkedin : https://www.linkedin.com/in/yann-c-39b64012b/
Website: https://ycardaillac.fr

[-- Attachment #2: Type: text/html, Size: 2674 bytes --]

Re: [OE-core] [yocto][bitbake] variable expansion in files

[Ross Burton]

On 21 Jul 2023, at 10:24, Yann CARDAILLAC via lists.openembedded.org <ycnakajsph=gmail.com@lists.openembedded.org> wrote:
> There's something that has always been missing in my experience of Yocto, it's the ability to simply configure files with yocto variables.
> 
> Basically the idea is to have a class that will automatically sed all variables from yocto inside a file using a delimiter, or to reuse the way it is done with .wks.in with the ${var} syntax, even if I can already see some limitations with shell scripts.
> 
> What I intend to do is to provide some variables to a recipe and then to use them to complete a template. 
> 
> For instance I'd like to put some stuff inside my machine.conf and want those variables to be filled inside my templated script.
> 
> Is there any official way to do this?
> 
> As far as I can see there's not, the only examples I've found are using hand made seds. But I might not be looking in the right place.
> 
> If I'm right, I'm wondering why this is not generalized inside a poky class. I've found this meta : https://github.com/coreycothrum/meta-bitbake-variable-substitution doing exactly what I needed, but since it's not mainline I assume that as always there's a good reason why it's not something you want to have in mainline.
> 
> If that's only because there wasn't any submission and If you're up for it going mainline, I can work (with coreycothrum if he wants to?) it through the patch submission process.

There’s also a class that I hacked up a while ago that I never finished because I wanted to automatically handle variable dependencies, otherwise it’s no good:

https://github.com/rossburton/meta-ross/blob/master/classes/bbin.bbclass

I’d like to see something in core, but the scope should be relatively tight.  I don’t like the automatic expansion that meta-bitbake-variable-substitution does, I’m thinking of a pathological case where piglit wanted to use @@bindir@@ in a wrapper script: it would also attempt to process many gigabytes of installed files.

The bbin class _only_ works on files in SRC_URI, so foo.sh.bbin would be processed at do_unpack and end up in WORKDIR/foo.sh.  The scope is more limited but you know exactly what it’s doing.

Ross

Fwd: [OE-core] [yocto][bitbake] variable expansion in files

[Corey Cothrum]

[-- Attachment #1: Type: text/plain, Size: 3319 bytes --]

Can confirm that no attempt or submission was made to merge
meta-bitbake-variable-substitution into mainline/core. I'm happy if people
are finding it useful, but it was written very much for my use case(s). I
didn't spend much time designing it or thinking through how other people
may prefer it to work. Not to disparage it; I just mean to say there is
likely room for improvement.

Ross is also correct that automatic expansion shouldn't be the only mode of
operation. That's why meta-bitbake-variable-substitution has the option to
specify search and replace file paths:

${@bitbake_variables_search_and_sub("${PATH_OR_FILE_TO_PROCESS}",
r"${BITBAKE_VAR_SUB_DELIM}", d)}


I found cases where it was attempting variable substitution where I didn't
want it (e.g. when appending a recipe), and needed a method to be a bit
more surgical.

thanks,
Corey

On Fri, Jul 21, 2023 at 4:39 AM Ross Burton <Ross.Burton@arm.com> wrote:

> On 21 Jul 2023, at 10:24, Yann CARDAILLAC via lists.openembedded.org
> <ycnakajsph=gmail.com@lists.openembedded.org> wrote:
> > There's something that has always been missing in my experience of
> Yocto, it's the ability to simply configure files with yocto variables.
> >
> > Basically the idea is to have a class that will automatically sed all
> variables from yocto inside a file using a delimiter, or to reuse the way
> it is done with .wks.in with the ${var} syntax, even if I can already see
> some limitations with shell scripts.
> >
> > What I intend to do is to provide some variables to a recipe and then to
> use them to complete a template.
> >
> > For instance I'd like to put some stuff inside my machine.conf and want
> those variables to be filled inside my templated script.
> >
> > Is there any official way to do this?
> >
> > As far as I can see there's not, the only examples I've found are using
> hand made seds. But I might not be looking in the right place.
> >
> > If I'm right, I'm wondering why this is not generalized inside a poky
> class. I've found this meta :
> https://github.com/coreycothrum/meta-bitbake-variable-substitution doing
> exactly what I needed, but since it's not mainline I assume that as always
> there's a good reason why it's not something you want to have in mainline.
> >
> > If that's only because there wasn't any submission and If you're up for
> it going mainline, I can work (with coreycothrum if he wants to?) it
> through the patch submission process.
>
> There’s also a class that I hacked up a while ago that I never finished
> because I wanted to automatically handle variable dependencies, otherwise
> it’s no good:
>
> https://github.com/rossburton/meta-ross/blob/master/classes/bbin.bbclass
>
> I’d like to see something in core, but the scope should be relatively
> tight.  I don’t like the automatic expansion that
> meta-bitbake-variable-substitution does, I’m thinking of a pathological
> case where piglit wanted to use @@bindir@@ in a wrapper script: it would
> also attempt to process many gigabytes of installed files.
>
> The bbin class _only_ works on files in SRC_URI, so foo.sh.bbin would be
> processed at do_unpack and end up in WORKDIR/foo.sh.  The scope is more
> limited but you know exactly what it’s doing.
>
> Ross

[-- Attachment #2: Type: text/html, Size: 5260 bytes --]