[yocto] Bitbake immediate assignment behavior

[yocto] Bitbake immediate assignment behavior

[Alexandru N. Onea]

[-- Attachment #1: Type: text/plain, Size: 2168 bytes --]

Hello community,

Let's consider the following example in a bb file:

X = "1"
A := "X is '${X}' and Y is '${Y}'"
B = "X is '${X}' and Y is '${Y}'"
X = "2"
Y = "3"

According to the bitbake user manual, section 3.1.7 [1], the expected
values for A and B after parsing is done, are

A="X is '1' and Y is ''"
B="X is '2' and Y is '3'"

meaning that, since Y is not defined at the time of parsing the
assignment to A, it is replaced with blank, while X is substituted with the
value that it had at that time. On the other hand, B will substitute the
last defined values for both X and Y.

However, if I run the following example through bitbake, I get the
following result:

A="X is '1' and Y is '3'"
B="X is '2' and Y is '3'"

Although unexpected and a bit counter-intuitive at first, the result is
easily explained by the fact that, according to section 3.1.4 [2], an
undefined variable expansion results in keeping the actual literal string
in place, in our case "${Y}", which probably leads, at the end of the
parsing, to a re-evaluation of A and, since now Y is defined, the
substitution of that literal with the last value that has been assigned to
Y (i.e. 3).

Question: is this behavior intentional? Or is it a bug? The documentation
for the immediate assignment states that we should expect undefined
variables to be substituted with blank while in practice they obey the
rules of the simple assignments.

Should at least the documentation be updated?

PS. I ran the same example through GNU Make since it also implements the
concept of immediate assignment and make does replace undefined variables
with blank. It seems that bitbake has a different perspective on this
concept.

Looking forward to your feedback on this.

PS2. The question might be too bitbake specific for this list. Let me know
if I should post it to the bitbake dedicated list instead.

[1]
https://www.yoctoproject.org/docs/latest/bitbake-user-manual/bitbake-user-manual.html#immediate-variable-expansion
[2]
https://www.yoctoproject.org/docs/latest/bitbake-user-manual/bitbake-user-manual.html#variable-expansion

Alexandru N. Onea

[-- Attachment #2: Type: text/html, Size: 3128 bytes --]

Re: [yocto] Bitbake immediate assignment behavior

[Richard Purdie]

Hi,

On Thu, 2020-01-30 at 13:32 +0200, Alexandru N. Onea wrote:
> Let's consider the following example in a bb file:
> 
> X = "1"
> A := "X is '${X}' and Y is '${Y}'"
> B = "X is '${X}' and Y is '${Y}'"
> X = "2"
> Y = "3"
> 
> According to the bitbake user manual, section 3.1.7 [1], the expected
> values for A and B after parsing is done, are 
> 
> A="X is '1' and Y is ''"
> B="X is '2' and Y is '3'"
> 
> meaning that, since Y is not defined at the time of parsing the
> assignment to A, it is replaced with blank, while X is substituted
> with the value that it had at that time. On the other hand, B will
> substitute the last defined values for both X and Y.
> 
> However, if I run the following example through bitbake, I get the
> following result:
> 
> A="X is '1' and Y is '3'"
> B="X is '2' and Y is '3'"
> 
> Although unexpected and a bit counter-intuitive at first, the result
> is easily explained by the fact that, according to section 3.1.4 [2],
> an undefined variable expansion results in keeping the actual literal
> string in place, in our case "${Y}", which probably leads, at the end
> of the parsing, to a re-evaluation of A and, since now Y is defined,
> the substitution of that literal with the last value that has been
> assigned to Y (i.e. 3).
> 
> Question: is this behavior intentional? Or is it a bug? The
> documentation for the immediate assignment states that we should
> expect undefined variables to be substituted with blank while in
> practice they obey the rules of the simple assignments.

I'd say bitbake is behaving as it was written to behave. I'm not sure
its ideal but we probably do rely on it in places.

> Should at least the documentation be updated?

Yes, and we should add some tests for it to better "document" it.

There is a patch on the bitbake list which tweaks the docs already but
I'd like to fix the example too.

Cheers,

Richard

Re: [yocto] Bitbake immediate assignment behavior

[Alexandru N. Onea]

[-- Attachment #1: Type: text/plain, Size: 3547 bytes --]

Hi Richard,

Thank you for the clarification.

I am definitely not in a position to make any suggestions regarding how
bitbake should behave, however I would like to at least give my view:
    * to me, the reason for having an immediate assignment seems to be
related (among others, maybe) to the fact that at times we want / need to
be certain about the value of a variable, and we can achieve that by
forcing the assignment to use "known" values (i.e. values that are known at
the moment of the parsing) instead of relying on future values. In short,
in my opinion, the immediate assignment aims to guarantee that the value of
a variable depends ONLY on known values at the time of the parsing, and not
on future, uncertain values.
    * the current behavior of the immediate assignment directly contradicts
the above by introducing the very same "uncertainty" which we try to avoid
by using it in the first place. That is, the immediate assignment cannot
guarantee that the value of the assigned variable depends ONLY on values
known at the time of the parsing.

Of course, it is also possible that the use of immediate assignments in
practical contexts is not so much driven by the need for the above
guarantee, but instead used in ways that I don't see at the moment. I would
like to learn more about how this assignment is used in practice.

Thanks again,
Alexandru Onea


On Thu, 30 Jan 2020 at 14:04, Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> Hi,
>
> On Thu, 2020-01-30 at 13:32 +0200, Alexandru N. Onea wrote:
> > Let's consider the following example in a bb file:
> >
> > X = "1"
> > A := "X is '${X}' and Y is '${Y}'"
> > B = "X is '${X}' and Y is '${Y}'"
> > X = "2"
> > Y = "3"
> >
> > According to the bitbake user manual, section 3.1.7 [1], the expected
> > values for A and B after parsing is done, are
> >
> > A="X is '1' and Y is ''"
> > B="X is '2' and Y is '3'"
> >
> > meaning that, since Y is not defined at the time of parsing the
> > assignment to A, it is replaced with blank, while X is substituted
> > with the value that it had at that time. On the other hand, B will
> > substitute the last defined values for both X and Y.
> >
> > However, if I run the following example through bitbake, I get the
> > following result:
> >
> > A="X is '1' and Y is '3'"
> > B="X is '2' and Y is '3'"
> >
> > Although unexpected and a bit counter-intuitive at first, the result
> > is easily explained by the fact that, according to section 3.1.4 [2],
> > an undefined variable expansion results in keeping the actual literal
> > string in place, in our case "${Y}", which probably leads, at the end
> > of the parsing, to a re-evaluation of A and, since now Y is defined,
> > the substitution of that literal with the last value that has been
> > assigned to Y (i.e. 3).
> >
> > Question: is this behavior intentional? Or is it a bug? The
> > documentation for the immediate assignment states that we should
> > expect undefined variables to be substituted with blank while in
> > practice they obey the rules of the simple assignments.
>
> I'd say bitbake is behaving as it was written to behave. I'm not sure
> its ideal but we probably do rely on it in places.
>
> > Should at least the documentation be updated?
>
> Yes, and we should add some tests for it to better "document" it.
>
> There is a patch on the bitbake list which tweaks the docs already but
> I'd like to fix the example too.
>
> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 4535 bytes --]