[PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Jacob Kroon]

Stating that the assignment is done at the end of parsing is misleading.
The weak default value is the value which a variable will expand to if no value
has been assigned to it using any of the assignment operators.

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
---

Changes in v3:
* Update commit message
* Change "... via one of ..." to "... via any of ..."

Changes in v2:
* Add comment to line with immediate variable expansion

 .../bitbake-user-manual-metadata.rst          | 31 +++++++++----------
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
index af4b1358..45355b6c 100644
--- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
+++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
@@ -195,22 +195,21 @@ value. However, if ``A`` is not set, the variable is set to "aval".
 Setting a weak default value (??=)
 ----------------------------------
 
-It is possible to use a "weaker" assignment than in the previous section
-by using the "??=" operator. This assignment behaves identical to "?="
-except that the assignment is made at the end of the parsing process
-rather than immediately. Consequently, when multiple "??=" assignments
-exist, the last one is used. Also, any "=" or "?=" assignment will
-override the value set with "??=". Here is an example::
-
-   A ??= "somevalue"
-   A ??= "someothervalue"
-
-If ``A`` is set before the above statements are
-parsed, the variable retains its value. If ``A`` is not set, the
-variable is set to "someothervalue".
-
-Again, this assignment is a "lazy" or "weak" assignment because it does
-not occur until the end of the parsing process.
+The weak default value of a variable is the value which that variable
+will expand to if no value has been assigned to it via any of the other
+assignment operators. Here is an example::
+
+   W ??= "x"
+   A := "${W}" # Immediate variable expansion
+   W ??= "y"
+   B = "${W}"
+   W ?= "z"
+
+After parsing we will have::
+
+   A = "x"
+   B = "z"
+   W = "z"
 
 Immediate variable expansion (:=)
 ---------------------------------

Re: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Michael Opdenacker]

Hi Jacob

On 6/10/22 21:08, Jacob Kroon wrote:
> Stating that the assignment is done at the end of parsing is misleading.
> The weak default value is the value which a variable will expand to if no value
> has been assigned to it using any of the assignment operators.
>
> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>


Many thanks for this update!
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

RE: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Peter Kjellerstedt]

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-
> devel@lists.openembedded.org> On Behalf Of Jacob Kroon
> Sent: den 10 juni 2022 21:09
> To: bitbake-devel@lists.openembedded.org
> Cc: docs@lists.yoctoproject.org
> Subject: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct
> description of the ??= operator
> 
> Stating that the assignment is done at the end of parsing is misleading.
> The weak default value is the value which a variable will expand to if no
> value
> has been assigned to it using any of the assignment operators.
> 
> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
> ---
> 
> Changes in v3:
> * Update commit message
> * Change "... via one of ..." to "... via any of ..."
> 
> Changes in v2:
> * Add comment to line with immediate variable expansion
> 
>  .../bitbake-user-manual-metadata.rst          | 31 +++++++++----------
>  1 file changed, 15 insertions(+), 16 deletions(-)
> 
> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> index af4b1358..45355b6c 100644
> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> @@ -195,22 +195,21 @@ value. However, if ``A`` is not set, the variable is
> set to "aval".
>  Setting a weak default value (??=)
>  ----------------------------------
> 
> -It is possible to use a "weaker" assignment than in the previous section
> -by using the "??=" operator. This assignment behaves identical to "?="
> -except that the assignment is made at the end of the parsing process
> -rather than immediately. Consequently, when multiple "??=" assignments
> -exist, the last one is used. Also, any "=" or "?=" assignment will
> -override the value set with "??=". Here is an example::
> -
> -   A ??= "somevalue"
> -   A ??= "someothervalue"
> -
> -If ``A`` is set before the above statements are
> -parsed, the variable retains its value. If ``A`` is not set, the
> -variable is set to "someothervalue".
> -
> -Again, this assignment is a "lazy" or "weak" assignment because it does
> -not occur until the end of the parsing process.
> +The weak default value of a variable is the value which that variable
> +will expand to if no value has been assigned to it via any of the other
> +assignment operators. Here is an example::
> +
> +   W ??= "x"
> +   A := "${W}" # Immediate variable expansion
> +   W ??= "y"
> +   B = "${W}"
> +   W ?= "z"
> +
> +After parsing we will have::
> +
> +   A = "x"
> +   B = "z"
> +   W = "z"
> 
>  Immediate variable expansion (:=)
>  ---------------------------------

I think the new description is missing some of the information of the 
original description, e.g., "when multiple "??=" assignments exist, the 
last one is used". So, e.g.:

W ??= "x"
W ??= "y"

gives

W = "y"

Another interesting case that is easy to get wrong is:

W ??= "x"
W += "y"

that gives

W = " y"
	
and which should be compared to

W ??= "x"
W:append = " y"

that gives

W = "x y"

//Peter

Re: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Jacob Kroon]

Hi Peter,

Thanks for the comments, I don't claim to be a data_smart.py expert so 
bear with me below:

On 6/13/22 18:57, Peter Kjellerstedt wrote:
>> -----Original Message-----
>> From: bitbake-devel@lists.openembedded.org <bitbake-
>> devel@lists.openembedded.org> On Behalf Of Jacob Kroon
>> Sent: den 10 juni 2022 21:09
>> To: bitbake-devel@lists.openembedded.org
>> Cc: docs@lists.yoctoproject.org
>> Subject: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct
>> description of the ??= operator
>>
>> Stating that the assignment is done at the end of parsing is misleading.
>> The weak default value is the value which a variable will expand to if no
>> value
>> has been assigned to it using any of the assignment operators.
>>
>> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
>> ---
>>
>> Changes in v3:
>> * Update commit message
>> * Change "... via one of ..." to "... via any of ..."
>>
>> Changes in v2:
>> * Add comment to line with immediate variable expansion
>>
>>   .../bitbake-user-manual-metadata.rst          | 31 +++++++++----------
>>   1 file changed, 15 insertions(+), 16 deletions(-)
>>
>> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> index af4b1358..45355b6c 100644
>> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>> @@ -195,22 +195,21 @@ value. However, if ``A`` is not set, the variable is
>> set to "aval".
>>   Setting a weak default value (??=)
>>   ----------------------------------
>>
>> -It is possible to use a "weaker" assignment than in the previous section
>> -by using the "??=" operator. This assignment behaves identical to "?="
>> -except that the assignment is made at the end of the parsing process
>> -rather than immediately. Consequently, when multiple "??=" assignments
>> -exist, the last one is used. Also, any "=" or "?=" assignment will
>> -override the value set with "??=". Here is an example::
>> -
>> -   A ??= "somevalue"
>> -   A ??= "someothervalue"
>> -
>> -If ``A`` is set before the above statements are
>> -parsed, the variable retains its value. If ``A`` is not set, the
>> -variable is set to "someothervalue".
>> -
>> -Again, this assignment is a "lazy" or "weak" assignment because it does
>> -not occur until the end of the parsing process.
>> +The weak default value of a variable is the value which that variable
>> +will expand to if no value has been assigned to it via any of the other
>> +assignment operators. Here is an example::
>> +
>> +   W ??= "x"
>> +   A := "${W}" # Immediate variable expansion
>> +   W ??= "y"
>> +   B = "${W}"
>> +   W ?= "z"
>> +
>> +After parsing we will have::
>> +
>> +   A = "x"
>> +   B = "z"
>> +   W = "z"
>>
>>   Immediate variable expansion (:=)
>>   ---------------------------------
> 
> I think the new description is missing some of the information of the
> original description, e.g., "when multiple "??=" assignments exist, the
> last one is used". So, e.g.:
> 
> W ??= "x"
> W ??= "y"
> 
> gives
> 
> W = "y"
> 

I think saying

"when multiple "??=" assignments exist, the last one is used"

is too vague because it depends on when and how you reference the 
variable. It is not always the "last one" that ends up "being used".

How about something like:

"The ??= assignment operator takes effect immediately, overriding any 
previously defined weak default value"

?

> Another interesting case that is easy to get wrong is:
> 
> W ??= "x"
> W += "y"
> 
> that gives
> 
> W = " y"
> 	

Given the definition of the weak default value and the fact that the += 
operator does not involve expanding the LHS before appending, this is 
the result I would expect. Should we add an example to the docs ?

> and which should be compared to
> 
> W ??= "x"
> W:append = " y"
> 
> that gives
> 
> W = "x y"
> 

The docs states overrides are applied "at variable expansion time". But 
if the new definition of a weak default value that I am proposing is 
true, the variable content is expanded first, then the overrides are 
applied, in order to match the result above. But is that true ? I'm not 
that acquainted with data_smart.py to have the confidence in saying so.

Richard or any other data_smart.py expert, can you shed some light on 
all of this ?

Jacob

Re: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Richard Purdie]

On Tue, 2022-06-14 at 09:46 +0200, Jacob Kroon wrote:
> Hi Peter,
> 
> Thanks for the comments, I don't claim to be a data_smart.py expert so 
> bear with me below:
> 
> On 6/13/22 18:57, Peter Kjellerstedt wrote:
> > > -----Original Message-----
> > > From: bitbake-devel@lists.openembedded.org <bitbake-
> > > devel@lists.openembedded.org> On Behalf Of Jacob Kroon
> > > Sent: den 10 juni 2022 21:09
> > > To: bitbake-devel@lists.openembedded.org
> > > Cc: docs@lists.yoctoproject.org
> > > Subject: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct
> > > description of the ??= operator
> > > 
> > > Stating that the assignment is done at the end of parsing is misleading.
> > > The weak default value is the value which a variable will expand to if no
> > > value
> > > has been assigned to it using any of the assignment operators.
> > > 
> > > Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
> > > ---
> > > 
> > > Changes in v3:
> > > * Update commit message
> > > * Change "... via one of ..." to "... via any of ..."
> > > 
> > > Changes in v2:
> > > * Add comment to line with immediate variable expansion
> > > 
> > >   .../bitbake-user-manual-metadata.rst          | 31 +++++++++----------
> > >   1 file changed, 15 insertions(+), 16 deletions(-)
> > > 
> > > diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > index af4b1358..45355b6c 100644
> > > --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
> > > @@ -195,22 +195,21 @@ value. However, if ``A`` is not set, the variable is
> > > set to "aval".
> > >   Setting a weak default value (??=)
> > >   ----------------------------------
> > > 
> > > -It is possible to use a "weaker" assignment than in the previous section
> > > -by using the "??=" operator. This assignment behaves identical to "?="
> > > -except that the assignment is made at the end of the parsing process
> > > -rather than immediately. Consequently, when multiple "??=" assignments
> > > -exist, the last one is used. Also, any "=" or "?=" assignment will
> > > -override the value set with "??=". Here is an example::
> > > -
> > > -   A ??= "somevalue"
> > > -   A ??= "someothervalue"
> > > -
> > > -If ``A`` is set before the above statements are
> > > -parsed, the variable retains its value. If ``A`` is not set, the
> > > -variable is set to "someothervalue".
> > > -
> > > -Again, this assignment is a "lazy" or "weak" assignment because it does
> > > -not occur until the end of the parsing process.
> > > +The weak default value of a variable is the value which that variable
> > > +will expand to if no value has been assigned to it via any of the other
> > > +assignment operators. Here is an example::
> > > +
> > > +   W ??= "x"
> > > +   A := "${W}" # Immediate variable expansion
> > > +   W ??= "y"
> > > +   B = "${W}"
> > > +   W ?= "z"
> > > +
> > > +After parsing we will have::
> > > +
> > > +   A = "x"
> > > +   B = "z"
> > > +   W = "z"
> > > 
> > >   Immediate variable expansion (:=)
> > >   ---------------------------------
> > 
> > I think the new description is missing some of the information of the
> > original description, e.g., "when multiple "??=" assignments exist, the
> > last one is used". So, e.g.:
> > 
> > W ??= "x"
> > W ??= "y"
> > 
> > gives
> > 
> > W = "y"
> > 
> 
> I think saying
> 
> "when multiple "??=" assignments exist, the last one is used"
> 
> is too vague because it depends on when and how you reference the 
> variable. It is not always the "last one" that ends up "being used".

The last value set during parsing order is used as the weak default
value.

> How about something like:
> 
> "The ??= assignment operator takes effect immediately, overriding any 
> previously defined weak default value"

I do think you need to spell this out with an example as it isn't what
people expect.

> > Another interesting case that is easy to get wrong is:
> > 
> > W ??= "x"
> > W += "y"
> > 
> > that gives
> > 
> > W = " y"
> > 	
> 
> Given the definition of the weak default value and the fact that the += 
> operator does not involve expanding the LHS before appending, this is 
> the result I would expect. Should we add an example to the docs ?

Yes, definitely. Whilst it makes sense, people often don't expect it.

> > and which should be compared to
> > 
> > W ??= "x"
> > W:append = " y"
> > 
> > that gives
> > 
> > W = "x y"
> > 
> 
> The docs states overrides are applied "at variable expansion time". But 
> if the new definition of a weak default value that I am proposing is 
> true, the variable content is expanded first, then the overrides are 
> applied, in order to match the result above. But is that true ? I'm not 
> that acquainted with data_smart.py to have the confidence in saying so.
> 
> Richard or any other data_smart.py expert, can you shed some light on 
> all of this ?

??= is the value given to a variable if nothing else sets it.

In the above example it is given a weak default value of "x". " y" is
then appended to the base value which would be "x" since nothing else
sets it. Again, it does make sense we need to have examples in the
manual to make it clear.

Cheers,

Richard

Re: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct description of the ??= operator

[Jacob Kroon]

Hi Richard,

On 6/14/22 09:57, Richard Purdie wrote:
> On Tue, 2022-06-14 at 09:46 +0200, Jacob Kroon wrote:
>> Hi Peter,
>>
>> Thanks for the comments, I don't claim to be a data_smart.py expert so
>> bear with me below:
>>
>> On 6/13/22 18:57, Peter Kjellerstedt wrote:
>>>> -----Original Message-----
>>>> From: bitbake-devel@lists.openembedded.org <bitbake-
>>>> devel@lists.openembedded.org> On Behalf Of Jacob Kroon
>>>> Sent: den 10 juni 2022 21:09
>>>> To: bitbake-devel@lists.openembedded.org
>>>> Cc: docs@lists.yoctoproject.org
>>>> Subject: [bitbake-devel] [PATCH v3] bitbake-user-manual: Correct
>>>> description of the ??= operator
>>>>
>>>> Stating that the assignment is done at the end of parsing is misleading.
>>>> The weak default value is the value which a variable will expand to if no
>>>> value
>>>> has been assigned to it using any of the assignment operators.
>>>>
>>>> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
>>>> ---
>>>>
>>>> Changes in v3:
>>>> * Update commit message
>>>> * Change "... via one of ..." to "... via any of ..."
>>>>
>>>> Changes in v2:
>>>> * Add comment to line with immediate variable expansion
>>>>
>>>>    .../bitbake-user-manual-metadata.rst          | 31 +++++++++----------
>>>>    1 file changed, 15 insertions(+), 16 deletions(-)
>>>>
>>>> diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> index af4b1358..45355b6c 100644
>>>> --- a/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst
>>>> @@ -195,22 +195,21 @@ value. However, if ``A`` is not set, the variable is
>>>> set to "aval".
>>>>    Setting a weak default value (??=)
>>>>    ----------------------------------
>>>>
>>>> -It is possible to use a "weaker" assignment than in the previous section
>>>> -by using the "??=" operator. This assignment behaves identical to "?="
>>>> -except that the assignment is made at the end of the parsing process
>>>> -rather than immediately. Consequently, when multiple "??=" assignments
>>>> -exist, the last one is used. Also, any "=" or "?=" assignment will
>>>> -override the value set with "??=". Here is an example::
>>>> -
>>>> -   A ??= "somevalue"
>>>> -   A ??= "someothervalue"
>>>> -
>>>> -If ``A`` is set before the above statements are
>>>> -parsed, the variable retains its value. If ``A`` is not set, the
>>>> -variable is set to "someothervalue".
>>>> -
>>>> -Again, this assignment is a "lazy" or "weak" assignment because it does
>>>> -not occur until the end of the parsing process.
>>>> +The weak default value of a variable is the value which that variable
>>>> +will expand to if no value has been assigned to it via any of the other
>>>> +assignment operators. Here is an example::
>>>> +
>>>> +   W ??= "x"
>>>> +   A := "${W}" # Immediate variable expansion
>>>> +   W ??= "y"
>>>> +   B = "${W}"
>>>> +   W ?= "z"
>>>> +
>>>> +After parsing we will have::
>>>> +
>>>> +   A = "x"
>>>> +   B = "z"
>>>> +   W = "z"
>>>>
>>>>    Immediate variable expansion (:=)
>>>>    ---------------------------------
>>>
>>> I think the new description is missing some of the information of the
>>> original description, e.g., "when multiple "??=" assignments exist, the
>>> last one is used". So, e.g.:
>>>
>>> W ??= "x"
>>> W ??= "y"
>>>
>>> gives
>>>
>>> W = "y"
>>>
>>
>> I think saying
>>
>> "when multiple "??=" assignments exist, the last one is used"
>>
>> is too vague because it depends on when and how you reference the
>> variable. It is not always the "last one" that ends up "being used".
> 
> The last value set during parsing order is used as the weak default
> value.
> 
>> How about something like:
>>
>> "The ??= assignment operator takes effect immediately, overriding any
>> previously defined weak default value"
> 
> I do think you need to spell this out with an example as it isn't what
> people expect.
> 
>>> Another interesting case that is easy to get wrong is:
>>>
>>> W ??= "x"
>>> W += "y"
>>>
>>> that gives
>>>
>>> W = " y"
>>> 	
>>
>> Given the definition of the weak default value and the fact that the +=
>> operator does not involve expanding the LHS before appending, this is
>> the result I would expect. Should we add an example to the docs ?
> 
> Yes, definitely. Whilst it makes sense, people often don't expect it.
> 
>>> and which should be compared to
>>>
>>> W ??= "x"
>>> W:append = " y"
>>>
>>> that gives
>>>
>>> W = "x y"
>>>
>>
>> The docs states overrides are applied "at variable expansion time". But
>> if the new definition of a weak default value that I am proposing is
>> true, the variable content is expanded first, then the overrides are
>> applied, in order to match the result above. But is that true ? I'm not
>> that acquainted with data_smart.py to have the confidence in saying so.
>>
>> Richard or any other data_smart.py expert, can you shed some light on
>> all of this ?
> 
> ??= is the value given to a variable if nothing else sets it.
> 

The problem I have with the above formulation is that "value given to a 
variable" does not mean that the variable will be assigned that value, 
which is why I prefer to use "expand to", which makes it clear that 
there is no assignment going on.

> In the above example it is given a weak default value of "x". " y" is
> then appended to the base value which would be "x" since nothing else
> sets it. Again, it does make sense we need to have examples in the
> manual to make it clear.
> 
> Cheers,
> 
> Richard

Jacob