Change in override behavior?

Change in override behavior?

[Gary Thomas]

I used to be able to use something like this:
   VARIABLE = "xyz"
and maybe in a .bbappend use
   VARIABLE_mymachine = "xyzzy"
where 'mymachine' is in OVERRIDES, which resulted in
   VARIABLE = "xyzzy"

This doesn't seem to work the same today, what I get is
   VARIABLE = "xyz"
   VARIABLE_mymachine = "xyzzy"

Has this changed (recently)?  Why?  How to I make use of overrides?

Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Change in override behavior?

[Paul Eggleton]

Hi Gary,

On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
> I used to be able to use something like this:
>    VARIABLE = "xyz"
> and maybe in a .bbappend use
>    VARIABLE_mymachine = "xyzzy"
> where 'mymachine' is in OVERRIDES, which resulted in
>    VARIABLE = "xyzzy"
> 
> This doesn't seem to work the same today, what I get is
>    VARIABLE = "xyz"
>    VARIABLE_mymachine = "xyzzy"
> 
> Has this changed (recently)?  Why?  How to I make use of overrides?
> 
> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28

When you say this is what you're seeing, how are you checking this? bitbake -e 
? Which part of that output are you looking at?

You *will* now see VARIABLE_mymachine in bitbake -e output even if "mymachine" 
is in OVERRIDES - that's because the overridden values stay in the datastore 
instead of a "finalisation" step eliminating them, in order to allow OVERRIDES 
to be changed dynamically (possibly multiple times) and still get the 
datastore contents you expect. That change was made in the 2.0 release 
(jethro). However That should not affect the final value of VARIABLE if 
"mymachine" is in OVERRIDES however. Are you sure you are seeing what you 
think you're seeing? If so, is the bbappend in fact being applied? Because if 
what you describe was really happening then pretty much the entire system 
would be broken. I just checked a similar case here and I see the expected 
results.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Change in override behavior?

[Gary Thomas]

On 2016-04-29 07:08, Paul Eggleton wrote:
> Hi Gary,
>
> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
>> I used to be able to use something like this:
>>     VARIABLE = "xyz"
>> and maybe in a .bbappend use
>>     VARIABLE_mymachine = "xyzzy"
>> where 'mymachine' is in OVERRIDES, which resulted in
>>     VARIABLE = "xyzzy"
>>
>> This doesn't seem to work the same today, what I get is
>>     VARIABLE = "xyz"
>>     VARIABLE_mymachine = "xyzzy"
>>
>> Has this changed (recently)?  Why?  How to I make use of overrides?
>>
>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
>
> When you say this is what you're seeing, how are you checking this? bitbake -e
> ? Which part of that output are you looking at?
>
> You *will* now see VARIABLE_mymachine in bitbake -e output even if "mymachine"
> is in OVERRIDES - that's because the overridden values stay in the datastore
> instead of a "finalisation" step eliminating them, in order to allow OVERRIDES
> to be changed dynamically (possibly multiple times) and still get the
> datastore contents you expect. That change was made in the 2.0 release
> (jethro). However That should not affect the final value of VARIABLE if
> "mymachine" is in OVERRIDES however. Are you sure you are seeing what you
> think you're seeing? If so, is the bbappend in fact being applied? Because if
> what you describe was really happening then pretty much the entire system
> would be broken. I just checked a similar case here and I see the expected
> results.

Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.

In my main recipe, I have this:
   SOME_OTHER_VARIABLE ?= "${MACHINE}"
and my .bbappend for that recipe has these lines:
   SOME_OTHER_VARIABLE_my-target-A = "something-else1"
   SOME_OTHER_VARIABLE_my-target-B = "something-else2"
and local.conf has
   MACHINE="my-target-B"

With Yocto 2.0 (Poky ceeb52a2544) I get this:
   SOME_OTHER_VARIABLE = "something-else2"
   SOME_OTHER_VARIABLE_my-target-A = "something-else1"
   SOME_OTHER_VARIABLE_my-target-B = "something-else2"

With today's (Poky a9b503b26) I get this:
   SOME_OTHER_VARIABLE = "my-target-B"
   SOME_OTHER_VARIABLE_my-target-A = "something-else1"
   SOME_OTHER_VARIABLE_my-target-B = "something-else2"

I also use this mechanism to set COMPATIBLE_MACHINE a lot and
that doesn't seem to be working the same either.

Am I doing something incorrectly (and just got away with it for years)?

n.b. If you're interested, I can share the .bb/.bbappend files
with you privately (I can't put them on the public list)

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Change in override behavior?

[Paul Eggleton]

On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
> On 2016-04-29 07:08, Paul Eggleton wrote:
> > On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
> >> I used to be able to use something like this:
> >>     VARIABLE = "xyz"
> >> 
> >> and maybe in a .bbappend use
> >> 
> >>     VARIABLE_mymachine = "xyzzy"
> >> 
> >> where 'mymachine' is in OVERRIDES, which resulted in
> >> 
> >>     VARIABLE = "xyzzy"
> >> 
> >> This doesn't seem to work the same today, what I get is
> >> 
> >>     VARIABLE = "xyz"
> >>     VARIABLE_mymachine = "xyzzy"
> >> 
> >> Has this changed (recently)?  Why?  How to I make use of overrides?
> >> 
> >> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
> > 
> > When you say this is what you're seeing, how are you checking this?
> > bitbake -e ? Which part of that output are you looking at?
> > 
> > You *will* now see VARIABLE_mymachine in bitbake -e output even if
> > "mymachine" is in OVERRIDES - that's because the overridden values stay
> > in the datastore instead of a "finalisation" step eliminating them, in
> > order to allow OVERRIDES to be changed dynamically (possibly multiple
> > times) and still get the datastore contents you expect. That change was
> > made in the 2.0 release (jethro). However That should not affect the
> > final value of VARIABLE if "mymachine" is in OVERRIDES however. Are you
> > sure you are seeing what you think you're seeing? If so, is the bbappend
> > in fact being applied? Because if what you describe was really happening
> > then pretty much the entire system would be broken. I just checked a
> > similar case here and I see the expected results.
> 
> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
> 
> In my main recipe, I have this:
>    SOME_OTHER_VARIABLE ?= "${MACHINE}"
> and my .bbappend for that recipe has these lines:
>    SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>    SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> and local.conf has
>    MACHINE="my-target-B"
> 
> With Yocto 2.0 (Poky ceeb52a2544) I get this:
>    SOME_OTHER_VARIABLE = "something-else2"
>    SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>    SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> 
> With today's (Poky a9b503b26) I get this:
>    SOME_OTHER_VARIABLE = "my-target-B"
>    SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>    SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> 
> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
> that doesn't seem to be working the same either.
> 
> Am I doing something incorrectly (and just got away with it for years)?
> 
> n.b. If you're interested, I can share the .bb/.bbappend files
> with you privately (I can't put them on the public list)

Ah, one thing... do your machine names have uppercase characters in them (as 
your pseudo-examples above do)? If so that was a 2.1 change; anything that is 
to be used as an override must now be lower case (so DISTRO, MACHINE, etc.)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Change in override behavior?

[Gary Thomas]

On 2016-04-29 09:16, Paul Eggleton wrote:
> On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
>> On 2016-04-29 07:08, Paul Eggleton wrote:
>>> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
>>>> I used to be able to use something like this:
>>>>      VARIABLE = "xyz"
>>>>
>>>> and maybe in a .bbappend use
>>>>
>>>>      VARIABLE_mymachine = "xyzzy"
>>>>
>>>> where 'mymachine' is in OVERRIDES, which resulted in
>>>>
>>>>      VARIABLE = "xyzzy"
>>>>
>>>> This doesn't seem to work the same today, what I get is
>>>>
>>>>      VARIABLE = "xyz"
>>>>      VARIABLE_mymachine = "xyzzy"
>>>>
>>>> Has this changed (recently)?  Why?  How to I make use of overrides?
>>>>
>>>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
>>>
>>> When you say this is what you're seeing, how are you checking this?
>>> bitbake -e ? Which part of that output are you looking at?
>>>
>>> You *will* now see VARIABLE_mymachine in bitbake -e output even if
>>> "mymachine" is in OVERRIDES - that's because the overridden values stay
>>> in the datastore instead of a "finalisation" step eliminating them, in
>>> order to allow OVERRIDES to be changed dynamically (possibly multiple
>>> times) and still get the datastore contents you expect. That change was
>>> made in the 2.0 release (jethro). However That should not affect the
>>> final value of VARIABLE if "mymachine" is in OVERRIDES however. Are you
>>> sure you are seeing what you think you're seeing? If so, is the bbappend
>>> in fact being applied? Because if what you describe was really happening
>>> then pretty much the entire system would be broken. I just checked a
>>> similar case here and I see the expected results.
>>
>> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
>>
>> In my main recipe, I have this:
>>     SOME_OTHER_VARIABLE ?= "${MACHINE}"
>> and my .bbappend for that recipe has these lines:
>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>> and local.conf has
>>     MACHINE="my-target-B"
>>
>> With Yocto 2.0 (Poky ceeb52a2544) I get this:
>>     SOME_OTHER_VARIABLE = "something-else2"
>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>
>> With today's (Poky a9b503b26) I get this:
>>     SOME_OTHER_VARIABLE = "my-target-B"
>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>
>> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
>> that doesn't seem to be working the same either.
>>
>> Am I doing something incorrectly (and just got away with it for years)?
>>
>> n.b. If you're interested, I can share the .bb/.bbappend files
>> with you privately (I can't put them on the public list)
>
> Ah, one thing... do your machine names have uppercase characters in them (as
> your pseudo-examples above do)? If so that was a 2.1 change; anything that is
> to be used as an override must now be lower case (so DISTRO, MACHINE, etc.)

Indeed, my machine names do have an upper case letter in them.
Any suggestions for how to handle this (short of renaming everything which
would be a HUGE chore)?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Change in override behavior?

[Gary Thomas]

On 2016-04-29 09:24, Gary Thomas wrote:
> On 2016-04-29 09:16, Paul Eggleton wrote:
>> On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
>>> On 2016-04-29 07:08, Paul Eggleton wrote:
>>>> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
>>>>> I used to be able to use something like this:
>>>>>      VARIABLE = "xyz"
>>>>>
>>>>> and maybe in a .bbappend use
>>>>>
>>>>>      VARIABLE_mymachine = "xyzzy"
>>>>>
>>>>> where 'mymachine' is in OVERRIDES, which resulted in
>>>>>
>>>>>      VARIABLE = "xyzzy"
>>>>>
>>>>> This doesn't seem to work the same today, what I get is
>>>>>
>>>>>      VARIABLE = "xyz"
>>>>>      VARIABLE_mymachine = "xyzzy"
>>>>>
>>>>> Has this changed (recently)?  Why?  How to I make use of overrides?
>>>>>
>>>>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
>>>>
>>>> When you say this is what you're seeing, how are you checking this?
>>>> bitbake -e ? Which part of that output are you looking at?
>>>>
>>>> You *will* now see VARIABLE_mymachine in bitbake -e output even if
>>>> "mymachine" is in OVERRIDES - that's because the overridden values stay
>>>> in the datastore instead of a "finalisation" step eliminating them, in
>>>> order to allow OVERRIDES to be changed dynamically (possibly multiple
>>>> times) and still get the datastore contents you expect. That change was
>>>> made in the 2.0 release (jethro). However That should not affect the
>>>> final value of VARIABLE if "mymachine" is in OVERRIDES however. Are you
>>>> sure you are seeing what you think you're seeing? If so, is the bbappend
>>>> in fact being applied? Because if what you describe was really happening
>>>> then pretty much the entire system would be broken. I just checked a
>>>> similar case here and I see the expected results.
>>>
>>> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
>>>
>>> In my main recipe, I have this:
>>>     SOME_OTHER_VARIABLE ?= "${MACHINE}"
>>> and my .bbappend for that recipe has these lines:
>>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>> and local.conf has
>>>     MACHINE="my-target-B"
>>>
>>> With Yocto 2.0 (Poky ceeb52a2544) I get this:
>>>     SOME_OTHER_VARIABLE = "something-else2"
>>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>>
>>> With today's (Poky a9b503b26) I get this:
>>>     SOME_OTHER_VARIABLE = "my-target-B"
>>>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>>
>>> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
>>> that doesn't seem to be working the same either.
>>>
>>> Am I doing something incorrectly (and just got away with it for years)?
>>>
>>> n.b. If you're interested, I can share the .bb/.bbappend files
>>> with you privately (I can't put them on the public list)
>>
>> Ah, one thing... do your machine names have uppercase characters in them (as
>> your pseudo-examples above do)? If so that was a 2.1 change; anything that is
>> to be used as an override must now be lower case (so DISTRO, MACHINE, etc.)
>
> Indeed, my machine names do have an upper case letter in them.
> Any suggestions for how to handle this (short of renaming everything which
> would be a HUGE chore)?
>

Query: I also use this mechanism for additional patches, etc, specified as
SRC_URI_append_my-target-A and these are being handled correctly.  Does the
"append" mode behave differently?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Change in override behavior?

[Paul Eggleton]

On Fri, 29 Apr 2016 09:24:23 Gary Thomas wrote:
> On 2016-04-29 09:16, Paul Eggleton wrote:
> > On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
> >> On 2016-04-29 07:08, Paul Eggleton wrote:
> >>> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
> >>>> I used to be able to use something like this:
> >>>>      VARIABLE = "xyz"
> >>>> 
> >>>> and maybe in a .bbappend use
> >>>> 
> >>>>      VARIABLE_mymachine = "xyzzy"
> >>>> 
> >>>> where 'mymachine' is in OVERRIDES, which resulted in
> >>>> 
> >>>>      VARIABLE = "xyzzy"
> >>>> 
> >>>> This doesn't seem to work the same today, what I get is
> >>>> 
> >>>>      VARIABLE = "xyz"
> >>>>      VARIABLE_mymachine = "xyzzy"
> >>>> 
> >>>> Has this changed (recently)?  Why?  How to I make use of overrides?
> >>>> 
> >>>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
> >>> 
> >>> When you say this is what you're seeing, how are you checking this?
> >>> bitbake -e ? Which part of that output are you looking at?
> >>> 
> >>> You *will* now see VARIABLE_mymachine in bitbake -e output even if
> >>> "mymachine" is in OVERRIDES - that's because the overridden values stay
> >>> in the datastore instead of a "finalisation" step eliminating them, in
> >>> order to allow OVERRIDES to be changed dynamically (possibly multiple
> >>> times) and still get the datastore contents you expect. That change was
> >>> made in the 2.0 release (jethro). However That should not affect the
> >>> final value of VARIABLE if "mymachine" is in OVERRIDES however. Are you
> >>> sure you are seeing what you think you're seeing? If so, is the bbappend
> >>> in fact being applied? Because if what you describe was really happening
> >>> then pretty much the entire system would be broken. I just checked a
> >>> similar case here and I see the expected results.
> >> 
> >> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
> >> 
> >> In my main recipe, I have this:
> >>     SOME_OTHER_VARIABLE ?= "${MACHINE}"
> >> 
> >> and my .bbappend for that recipe has these lines:
> >>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >> 
> >> and local.conf has
> >> 
> >>     MACHINE="my-target-B"
> >> 
> >> With Yocto 2.0 (Poky ceeb52a2544) I get this:
> >>     SOME_OTHER_VARIABLE = "something-else2"
> >>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >> 
> >> With today's (Poky a9b503b26) I get this:
> >>     SOME_OTHER_VARIABLE = "my-target-B"
> >>     SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>     SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >> 
> >> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
> >> that doesn't seem to be working the same either.
> >> 
> >> Am I doing something incorrectly (and just got away with it for years)?
> >> 
> >> n.b. If you're interested, I can share the .bb/.bbappend files
> >> with you privately (I can't put them on the public list)
> > 
> > Ah, one thing... do your machine names have uppercase characters in them
> > (as your pseudo-examples above do)? If so that was a 2.1 change; anything
> > that is to be used as an override must now be lower case (so DISTRO,
> > MACHINE, etc.)
>
> Indeed, my machine names do have an upper case letter in them.
> Any suggestions for how to handle this (short of renaming everything which
> would be a HUGE chore)?

It's no longer supported I'm afraid, BitBake's datastore is now coded to 
expect lowercase for all override values. You say it's a huge chore but surely 
a grep -l | xargs -n1 sed -i would do it?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Change in override behavior?

[Gary Thomas]

On 2016-04-29 11:51, Paul Eggleton wrote:
> On Fri, 29 Apr 2016 09:24:23 Gary Thomas wrote:
>> On 2016-04-29 09:16, Paul Eggleton wrote:
>>> On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
>>>> On 2016-04-29 07:08, Paul Eggleton wrote:
>>>>> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
>>>>>> I used to be able to use something like this:
>>>>>>       VARIABLE = "xyz"
>>>>>>
>>>>>> and maybe in a .bbappend use
>>>>>>
>>>>>>       VARIABLE_mymachine = "xyzzy"
>>>>>>
>>>>>> where 'mymachine' is in OVERRIDES, which resulted in
>>>>>>
>>>>>>       VARIABLE = "xyzzy"
>>>>>>
>>>>>> This doesn't seem to work the same today, what I get is
>>>>>>
>>>>>>       VARIABLE = "xyz"
>>>>>>       VARIABLE_mymachine = "xyzzy"
>>>>>>
>>>>>> Has this changed (recently)?  Why?  How to I make use of overrides?
>>>>>>
>>>>>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
>>>>>
>>>>> When you say this is what you're seeing, how are you checking this?
>>>>> bitbake -e ? Which part of that output are you looking at?
>>>>>
>>>>> You *will* now see VARIABLE_mymachine in bitbake -e output even if
>>>>> "mymachine" is in OVERRIDES - that's because the overridden values stay
>>>>> in the datastore instead of a "finalisation" step eliminating them, in
>>>>> order to allow OVERRIDES to be changed dynamically (possibly multiple
>>>>> times) and still get the datastore contents you expect. That change was
>>>>> made in the 2.0 release (jethro). However That should not affect the
>>>>> final value of VARIABLE if "mymachine" is in OVERRIDES however. Are you
>>>>> sure you are seeing what you think you're seeing? If so, is the bbappend
>>>>> in fact being applied? Because if what you describe was really happening
>>>>> then pretty much the entire system would be broken. I just checked a
>>>>> similar case here and I see the expected results.
>>>>
>>>> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
>>>>
>>>> In my main recipe, I have this:
>>>>      SOME_OTHER_VARIABLE ?= "${MACHINE}"
>>>>
>>>> and my .bbappend for that recipe has these lines:
>>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>>>
>>>> and local.conf has
>>>>
>>>>      MACHINE="my-target-B"
>>>>
>>>> With Yocto 2.0 (Poky ceeb52a2544) I get this:
>>>>      SOME_OTHER_VARIABLE = "something-else2"
>>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>>>
>>>> With today's (Poky a9b503b26) I get this:
>>>>      SOME_OTHER_VARIABLE = "my-target-B"
>>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
>>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
>>>>
>>>> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
>>>> that doesn't seem to be working the same either.
>>>>
>>>> Am I doing something incorrectly (and just got away with it for years)?
>>>>
>>>> n.b. If you're interested, I can share the .bb/.bbappend files
>>>> with you privately (I can't put them on the public list)
>>>
>>> Ah, one thing... do your machine names have uppercase characters in them
>>> (as your pseudo-examples above do)? If so that was a 2.1 change; anything
>>> that is to be used as an override must now be lower case (so DISTRO,
>>> MACHINE, etc.)
>>
>> Indeed, my machine names do have an upper case letter in them.
>> Any suggestions for how to handle this (short of renaming everything which
>> would be a HUGE chore)?
>
> It's no longer supported I'm afraid, BitBake's datastore is now coded to
> expect lowercase for all override values. You say it's a huge chore but surely
> a grep -l | xargs -n1 sed -i would do it?

If I were to rename my target (MACHINE), I'd then have to go through
and rename all of the directories in the recipes that hold machine
customizations and there are quite a few of those.  Similarly I'd
have to rework all the recipes that use the old names (with upper case
letters) to the new ones.  Not impossible, but quite a lot of work.

Short of actually renaming the target machines, I could add an additional
override (in <MACHINE>.conf) that is lowercase only and update just the
recipes to use that instead.

By experimentation, I have found that XYZ_append_<MACHINE> seems to do
the correct thing, even when <MACHINE> has upper case letters.  Maybe this
is an oversight that will be fixed?

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Change in override behavior?

[Paul Eggleton]

On Fri, 29 Apr 2016 12:00:58 Gary Thomas wrote:
> On 2016-04-29 11:51, Paul Eggleton wrote:
> > On Fri, 29 Apr 2016 09:24:23 Gary Thomas wrote:
> >> On 2016-04-29 09:16, Paul Eggleton wrote:
> >>> On Fri, 29 Apr 2016 08:06:16 Gary Thomas wrote:
> >>>> On 2016-04-29 07:08, Paul Eggleton wrote:
> >>>>> On Fri, 29 Apr 2016 06:24:55 Gary Thomas wrote:
> >>>>>> I used to be able to use something like this:
> >>>>>>       VARIABLE = "xyz"
> >>>>>> 
> >>>>>> and maybe in a .bbappend use
> >>>>>> 
> >>>>>>       VARIABLE_mymachine = "xyzzy"
> >>>>>> 
> >>>>>> where 'mymachine' is in OVERRIDES, which resulted in
> >>>>>> 
> >>>>>>       VARIABLE = "xyzzy"
> >>>>>> 
> >>>>>> This doesn't seem to work the same today, what I get is
> >>>>>> 
> >>>>>>       VARIABLE = "xyz"
> >>>>>>       VARIABLE_mymachine = "xyzzy"
> >>>>>> 
> >>>>>> Has this changed (recently)?  Why?  How to I make use of overrides?
> >>>>>> 
> >>>>>> Note: I'm using the latest Poky master (a9b503b26) updated 2016-04-28
> >>>>> 
> >>>>> When you say this is what you're seeing, how are you checking this?
> >>>>> bitbake -e ? Which part of that output are you looking at?
> >>>>> 
> >>>>> You *will* now see VARIABLE_mymachine in bitbake -e output even if
> >>>>> "mymachine" is in OVERRIDES - that's because the overridden values
> >>>>> stay
> >>>>> in the datastore instead of a "finalisation" step eliminating them, in
> >>>>> order to allow OVERRIDES to be changed dynamically (possibly multiple
> >>>>> times) and still get the datastore contents you expect. That change
> >>>>> was
> >>>>> made in the 2.0 release (jethro). However That should not affect the
> >>>>> final value of VARIABLE if "mymachine" is in OVERRIDES however. Are
> >>>>> you
> >>>>> sure you are seeing what you think you're seeing? If so, is the
> >>>>> bbappend
> >>>>> in fact being applied? Because if what you describe was really
> >>>>> happening
> >>>>> then pretty much the entire system would be broken. I just checked a
> >>>>> similar case here and I see the expected results.
> >>>> 
> >>>> Yes, I am using 'bitbake -e <recipe> | grep ^VARIABLE' to look at this.
> >>>> 
> >>>> In my main recipe, I have this:
> >>>>      SOME_OTHER_VARIABLE ?= "${MACHINE}"
> >>>> 
> >>>> and my .bbappend for that recipe has these lines:
> >>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >>>> 
> >>>> and local.conf has
> >>>> 
> >>>>      MACHINE="my-target-B"
> >>>> 
> >>>> With Yocto 2.0 (Poky ceeb52a2544) I get this:
> >>>>      SOME_OTHER_VARIABLE = "something-else2"
> >>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >>>> 
> >>>> With today's (Poky a9b503b26) I get this:
> >>>>      SOME_OTHER_VARIABLE = "my-target-B"
> >>>>      SOME_OTHER_VARIABLE_my-target-A = "something-else1"
> >>>>      SOME_OTHER_VARIABLE_my-target-B = "something-else2"
> >>>> 
> >>>> I also use this mechanism to set COMPATIBLE_MACHINE a lot and
> >>>> that doesn't seem to be working the same either.
> >>>> 
> >>>> Am I doing something incorrectly (and just got away with it for years)?
> >>>> 
> >>>> n.b. If you're interested, I can share the .bb/.bbappend files
> >>>> with you privately (I can't put them on the public list)
> >>> 
> >>> Ah, one thing... do your machine names have uppercase characters in them
> >>> (as your pseudo-examples above do)? If so that was a 2.1 change;
> >>> anything
> >>> that is to be used as an override must now be lower case (so DISTRO,
> >>> MACHINE, etc.)
> >> 
> >> Indeed, my machine names do have an upper case letter in them.
> >> Any suggestions for how to handle this (short of renaming everything
> >> which
> >> would be a HUGE chore)?
> > 
> > It's no longer supported I'm afraid, BitBake's datastore is now coded to
> > expect lowercase for all override values. You say it's a huge chore but
> > surely a grep -l | xargs -n1 sed -i would do it?
> 
> If I were to rename my target (MACHINE), I'd then have to go through
> and rename all of the directories in the recipes that hold machine
> customizations and there are quite a few of those.  Similarly I'd
> have to rework all the recipes that use the old names (with upper case
> letters) to the new ones.  Not impossible, but quite a lot of work.

That is unfortunate, but again should be pretty easy to automate (assuming 
they are local of course and not proliferated across many users' build 
machines).

> Short of actually renaming the target machines, I could add an additional
> override (in <MACHINE>.conf) that is lowercase only and update just the
> recipes to use that instead.
> 
> By experimentation, I have found that XYZ_append_<MACHINE> seems to do
> the correct thing, even when <MACHINE> has upper case letters.  Maybe this
> is an oversight that will be fixed?

I'm not sure - Richard?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Change in override behavior?

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 465 bytes --]


> On Apr 29, 2016, at 3:09 AM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote:
> 
>> 
>> By experimentation, I have found that XYZ_append_<MACHINE> seems to do
>> the correct thing, even when <MACHINE> has upper case letters.  Maybe this
>> is an oversight that will be fixed?
> 

This seems to be an oversight and should be fixed for consistency. packagers may have
gripes about using uppercase too, not sure how thats working out in your case.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: Change in override behavior?

[Richard Purdie]

On Fri, 2016-04-29 at 22:09 +1200, Paul Eggleton wrote:
> On Fri, 29 Apr 2016 12:00:58 Gary Thomas wrote:
> > On 2016-04-29 11:51, Paul Eggleton wrote:
> > > On Fri, 29 Apr 2016 09:24:23 Gary Thomas wrote:
> > > > On 2016-04-29 09:16, Paul Eggleton wrote:
> > > > Indeed, my machine names do have an upper case letter in them.
> > > > Any suggestions for how to handle this (short of renaming
> > > > everything
> > > > which
> > > > would be a HUGE chore)?
> > > 
> > > It's no longer supported I'm afraid, BitBake's datastore is now
> > > coded to
> > > expect lowercase for all override values. You say it's a huge
> > > chore but
> > > surely a grep -l | xargs -n1 sed -i would do it?
> > 
> > If I were to rename my target (MACHINE), I'd then have to go
> > through
> > and rename all of the directories in the recipes that hold machine
> > customizations and there are quite a few of those.  Similarly I'd
> > have to rework all the recipes that use the old names (with upper
> > case
> > letters) to the new ones.  Not impossible, but quite a lot of work.
> 
> That is unfortunate, but again should be pretty easy to automate
> (assuming 
> they are local of course and not proliferated across many users'
> build 
> machines).
> 
> > Short of actually renaming the target machines, I could add an
> > additional
> > override (in <MACHINE>.conf) that is lowercase only and update just
> > the
> > recipes to use that instead.
> > 
> > By experimentation, I have found that XYZ_append_<MACHINE> seems to
> > do
> > the correct thing, even when <MACHINE> has upper case letters. 
> >  Maybe this
> > is an oversight that will be fixed?
> 
> I'm not sure - Richard?

Sorry for the delay in getting to this. It does sound like there is a
codepath in bitbake which is still recognising variables with mixed
case. I took a quick look and think this change appears to address it:

diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
index 25c412c..ec6709f 100644
--- a/bitbake/lib/bb/data_smart.py
+++ b/bitbake/lib/bb/data_smart.py
@@ -39,7 +39,7 @@ from bb.COW  import COWDictBase
 logger = logging.getLogger("BitBake.Data")
 
 __setvar_keyword__ = ["_append", "_prepend", "_remove"]
-__setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>.*))?$')
+__setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>[^A-Z]*))?$')
 __expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
 __expand_python_regexp__ = re.compile(r"\${@.+?}")


Cheers,

Richard

Re: Change in override behavior?

[Andre McCurdy]

On Tue, May 24, 2016 at 5:31 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2016-04-29 at 22:09 +1200, Paul Eggleton wrote:
>> On Fri, 29 Apr 2016 12:00:58 Gary Thomas wrote:
>> > On 2016-04-29 11:51, Paul Eggleton wrote:
>> > > On Fri, 29 Apr 2016 09:24:23 Gary Thomas wrote:
>> > > > On 2016-04-29 09:16, Paul Eggleton wrote:
>> > > > Indeed, my machine names do have an upper case letter in them.
>> > > > Any suggestions for how to handle this (short of renaming
>> > > > everything
>> > > > which
>> > > > would be a HUGE chore)?
>> > >
>> > > It's no longer supported I'm afraid, BitBake's datastore is now
>> > > coded to
>> > > expect lowercase for all override values. You say it's a huge
>> > > chore but
>> > > surely a grep -l | xargs -n1 sed -i would do it?
>> >
>> > If I were to rename my target (MACHINE), I'd then have to go
>> > through
>> > and rename all of the directories in the recipes that hold machine
>> > customizations and there are quite a few of those.  Similarly I'd
>> > have to rework all the recipes that use the old names (with upper
>> > case
>> > letters) to the new ones.  Not impossible, but quite a lot of work.
>>
>> That is unfortunate, but again should be pretty easy to automate
>> (assuming
>> they are local of course and not proliferated across many users'
>> build
>> machines).
>>
>> > Short of actually renaming the target machines, I could add an
>> > additional
>> > override (in <MACHINE>.conf) that is lowercase only and update just
>> > the
>> > recipes to use that instead.
>> >
>> > By experimentation, I have found that XYZ_append_<MACHINE> seems to
>> > do
>> > the correct thing, even when <MACHINE> has upper case letters.
>> >  Maybe this
>> > is an oversight that will be fixed?
>>
>> I'm not sure - Richard?
>
> Sorry for the delay in getting to this. It does sound like there is a
> codepath in bitbake which is still recognising variables with mixed
> case. I took a quick look and think this change appears to address it:
>
> diff --git a/bitbake/lib/bb/data_smart.py b/bitbake/lib/bb/data_smart.py
> index 25c412c..ec6709f 100644
> --- a/bitbake/lib/bb/data_smart.py
> +++ b/bitbake/lib/bb/data_smart.py
> @@ -39,7 +39,7 @@ from bb.COW  import COWDictBase
>  logger = logging.getLogger("BitBake.Data")
>
>  __setvar_keyword__ = ["_append", "_prepend", "_remove"]
> -__setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>.*))?$')
> +__setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>[^A-Z]*))?$')
>  __expand_var_regexp__ = re.compile(r"\${[^{}@\n\t :]+}")
>  __expand_python_regexp__ = re.compile(r"\${@.+?}")

What about over-rides which contain only numbers?

It doesn't look like this change is going to filter them out, but they
haven't worked as regular over-rides since:

  http://git.openembedded.org/bitbake/commit/?id=c9b9443faa76ee7366b1400a56f826f3f9dec1be

(My personal preference for resolving that inconsistency would be to
re-enable and officially support over-rides containing only numbers).

>
> Cheers,
>
> Richard
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto