Override syntax change proposal

Override syntax change proposal

[Richard Purdie]

Hi All,

I now have an actual proposed plan for changing override syntax which I hope
will mitigate the issues raised in the previous discussion as far a is possible
whilst still letting the project move forward.

Firstly, I'd like to merge a patch I sent to the bitbake list:

https://lists.openembedded.org/g/bitbake-devel/topic/patch_data_smart_parse/84285770

This patch allows bitbake to accept ":" in variable/function names. It is designed
to be simple and could be easily backported to previous bitbake versions. At the
very least I'd like to do that for master, allowing at least OE-Core master branch 
to start using an explicit override syntax.

I'd like to go further and suggest we backport to older bitbake versions (back to 
dunfell?) so that anyone using metadata over multiple versions could continue to
do so.

I'm hoping the above isn't too controversial and would at least let us move forward
to trying to improve.

The next steps beyond that are a little trickier as they start to require the change.
The next logical move is to require ":" as the separator for current master branches.
I believe it is reasonable to do this if we can show how layers can be converted and 
have a way for older layers to continue to work with master.

There is a script I wrote and linked to (~100 lines of python) which with some
priming about the overrides in use in a layer (which a layer maintainer should know), 
is able to automatically convert a layer. For oe-core+meta-yocto+docs+bitbake, that
is 9000+ changes but the script seems reliable if not very nice code. I did end up
adding a handful of manual fixes but that was for python code manipulating variables
which most layers wouldn't have, in general the core is way more complex than most
layers.

This means that people with older layers trying to work with master now have two
options:

a) they convert to use ":" and update to a version of bitbake with the above tweak
   to allow it to accept that (no behaviour change)
b) they dynamically translate their layer with a script when testing with master

Given those options two options both being shown to be viable, I think this
is enough to allow us to proceed and update the syntax on shorter timescales such
as during this release, ready for the next LTS.

In working out compatibility pieces, I don't have any optimisations using the 
new syntax yet, or a plan worked out for what we can do beyond this other that ideas
and my previous experiments. Some changes "feel" right and this one definitely does 
as a good move in it's own right regardless of whether we ultimately can improve 
syntax or not. I'm therefore keen to proceed.

Cheers.

Richard