Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Peter Kjellerstedt]

[ Cross-posting since these changes affect all of bitbake, openembedded-core 
  and meta-openembedded and need to be made (at least somewhat) in sync. ]

Can the solution for _PYTHON_SYSCONFIGDATA_NAME please be backported to 
Gatesgarth and Dunfell? We are seeing problems for developers who have 
updated the python3 version on their hosts when they try to do devtool 
modify on recipes that inherit python3native. The following relevant 
changes should cherry-pick cleanly to the respective repositories for 
both gatesgarth and dunfell:

openembedded-core:
5a118d4e python3: split python target configuration into own class
dadf001c python3-pycairo: use python3targetconfig
9c8f6660 distutils3-base.bbclass: use python3targetconfig
d3a81dd0 meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
38ecb83c gpgme: use python3targetconfig
c99bb790 python3targetconfig.bbclass: Make py3 dep and tasks only for target recipes

bitbake:
47b64cfa lib/bb/fetch2/__init__.py: drop _PYTHON_SYSCONFIGDATA_NAME unsetting

meta-openembedded:
a06cdf5a gedit: Inherit python3targetconfig
59f817bb openipmi: Inherit python3targetconfig
be7d2286 libplist: Inherit python3targetconfig
c499aaee postgresql: Inherit python3targetconfig

I guess there may be some controversy regarding whether these changes should 
be backported to the stable branches since they modify what python3native 
does and may require existing recipes to be modified to inherit the new 
python3targetconfig class. However, I believe that having devtool modify 
fail on random recipes without an easy solution for the user is much worse 
than having to update a recipe or two once due to a build failure after 
upgrading to a new version of OE-Core (in my book that is pretty normal and 
something I always have to do anyway when we update OE-Core). And based on 
the number of recipes in openembedded-core and meta-openembedded that needed 
to be updated (2+4), I do not expect there to be many other recipes in the 
first place that actually need to be updated.

//Peter

Re: [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 2494 bytes --]

Aren't the missing spaces in appends fixes also needed for meta-oe recipes?

I think at least top 5 commits from:
https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
were also follow-up from these changes in oe-core.

On Thu, Jan 14, 2021 at 6:50 PM Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> [ Cross-posting since these changes affect all of bitbake,
> openembedded-core
>   and meta-openembedded and need to be made (at least somewhat) in sync. ]
>
> Can the solution for _PYTHON_SYSCONFIGDATA_NAME please be backported to
> Gatesgarth and Dunfell? We are seeing problems for developers who have
> updated the python3 version on their hosts when they try to do devtool
> modify on recipes that inherit python3native. The following relevant
> changes should cherry-pick cleanly to the respective repositories for
> both gatesgarth and dunfell:
>
> openembedded-core:
> 5a118d4e python3: split python target configuration into own class
> dadf001c python3-pycairo: use python3targetconfig
> 9c8f6660 distutils3-base.bbclass: use python3targetconfig
> d3a81dd0 meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
> 38ecb83c gpgme: use python3targetconfig
> c99bb790 python3targetconfig.bbclass: Make py3 dep and tasks only for
> target recipes
>
> bitbake:
> 47b64cfa lib/bb/fetch2/__init__.py: drop _PYTHON_SYSCONFIGDATA_NAME
> unsetting
>
> meta-openembedded:
> a06cdf5a gedit: Inherit python3targetconfig
> 59f817bb openipmi: Inherit python3targetconfig
> be7d2286 libplist: Inherit python3targetconfig
> c499aaee postgresql: Inherit python3targetconfig
>
> I guess there may be some controversy regarding whether these changes
> should
> be backported to the stable branches since they modify what python3native
> does and may require existing recipes to be modified to inherit the new
> python3targetconfig class. However, I believe that having devtool modify
> fail on random recipes without an easy solution for the user is much worse
> than having to update a recipe or two once due to a build failure after
> upgrading to a new version of OE-Core (in my book that is pretty normal
> and
> something I always have to do anyway when we update OE-Core). And based on
> the number of recipes in openembedded-core and meta-openembedded that
> needed
> to be updated (2+4), I do not expect there to be many other recipes in the
> first place that actually need to be updated.
>
> //Peter
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3014 bytes --]

Re: [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[akuster]

On 1/16/21 9:44 AM, Martin Jansa wrote:
> Aren't the missing spaces in appends fixes also needed for meta-oe
> recipes?


>
> I think at least top 5 commits from:
> https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> were also follow-up from these changes in oe-core.
>
Do you mean something beyond the meta-openembedded commits mentioned
near the bottom of the email?

I need clarity so I can open an issue in gitlab for tracking purposes as
I am sure I will forget this

-armin

> On Thu, Jan 14, 2021 at 6:50 PM Peter Kjellerstedt
> <peter.kjellerstedt@axis.com <mailto:peter.kjellerstedt@axis.com>> wrote:
>
>     [ Cross-posting since these changes affect all of bitbake,
>     openembedded-core
>       and meta-openembedded and need to be made (at least somewhat) in
>     sync. ]
>
>     Can the solution for _PYTHON_SYSCONFIGDATA_NAME please be
>     backported to
>     Gatesgarth and Dunfell? We are seeing problems for developers who
>     have
>     updated the python3 version on their hosts when they try to do
>     devtool
>     modify on recipes that inherit python3native. The following relevant
>     changes should cherry-pick cleanly to the respective repositories for
>     both gatesgarth and dunfell:
>
>     openembedded-core:
>     5a118d4e python3: split python target configuration into own class
>     dadf001c python3-pycairo: use python3targetconfig
>     9c8f6660 distutils3-base.bbclass: use python3targetconfig
>     d3a81dd0 meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
>     38ecb83c gpgme: use python3targetconfig
>     c99bb790 python3targetconfig.bbclass: Make py3 dep and tasks only
>     for target recipes
>
>     bitbake:
>     47b64cfa lib/bb/fetch2/__init__.py: drop
>     _PYTHON_SYSCONFIGDATA_NAME unsetting
>
>     meta-openembedded:
>     a06cdf5a gedit: Inherit python3targetconfig
>     59f817bb openipmi: Inherit python3targetconfig
>     be7d2286 libplist: Inherit python3targetconfig
>     c499aaee postgresql: Inherit python3targetconfig
>
>     I guess there may be some controversy regarding whether these
>     changes should
>     be backported to the stable branches since they modify what
>     python3native
>     does and may require existing recipes to be modified to inherit
>     the new
>     python3targetconfig class. However, I believe that having devtool
>     modify
>     fail on random recipes without an easy solution for the user is
>     much worse
>     than having to update a recipe or two once due to a build failure
>     after
>     upgrading to a new version of OE-Core (in my book that is pretty
>     normal and
>     something I always have to do anyway when we update OE-Core). And
>     based on
>     the number of recipes in openembedded-core and meta-openembedded
>     that needed
>     to be updated (2+4), I do not expect there to be many other
>     recipes in the
>     first place that actually need to be updated.
>
>     //Peter
>
>
>     
>

Re: [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 2277 bytes --]

I fully support this.

Alex

On Thu, 14 Jan 2021 at 18:50, Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> [ Cross-posting since these changes affect all of bitbake,
> openembedded-core
>   and meta-openembedded and need to be made (at least somewhat) in sync. ]
>
> Can the solution for _PYTHON_SYSCONFIGDATA_NAME please be backported to
> Gatesgarth and Dunfell? We are seeing problems for developers who have
> updated the python3 version on their hosts when they try to do devtool
> modify on recipes that inherit python3native. The following relevant
> changes should cherry-pick cleanly to the respective repositories for
> both gatesgarth and dunfell:
>
> openembedded-core:
> 5a118d4e python3: split python target configuration into own class
> dadf001c python3-pycairo: use python3targetconfig
> 9c8f6660 distutils3-base.bbclass: use python3targetconfig
> d3a81dd0 meta: drop _PYTHON_SYSCONFIGDATA_NAME hacks
> 38ecb83c gpgme: use python3targetconfig
> c99bb790 python3targetconfig.bbclass: Make py3 dep and tasks only for
> target recipes
>
> bitbake:
> 47b64cfa lib/bb/fetch2/__init__.py: drop _PYTHON_SYSCONFIGDATA_NAME
> unsetting
>
> meta-openembedded:
> a06cdf5a gedit: Inherit python3targetconfig
> 59f817bb openipmi: Inherit python3targetconfig
> be7d2286 libplist: Inherit python3targetconfig
> c499aaee postgresql: Inherit python3targetconfig
>
> I guess there may be some controversy regarding whether these changes
> should
> be backported to the stable branches since they modify what python3native
> does and may require existing recipes to be modified to inherit the new
> python3targetconfig class. However, I believe that having devtool modify
> fail on random recipes without an easy solution for the user is much worse
> than having to update a recipe or two once due to a build failure after
> upgrading to a new version of OE-Core (in my book that is pretty normal
> and
> something I always have to do anyway when we update OE-Core). And based on
> the number of recipes in openembedded-core and meta-openembedded that
> needed
> to be updated (2+4), I do not expect there to be many other recipes in the
> first place that actually need to be updated.
>
> //Peter
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 2688 bytes --]

Re: [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Anuj Mittal]

On Thu, 2021-01-14 at 17:50 +0000, Peter Kjellerstedt wrote:
> I guess there may be some controversy regarding whether these changes
> should 
> be backported to the stable branches since they modify what
> python3native 
> does and may require existing recipes to be modified to inherit the
> new 
> python3targetconfig class. However, I believe that having devtool
> modify 
> fail on random recipes without an easy solution for the user is much
> worse 
> than having to update a recipe or two once due to a build failure
> after 
> upgrading to a new version of OE-Core (in my book that is pretty
> normal and 
> something I always have to do anyway when we update OE-Core).

I agree that devtool failing is not good especially when we recommend
that people run it in some help messages.

Is there a way we can show a helpful message for stable branches when
such a change might be needed?

Thanks,

Anuj

Re: [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1008 bytes --]

On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com> wrote:

>
>
> On 1/16/21 9:44 AM, Martin Jansa wrote:
> > Aren't the missing spaces in appends fixes also needed for meta-oe
> > recipes?
>
>
> >
> > I think at least top 5 commits from:
> >
> https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> > were also follow-up from these changes in oe-core.
> >
> Do you mean something beyond the meta-openembedded commits mentioned
> near the bottom of the email?
>

Yes I mean these 5 commits at least.

with c99bb790 DEPENDS variable doesn't end with a space, so the missing
leading space in these appends (which was fine until now because of
trailing space from DEPENDS set in bbclass) is now causing wrong dependency
(should be easily reproducible with the patches backported, just by parsing
the recipes).


> I need clarity so I can open an issue in gitlab for tracking purposes as
> I am sure I will forget this
>

we're using gitlab?

[-- Attachment #2: Type: text/html, Size: 1661 bytes --]

Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1732 bytes --]

On Sun, Feb 7, 2021 at 12:15 AM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

> On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com> wrote:
>
>>
>>
>> On 1/16/21 9:44 AM, Martin Jansa wrote:
>> > Aren't the missing spaces in appends fixes also needed for meta-oe
>> > recipes?
>>
>>
>> >
>> > I think at least top 5 commits from:
>> >
>> https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
>> > were also follow-up from these changes in oe-core.
>> >
>> Do you mean something beyond the meta-openembedded commits mentioned
>> near the bottom of the email?
>>
>
> Yes I mean these 5 commits at least.
>

50bbf80abf python3-pykwalify: Do not unset _PYTHON_SYSCONFIGDATA_NAME
6b3e3bdaf8 python-grpcio-tools: Add missing space for append

The first one is just an additional cleanup, but without the 2nd one and
with the cherry-picks from the first e-mail applied you would get:
ERROR: Nothing PROVIDES 'python3python3-grpcio' (but
meta-oe/meta-python/recipes-devtools/python/python3-grpcio-tools_1.14.1.bb
DEPENDS on or otherwise requires it). Close matches:
  python-grpcio
  python3-grpcio

my world build is still running..


> with c99bb790 DEPENDS variable doesn't end with a space, so the missing
> leading space in these appends (which was fine until now because of
> trailing space from DEPENDS set in bbclass) is now causing wrong dependency
> (should be easily reproducible with the patches backported, just by parsing
> the recipes).
>
>
>> I need clarity so I can open an issue in gitlab for tracking purposes as
>> I am sure I will forget this
>>
>
> we're using gitlab?
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3134 bytes --]

Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Peter Kjellerstedt]

[-- Attachment #1: Type: text/plain, Size: 3646 bytes --]

As agreed upon on the Yocto Project Technical Team Meeting a week ago, I have created branches for both Gatesgarth and Dunfell in openembedded-core-contrib and meta-openembedded-contrib with the required patches. The branches are called pkj/_PYTHON_SYSCONFIGDATA_NAME-gatesgarth and pkj/_PYTHON_SYSCONFIGDATA_NAME-dunfell (ok, horrible names, but they should be easy to spot). There is one additional patch for bitbake, but I forgot to ask for access to bitbake-contrib, so you will have to cherry-pick it from commit 47b64cfa (it is not strictly necessary as it is just clean up made possible after the changes in OE-Core).

@jansa, @akuster: I have included the two extra patches in meta-openembedded that Martin mentioned, which were lacking from my original mail. There were other patches mentioned, which fixed missing spaces related to the use of _append. However, as they were not necessary (they changed other variables than DEPENDS for target), I did not include them.

I have not done any excessive testing of these branches. I have verified that I can run `devtool modify libxml2`, which I could not do before, and I have run `bitbake core-image-minimal`.

//Peter

From: Martin Jansa <martin.jansa@gmail.com>
Sent: den 7 februari 2021 12:15
To: Martin Jansa <Martin.Jansa@gmail.com>
Cc: akuster808 <akuster808@gmail.com>; Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Steve Sakoman <steve@sakoman.com>; Mittal, Anuj <anuj.mittal@intel.com>; OE Core (openembedded-core@lists.openembedded.org) <openembedded-core@lists.openembedded.org>; OE Development (openembedded-devel@lists.openembedded.org) <openembedded-devel@lists.openembedded.org>; BitBake Development (bitbake-devel@lists.openembedded.org) <bitbake-devel@lists.openembedded.org>
Subject: Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

On Sun, Feb 7, 2021 at 12:15 AM Martin Jansa via lists.openembedded.org<http://lists.openembedded.org> <Martin.Jansa=gmail.com@lists.openembedded.org<mailto:gmail.com@lists.openembedded.org>> wrote:
On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com<mailto:akuster808@gmail.com>> wrote:


On 1/16/21 9:44 AM, Martin Jansa wrote:
> Aren't the missing spaces in appends fixes also needed for meta-oe
> recipes?


>
> I think at least top 5 commits from:
> https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> were also follow-up from these changes in oe-core.
>
Do you mean something beyond the meta-openembedded commits mentioned
near the bottom of the email?

Yes I mean these 5 commits at least.

50bbf80abf python3-pykwalify: Do not unset _PYTHON_SYSCONFIGDATA_NAME
6b3e3bdaf8 python-grpcio-tools: Add missing space for append

The first one is just an additional cleanup, but without the 2nd one and with the cherry-picks from the first e-mail applied you would get:
ERROR: Nothing PROVIDES 'python3python3-grpcio' (but meta-oe/meta-python/recipes-devtools/python/python3-grpcio-tools_1.14.1.bb<http://python3-grpcio-tools_1.14.1.bb> DEPENDS on or otherwise requires it). Close matches:
  python-grpcio
  python3-grpcio

my world build is still running..


with c99bb790 DEPENDS variable doesn't end with a space, so the missing leading space in these appends (which was fine until now because of trailing space from DEPENDS set in bbclass) is now causing wrong dependency (should be easily reproducible with the patches backported, just by parsing the recipes).

I need clarity so I can open an issue in gitlab for tracking purposes as
I am sure I will forget this

we're using gitlab?



[-- Attachment #2: Type: text/html, Size: 8622 bytes --]

Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4648 bytes --]

Hi,

I believe I have the same set of patches in jansa/dunfell and
jansa/gatesgarth branchesand bitbake world passed OK with them (with my set
of layers).

The only new issue I see in dunfell is libvirt from meta-virtualization
failing, because it uses hosts gcc to build python module, from quick look
I guess it shouldn't be using host's gcc and instead it should inherit
python3targetconfig or something to keep the old behavior. In the
meta-virtualization master branch there is this patch:
https://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization/commit/?id=08b5de42f9216a10bb2f83263b7e6789686b4c3d
which works for dunfell, but before backporting it I will double check if
this is really caused by these changes.

Cheers,

On Tue, Feb 9, 2021 at 12:45 AM Peter Kjellerstedt <
peter.kjellerstedt@axis.com> wrote:

> As agreed upon on the Yocto Project Technical Team Meeting a week ago, I
> have created branches for both Gatesgarth and Dunfell in
> openembedded-core-contrib and meta-openembedded-contrib with the required
> patches. The branches are called pkj/_PYTHON_SYSCONFIGDATA_NAME-gatesgarth
> and pkj/_PYTHON_SYSCONFIGDATA_NAME-dunfell (ok, horrible names, but they
> should be easy to spot). There is one additional patch for bitbake, but I
> forgot to ask for access to bitbake-contrib, so you will have to
> cherry-pick it from commit 47b64cfa (it is not strictly necessary as it is
> just clean up made possible after the changes in OE-Core).
>
>
>
> @jansa, @akuster: I have included the two extra patches in
> meta-openembedded that Martin mentioned, which were lacking from my
> original mail. There were other patches mentioned, which fixed missing
> spaces related to the use of _append. However, as they were not necessary
> (they changed other variables than DEPENDS for target), I did not include
> them.
>
>
>
> I have not done any excessive testing of these branches. I have verified
> that I can run `devtool modify libxml2`, which I could not do before, and I
> have run `bitbake core-image-minimal`.
>
>
>
> //Peter
>
>
>
> *From:* Martin Jansa <martin.jansa@gmail.com>
> *Sent:* den 7 februari 2021 12:15
> *To:* Martin Jansa <Martin.Jansa@gmail.com>
> *Cc:* akuster808 <akuster808@gmail.com>; Peter Kjellerstedt <
> peter.kjellerstedt@axis.com>; Steve Sakoman <steve@sakoman.com>; Mittal,
> Anuj <anuj.mittal@intel.com>; OE Core (
> openembedded-core@lists.openembedded.org) <
> openembedded-core@lists.openembedded.org>; OE Development (
> openembedded-devel@lists.openembedded.org) <
> openembedded-devel@lists.openembedded.org>; BitBake Development (
> bitbake-devel@lists.openembedded.org) <
> bitbake-devel@lists.openembedded.org>
> *Subject:* Re: [oe] [bitbake-devel] Backport changes for
> _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell
>
>
>
> On Sun, Feb 7, 2021 at 12:15 AM Martin Jansa via lists.openembedded.org
> <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
>
> On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com> wrote:
>
>
>
> On 1/16/21 9:44 AM, Martin Jansa wrote:
> > Aren't the missing spaces in appends fixes also needed for meta-oe
> > recipes?
>
>
> >
> > I think at least top 5 commits from:
> >
> https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> > were also follow-up from these changes in oe-core.
> >
> Do you mean something beyond the meta-openembedded commits mentioned
> near the bottom of the email?
>
>
>
> Yes I mean these 5 commits at least.
>
>
>
> 50bbf80abf python3-pykwalify: Do not unset _PYTHON_SYSCONFIGDATA_NAME
>
> 6b3e3bdaf8 python-grpcio-tools: Add missing space for append
>
>
>
> The first one is just an additional cleanup, but without the 2nd one and
> with the cherry-picks from the first e-mail applied you would get:
>
> ERROR: Nothing PROVIDES 'python3python3-grpcio' (but
> meta-oe/meta-python/recipes-devtools/python/python3-grpcio-tools_1.14.1.bb
> DEPENDS on or otherwise requires it). Close matches:
>   python-grpcio
>   python3-grpcio
>
>
>
> my world build is still running..
>
>
>
>
>
> with c99bb790 DEPENDS variable doesn't end with a space, so the missing
> leading space in these appends (which was fine until now because of
> trailing space from DEPENDS set in bbclass) is now causing wrong dependency
> (should be easily reproducible with the patches backported, just by parsing
> the recipes).
>
>
>
> I need clarity so I can open an issue in gitlab for tracking purposes as
> I am sure I will forget this
>
>
>
> we're using gitlab?
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 9517 bytes --]

Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Steve Sakoman]

On Mon, Feb 8, 2021 at 1:45 PM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> As agreed upon on the Yocto Project Technical Team Meeting a week ago, I have created branches for both Gatesgarth and Dunfell in openembedded-core-contrib and meta-openembedded-contrib with the required patches. The branches are called pkj/_PYTHON_SYSCONFIGDATA_NAME-gatesgarth and pkj/_PYTHON_SYSCONFIGDATA_NAME-dunfell (ok, horrible names, but they should be easy to spot). There is one additional patch for bitbake, but I forgot to ask for access to bitbake-contrib, so you will have to cherry-pick it from commit 47b64cfa (it is not strictly necessary as it is just clean up made possible after the changes in OE-Core).
>
>
>
> @jansa, @akuster: I have included the two extra patches in meta-openembedded that Martin mentioned, which were lacking from my original mail. There were other patches mentioned, which fixed missing spaces related to the use of _append. However, as they were not necessary (they changed other variables than DEPENDS for target), I did not include them.
>
>
>
> I have not done any excessive testing of these branches. I have verified that I can run `devtool modify libxml2`, which I could not do before, and I have run `bitbake core-image-minimal`.

Thanks Peter!  I'll start testing with dunfell this morning.

Steve

> From: Martin Jansa <martin.jansa@gmail.com>
> Sent: den 7 februari 2021 12:15
> To: Martin Jansa <Martin.Jansa@gmail.com>
> Cc: akuster808 <akuster808@gmail.com>; Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Steve Sakoman <steve@sakoman.com>; Mittal, Anuj <anuj.mittal@intel.com>; OE Core (openembedded-core@lists.openembedded.org) <openembedded-core@lists.openembedded.org>; OE Development (openembedded-devel@lists.openembedded.org) <openembedded-devel@lists.openembedded.org>; BitBake Development (bitbake-devel@lists.openembedded.org) <bitbake-devel@lists.openembedded.org>
> Subject: Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell
>
>
>
> On Sun, Feb 7, 2021 at 12:15 AM Martin Jansa via lists.openembedded.org <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
>
> On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com> wrote:
>
>
>
> On 1/16/21 9:44 AM, Martin Jansa wrote:
> > Aren't the missing spaces in appends fixes also needed for meta-oe
> > recipes?
>
>
> >
> > I think at least top 5 commits from:
> > https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> > were also follow-up from these changes in oe-core.
> >
> Do you mean something beyond the meta-openembedded commits mentioned
> near the bottom of the email?
>
>
>
> Yes I mean these 5 commits at least.
>
>
>
> 50bbf80abf python3-pykwalify: Do not unset _PYTHON_SYSCONFIGDATA_NAME
>
> 6b3e3bdaf8 python-grpcio-tools: Add missing space for append
>
>
>
> The first one is just an additional cleanup, but without the 2nd one and with the cherry-picks from the first e-mail applied you would get:
>
> ERROR: Nothing PROVIDES 'python3python3-grpcio' (but meta-oe/meta-python/recipes-devtools/python/python3-grpcio-tools_1.14.1.bb DEPENDS on or otherwise requires it). Close matches:
>   python-grpcio
>   python3-grpcio
>
>
>
> my world build is still running..
>
>
>
>
>
> with c99bb790 DEPENDS variable doesn't end with a space, so the missing leading space in these appends (which was fine until now because of trailing space from DEPENDS set in bbclass) is now causing wrong dependency (should be easily reproducible with the patches backported, just by parsing the recipes).
>
>
>
> I need clarity so I can open an issue in gitlab for tracking purposes as
> I am sure I will forget this
>
>
>
> we're using gitlab?
>
>
>
> 
>

Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell

[Steve Sakoman]

On Tue, Feb 9, 2021 at 4:11 AM Steve Sakoman <sakoman@gmail.com> wrote:
>
> On Mon, Feb 8, 2021 at 1:45 PM Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> >
> > As agreed upon on the Yocto Project Technical Team Meeting a week ago, I have created branches for both Gatesgarth and Dunfell in openembedded-core-contrib and meta-openembedded-contrib with the required patches. The branches are called pkj/_PYTHON_SYSCONFIGDATA_NAME-gatesgarth and pkj/_PYTHON_SYSCONFIGDATA_NAME-dunfell (ok, horrible names, but they should be easy to spot). There is one additional patch for bitbake, but I forgot to ask for access to bitbake-contrib, so you will have to cherry-pick it from commit 47b64cfa (it is not strictly necessary as it is just clean up made possible after the changes in OE-Core).
> >
> >
> >
> > @jansa, @akuster: I have included the two extra patches in meta-openembedded that Martin mentioned, which were lacking from my original mail. There were other patches mentioned, which fixed missing spaces related to the use of _append. However, as they were not necessary (they changed other variables than DEPENDS for target), I did not include them.
> >
> >
> >
> > I have not done any excessive testing of these branches. I have verified that I can run `devtool modify libxml2`, which I could not do before, and I have run `bitbake core-image-minimal`.
>
> Thanks Peter!  I'll start testing with dunfell this morning.

I did an autobuilder run with your meta-openembedded branch, my
stable/dunfell-nut branch with the 6 patches, and my bitbake
stable/1.46-nut branch with the single patch:

https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/921

All was fine except for an error in sysdig: do_compile, which doesn't
appear to be related:

https://errors.yoctoproject.org/Errors/Details/570303/

Unless someone has an objection I'll add the oe-core and bitbake
patches to my next set of review patches.

Steve


> > From: Martin Jansa <martin.jansa@gmail.com>
> > Sent: den 7 februari 2021 12:15
> > To: Martin Jansa <Martin.Jansa@gmail.com>
> > Cc: akuster808 <akuster808@gmail.com>; Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Steve Sakoman <steve@sakoman.com>; Mittal, Anuj <anuj.mittal@intel.com>; OE Core (openembedded-core@lists.openembedded.org) <openembedded-core@lists.openembedded.org>; OE Development (openembedded-devel@lists.openembedded.org) <openembedded-devel@lists.openembedded.org>; BitBake Development (bitbake-devel@lists.openembedded.org) <bitbake-devel@lists.openembedded.org>
> > Subject: Re: [oe] [bitbake-devel] Backport changes for _PYTHON_SYSCONFIGDATA_NAME to Gatesgarth and Dunfell
> >
> >
> >
> > On Sun, Feb 7, 2021 at 12:15 AM Martin Jansa via lists.openembedded.org <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
> >
> > On Sat, Jan 16, 2021 at 6:52 PM akuster808 <akuster808@gmail.com> wrote:
> >
> >
> >
> > On 1/16/21 9:44 AM, Martin Jansa wrote:
> > > Aren't the missing spaces in appends fixes also needed for meta-oe
> > > recipes?
> >
> >
> > >
> > > I think at least top 5 commits from:
> > > https://git.openembedded.org/meta-openembedded/log/?qt=grep&q=space.*append
> > > were also follow-up from these changes in oe-core.
> > >
> > Do you mean something beyond the meta-openembedded commits mentioned
> > near the bottom of the email?
> >
> >
> >
> > Yes I mean these 5 commits at least.
> >
> >
> >
> > 50bbf80abf python3-pykwalify: Do not unset _PYTHON_SYSCONFIGDATA_NAME
> >
> > 6b3e3bdaf8 python-grpcio-tools: Add missing space for append
> >
> >
> >
> > The first one is just an additional cleanup, but without the 2nd one and with the cherry-picks from the first e-mail applied you would get:
> >
> > ERROR: Nothing PROVIDES 'python3python3-grpcio' (but meta-oe/meta-python/recipes-devtools/python/python3-grpcio-tools_1.14.1.bb DEPENDS on or otherwise requires it). Close matches:
> >   python-grpcio
> >   python3-grpcio
> >
> >
> >
> > my world build is still running..
> >
> >
> >
> >
> >
> > with c99bb790 DEPENDS variable doesn't end with a space, so the missing leading space in these appends (which was fine until now because of trailing space from DEPENDS set in bbclass) is now causing wrong dependency (should be easily reproducible with the patches backported, just by parsing the recipes).
> >
> >
> >
> > I need clarity so I can open an issue in gitlab for tracking purposes as
> > I am sure I will forget this
> >
> >
> >
> > we're using gitlab?
> >
> >
> >
> >
> >
>
> 
>