Re: [bitbake-devel] [dunfell 1/5] python3: split python target configuration into own class

Re: [bitbake-devel] [dunfell 1/5] python3: split python target configuration into own class

[Ernst Sjöstrand]

[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]

Doh, wrong list! Sorry for the spam.
//E

Den tis 1 dec. 2020 kl 17:27 skrev Ernst Sjöstrand via
lists.openembedded.org <ernstp=gmail.com@lists.openembedded.org>:

> From: Alexander Kanavin <alex.kanavin@gmail.com>
>
> Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was
> problematic as it was leaking into host python environment, which
> was causing tracebacks depending on host distro and action
> (typically anything involving importing sysconfig module).
>
> The new class sets the variable only in specific tasks where it is needed,
> and should be inherited explicitly:
> - use python3native to run scripts with native python
> - use python3targetconfig to run scripts with native python
> if those scripts need to access target config data (such
> as correct installation directories). This also adds a dependency
> on target python, so should be used carefully to avoid lengthening builds.
>
> (From OE-Core rev: 5a118d4e7985fa88f04c3611f8db813f0dafce75)
>
> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> (cherry picked from commit c9f7ed88a946078c27fa1a62d54d6310fad9d37a)
> ---
>  meta/classes/python3native.bbclass       |  2 --
>  meta/classes/python3targetconfig.bbclass | 15 +++++++++++++++
>  2 files changed, 15 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/python3targetconfig.bbclass
>
> diff --git a/meta/classes/python3native.bbclass
> b/meta/classes/python3native.bbclass
> index d98fb4c758..2e3a88c126 100644
> --- a/meta/classes/python3native.bbclass
> +++ b/meta/classes/python3native.bbclass
> @@ -17,8 +17,6 @@ export STAGING_LIBDIR
>  export PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
>  export PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
>
> -export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> -
>  # suppress host user's site-packages dirs.
>  export PYTHONNOUSERSITE = "1"
>
> diff --git a/meta/classes/python3targetconfig.bbclass
> b/meta/classes/python3targetconfig.bbclass
> new file mode 100644
> index 0000000000..640d0c97b6
> --- /dev/null
> +++ b/meta/classes/python3targetconfig.bbclass
> @@ -0,0 +1,15 @@
> +inherit python3native
> +
> +DEPENDS_append = " python3"
> +
> +do_configure_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> +
> +do_compile_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> +
> +do_install_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> --
> 2.29.2
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3616 bytes --]

Re: [bitbake-devel] [dunfell 1/5] python3: split python target configuration into own class

[Ernst Sjöstrand]

[-- Attachment #1: Type: text/plain, Size: 3943 bytes --]

Yeah I was preparing for both lists and got it mixed up. :-) There's
another patch that goes to the bitbake list.
(bitbake: lib/bb/fetch2/__init__.py: drop _PYTHON_SYSCONFIGDATA_NAME
unsetting)

Ah the patch you mentioned is a bit newer than my initial local backport.

The reason for the backport is that many of the initial reports were for
Dunfell, and that Dunfell will be LTS.

Regards
//Ernst

Den tis 1 dec. 2020 kl 17:32 skrev Martin Jansa <martin.jansa@gmail.com>:

> Also all these e-mails belong to openembedded-core ML not bitbake-devel.
>
> On Tue, Dec 1, 2020 at 5:31 PM Martin Jansa via lists.openembedded.org
> <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:
>
>> Not sure if this is worth backport (it's very useful, but requires
>> changes in other layers - and these changes aren't backwards compatible so
>> they would need to be applied only in dunfell and not gatesgarth branches
>> of these layers or the same patchset would need to land in
>> oe-core/gatesgarth first).
>>
>> But you're definitely missing this fix from master:
>>
>> https://git.openembedded.org/openembedded-core/commit/?id=c99bb79087e74a967286469e1d8888a546ebec83
>>
>> On Tue, Dec 1, 2020 at 5:27 PM Ernst Sjöstrand <ernstp@gmail.com> wrote:
>>
>>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>>
>>> Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was
>>> problematic as it was leaking into host python environment, which
>>> was causing tracebacks depending on host distro and action
>>> (typically anything involving importing sysconfig module).
>>>
>>> The new class sets the variable only in specific tasks where it is
>>> needed,
>>> and should be inherited explicitly:
>>> - use python3native to run scripts with native python
>>> - use python3targetconfig to run scripts with native python
>>> if those scripts need to access target config data (such
>>> as correct installation directories). This also adds a dependency
>>> on target python, so should be used carefully to avoid lengthening
>>> builds.
>>>
>>> (From OE-Core rev: 5a118d4e7985fa88f04c3611f8db813f0dafce75)
>>>
>>> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
>>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>>> (cherry picked from commit c9f7ed88a946078c27fa1a62d54d6310fad9d37a)
>>> ---
>>>  meta/classes/python3native.bbclass       |  2 --
>>>  meta/classes/python3targetconfig.bbclass | 15 +++++++++++++++
>>>  2 files changed, 15 insertions(+), 2 deletions(-)
>>>  create mode 100644 meta/classes/python3targetconfig.bbclass
>>>
>>> diff --git a/meta/classes/python3native.bbclass
>>> b/meta/classes/python3native.bbclass
>>> index d98fb4c758..2e3a88c126 100644
>>> --- a/meta/classes/python3native.bbclass
>>> +++ b/meta/classes/python3native.bbclass
>>> @@ -17,8 +17,6 @@ export STAGING_LIBDIR
>>>  export
>>> PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
>>>  export PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
>>>
>>> -export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>>> -
>>>  # suppress host user's site-packages dirs.
>>>  export PYTHONNOUSERSITE = "1"
>>>
>>> diff --git a/meta/classes/python3targetconfig.bbclass
>>> b/meta/classes/python3targetconfig.bbclass
>>> new file mode 100644
>>> index 0000000000..640d0c97b6
>>> --- /dev/null
>>> +++ b/meta/classes/python3targetconfig.bbclass
>>> @@ -0,0 +1,15 @@
>>> +inherit python3native
>>> +
>>> +DEPENDS_append = " python3"
>>> +
>>> +do_configure_prepend() {
>>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>>> +}
>>> +
>>> +do_compile_prepend() {
>>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>>> +}
>>> +
>>> +do_install_prepend() {
>>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>>> +}
>>> --
>>> 2.29.2
>>>
>>>
>>>
>>>
>>>
>> 
>>
>>

[-- Attachment #2: Type: text/html, Size: 6109 bytes --]

Re: [bitbake-devel] [dunfell 1/5] python3: split python target configuration into own class

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3350 bytes --]

Also all these e-mails belong to openembedded-core ML not bitbake-devel.

On Tue, Dec 1, 2020 at 5:31 PM Martin Jansa via lists.openembedded.org
<Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

> Not sure if this is worth backport (it's very useful, but requires changes
> in other layers - and these changes aren't backwards compatible so they
> would need to be applied only in dunfell and not gatesgarth branches of
> these layers or the same patchset would need to land in oe-core/gatesgarth
> first).
>
> But you're definitely missing this fix from master:
>
> https://git.openembedded.org/openembedded-core/commit/?id=c99bb79087e74a967286469e1d8888a546ebec83
>
> On Tue, Dec 1, 2020 at 5:27 PM Ernst Sjöstrand <ernstp@gmail.com> wrote:
>
>> From: Alexander Kanavin <alex.kanavin@gmail.com>
>>
>> Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was
>> problematic as it was leaking into host python environment, which
>> was causing tracebacks depending on host distro and action
>> (typically anything involving importing sysconfig module).
>>
>> The new class sets the variable only in specific tasks where it is needed,
>> and should be inherited explicitly:
>> - use python3native to run scripts with native python
>> - use python3targetconfig to run scripts with native python
>> if those scripts need to access target config data (such
>> as correct installation directories). This also adds a dependency
>> on target python, so should be used carefully to avoid lengthening builds.
>>
>> (From OE-Core rev: 5a118d4e7985fa88f04c3611f8db813f0dafce75)
>>
>> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> (cherry picked from commit c9f7ed88a946078c27fa1a62d54d6310fad9d37a)
>> ---
>>  meta/classes/python3native.bbclass       |  2 --
>>  meta/classes/python3targetconfig.bbclass | 15 +++++++++++++++
>>  2 files changed, 15 insertions(+), 2 deletions(-)
>>  create mode 100644 meta/classes/python3targetconfig.bbclass
>>
>> diff --git a/meta/classes/python3native.bbclass
>> b/meta/classes/python3native.bbclass
>> index d98fb4c758..2e3a88c126 100644
>> --- a/meta/classes/python3native.bbclass
>> +++ b/meta/classes/python3native.bbclass
>> @@ -17,8 +17,6 @@ export STAGING_LIBDIR
>>  export
>> PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
>>  export PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
>>
>> -export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>> -
>>  # suppress host user's site-packages dirs.
>>  export PYTHONNOUSERSITE = "1"
>>
>> diff --git a/meta/classes/python3targetconfig.bbclass
>> b/meta/classes/python3targetconfig.bbclass
>> new file mode 100644
>> index 0000000000..640d0c97b6
>> --- /dev/null
>> +++ b/meta/classes/python3targetconfig.bbclass
>> @@ -0,0 +1,15 @@
>> +inherit python3native
>> +
>> +DEPENDS_append = " python3"
>> +
>> +do_configure_prepend() {
>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>> +}
>> +
>> +do_compile_prepend() {
>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>> +}
>> +
>> +do_install_prepend() {
>> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
>> +}
>> --
>> 2.29.2
>>
>>
>>
>>
>>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4508 bytes --]

Re: [bitbake-devel] [dunfell 1/5] python3: split python target configuration into own class

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3040 bytes --]

Not sure if this is worth backport (it's very useful, but requires changes
in other layers - and these changes aren't backwards compatible so they
would need to be applied only in dunfell and not gatesgarth branches of
these layers or the same patchset would need to land in oe-core/gatesgarth
first).

But you're definitely missing this fix from master:
https://git.openembedded.org/openembedded-core/commit/?id=c99bb79087e74a967286469e1d8888a546ebec83

On Tue, Dec 1, 2020 at 5:27 PM Ernst Sjöstrand <ernstp@gmail.com> wrote:

> From: Alexander Kanavin <alex.kanavin@gmail.com>
>
> Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was
> problematic as it was leaking into host python environment, which
> was causing tracebacks depending on host distro and action
> (typically anything involving importing sysconfig module).
>
> The new class sets the variable only in specific tasks where it is needed,
> and should be inherited explicitly:
> - use python3native to run scripts with native python
> - use python3targetconfig to run scripts with native python
> if those scripts need to access target config data (such
> as correct installation directories). This also adds a dependency
> on target python, so should be used carefully to avoid lengthening builds.
>
> (From OE-Core rev: 5a118d4e7985fa88f04c3611f8db813f0dafce75)
>
> Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> (cherry picked from commit c9f7ed88a946078c27fa1a62d54d6310fad9d37a)
> ---
>  meta/classes/python3native.bbclass       |  2 --
>  meta/classes/python3targetconfig.bbclass | 15 +++++++++++++++
>  2 files changed, 15 insertions(+), 2 deletions(-)
>  create mode 100644 meta/classes/python3targetconfig.bbclass
>
> diff --git a/meta/classes/python3native.bbclass
> b/meta/classes/python3native.bbclass
> index d98fb4c758..2e3a88c126 100644
> --- a/meta/classes/python3native.bbclass
> +++ b/meta/classes/python3native.bbclass
> @@ -17,8 +17,6 @@ export STAGING_LIBDIR
>  export PYTHON_LIBRARY="${STAGING_LIBDIR}/lib${PYTHON_DIR}${PYTHON_ABI}.so"
>  export PYTHON_INCLUDE_DIR="${STAGING_INCDIR}/${PYTHON_DIR}${PYTHON_ABI}"
>
> -export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> -
>  # suppress host user's site-packages dirs.
>  export PYTHONNOUSERSITE = "1"
>
> diff --git a/meta/classes/python3targetconfig.bbclass
> b/meta/classes/python3targetconfig.bbclass
> new file mode 100644
> index 0000000000..640d0c97b6
> --- /dev/null
> +++ b/meta/classes/python3targetconfig.bbclass
> @@ -0,0 +1,15 @@
> +inherit python3native
> +
> +DEPENDS_append = " python3"
> +
> +do_configure_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> +
> +do_compile_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> +
> +do_install_prepend() {
> +        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
> +}
> --
> 2.29.2
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 3921 bytes --]