[oe-core][PATCH] python3: add ${bindir}/python symlink

[oe-core][PATCH] python3: add ${bindir}/python symlink

[Markus Volk]

Currently /usr/bin/python is provided by the python2 package. If python2
is not installed, the built image lacks a provider for it.
This results in failed scripts when using '/usr/bin/python' shebang.

This patch adds a /usr/bin/python symlink for python3 to fix this issue.
For images containing python2 and python3, the do_rootfs task will fail
because now both of them provide the same file.
To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
if set, will prevent python3 from creating the symlink.

The link is created with relative path because using an absolute path would fail
for native and nativesdk.

Signed-off-by: Markus Volk <f_l_k@t-online.de>
---
 meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
 meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
index 64203cf0fc..7b6f509a45 100644
--- a/meta/recipes-devtools/python/python3/python3-manifest.json
+++ b/meta/recipes-devtools/python/python3/python3-manifest.json
@@ -203,6 +203,7 @@
         "files": [
             "${bindir}/python${PYTHON_MAJMIN}",
             "${bindir}/python${PYTHON_MAJMIN}.real",
+            "${bindir}/python",
             "${bindir}/python3",
             "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
             "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..18b631f79d 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -144,6 +144,7 @@ do_install:prepend() {
 
 do_install:append:class-target() {
         oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
 }
 
 do_install:append:class-native() {
@@ -156,6 +157,7 @@ do_install:append:class-native() {
         # (these often end up too long for the #! parser in the kernel as the
         # buffer is 128 bytes long).
         ln -s python3-native/python3 ${D}${bindir}/nativepython3
+        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}
 
         # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
         # and the overhead in each recipe-sysroot-native isn't worth it, particularly
@@ -213,6 +215,7 @@ do_install:append() {
 }
 
 do_install:append:class-nativesdk () {
+    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
     # Make sure we use /usr/bin/env python
     for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
          sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
@@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX
 
 # For historical reasons PN is empty and provided by python3-modules
 FILES:${PN} = ""
+RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
 RPROVIDES:${PN}-modules = "${PN}"
 
 FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
-- 
2.34.1

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

I have to ask you to resend and drop the DISTRO_FEATURE idea. We do
not want to make the impression that python2 is supported in any way,
and as explained earlier this implies that all scripts that ask for
python would work with both 2 and 3. No, this is a clean break, and
anyone who still has 2.x stuff has to adjust it to ask for
#!/usr/bin/python2.

Alex

On Sun, 20 Nov 2022 at 18:53, Markus Volk <f_l_k@t-online.de> wrote:
>
> Currently /usr/bin/python is provided by the python2 package. If python2
> is not installed, the built image lacks a provider for it.
> This results in failed scripts when using '/usr/bin/python' shebang.
>
> This patch adds a /usr/bin/python symlink for python3 to fix this issue.
> For images containing python2 and python3, the do_rootfs task will fail
> because now both of them provide the same file.
> To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced which,
> if set, will prevent python3 from creating the symlink.
>
> The link is created with relative path because using an absolute path would fail
> for native and nativesdk.
>
> Signed-off-by: Markus Volk <f_l_k@t-online.de>
> ---
>  meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
>  meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
>  2 files changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json b/meta/recipes-devtools/python/python3/python3-manifest.json
> index 64203cf0fc..7b6f509a45 100644
> --- a/meta/recipes-devtools/python/python3/python3-manifest.json
> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json
> @@ -203,6 +203,7 @@
>          "files": [
>              "${bindir}/python${PYTHON_MAJMIN}",
>              "${bindir}/python${PYTHON_MAJMIN}.real",
> +            "${bindir}/python",
>              "${bindir}/python3",
>              "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
>              "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
> index 92a1f69320..18b631f79d 100644
> --- a/meta/recipes-devtools/python/python3_3.11.0.bb
> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb
> @@ -144,6 +144,7 @@ do_install:prepend() {
>
>  do_install:append:class-target() {
>          oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
>  }
>
>  do_install:append:class-native() {
> @@ -156,6 +157,7 @@ do_install:append:class-native() {
>          # (these often end up too long for the #! parser in the kernel as the
>          # buffer is 128 bytes long).
>          ln -s python3-native/python3 ${D}${bindir}/nativepython3
> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3-native/python3 ${D}${bindir}/nativepython', d)}
>
>          # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
>          # and the overhead in each recipe-sysroot-native isn't worth it, particularly
> @@ -213,6 +215,7 @@ do_install:append() {
>  }
>
>  do_install:append:class-nativesdk () {
> +    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf ./python3 ${D}${bindir}/python', d)}
>      # Make sure we use /usr/bin/env python
>      for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
>           sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = " ${MLPREFIX}openssl ${MLPREFIX
>
>  # For historical reasons PN is empty and provided by python3-modules
>  FILES:${PN} = ""
> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', '${bindir}/python', d)}"
>  RPROVIDES:${PN}-modules = "${PN}"
>
>  FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173637): https://lists.openembedded.org/g/openembedded-core/message/173637
> Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Markus Volk]

[-- Attachment #1: Type: text/plain, Size: 472 bytes --]

Am So, 20. Nov 2022 um 19:00:39 +0100 schrieb Alexander Kanavin 
<alex.kanavin@gmail.com>:
> I have to ask you to resend and drop the DISTRO_FEATURE idea. We do
> not want to make the impression that python2 is supported in any way,
> and as explained earlier this implies that all scripts that ask for
> python would work with both 2 and 3. No, this is a clean break, and
> anyone who still has 2.x stuff has to adjust it to ask for
> #!/usr/bin/python2.

I fully agree


[-- Attachment #2: Type: text/html, Size: 636 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 4131 bytes --]

On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote:

> Currently /usr/bin/python is provided by the python2 package. If python2
> is not installed, the built image lacks a provider for it.
> This results in failed scripts when using '/usr/bin/python' shebang.
>
> This patch adds a /usr/bin/python symlink for python3 to fix this issue.
> For images containing python2 and python3, the do_rootfs task will fail
> because now both of them provide the same file.
> To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced
> which,
> if set, will prevent python3 from creating the symlink.
>
> The link is created with relative path because using an absolute path
> would fail
> for native and nativesdk.


What happens if and when python4 comes ?
This might get us into same mess we had migrating python2 to 3


>
> Signed-off-by: Markus Volk <f_l_k@t-online.de>
> ---
>  meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
>  meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
>  2 files changed, 5 insertions(+)
>
> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json
> b/meta/recipes-devtools/python/python3/python3-manifest.json
> index 64203cf0fc..7b6f509a45 100644
> --- a/meta/recipes-devtools/python/python3/python3-manifest.json
> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json
> @@ -203,6 +203,7 @@
>          "files": [
>              "${bindir}/python${PYTHON_MAJMIN}",
>              "${bindir}/python${PYTHON_MAJMIN}.real",
> +            "${bindir}/python",
>              "${bindir}/python3",
>              "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
>              "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb
> b/meta/recipes-devtools/python/python3_3.11.0.bb
> index 92a1f69320..18b631f79d 100644
> --- a/meta/recipes-devtools/python/python3_3.11.0.bb
> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb
> @@ -144,6 +144,7 @@ do_install:prepend() {
>
>  do_install:append:class-target() {
>          oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
> ./python3 ${D}${bindir}/python', d)}
>  }
>
>  do_install:append:class-native() {
> @@ -156,6 +157,7 @@ do_install:append:class-native() {
>          # (these often end up too long for the #! parser in the kernel as
> the
>          # buffer is 128 bytes long).
>          ln -s python3-native/python3 ${D}${bindir}/nativepython3
> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
> ./python3-native/python3 ${D}${bindir}/nativepython', d)}
>
>          # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000
> of them
>          # and the overhead in each recipe-sysroot-native isn't worth it,
> particularly
> @@ -213,6 +215,7 @@ do_install:append() {
>  }
>
>  do_install:append:class-nativesdk () {
> +    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
> ./python3 ${D}${bindir}/python', d)}
>      # Make sure we use /usr/bin/env python
>      for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
>           sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = "
> ${MLPREFIX}openssl ${MLPREFIX
>
>  # For historical reasons PN is empty and provided by python3-modules
>  FILES:${PN} = ""
> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2', '',
> '${bindir}/python', d)}"
>  RPROVIDES:${PN}-modules = "${PN}"
>
>  FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173637):
> https://lists.openembedded.org/g/openembedded-core/message/173637
> Mute This Topic: https://lists.openembedded.org/mt/95156228/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 6097 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 4683 bytes --]

Python community has no plans for python4 whatsoever. Anything but gentle
and steady evolution won’t be well received.

https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/


Alex

On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@gmail.com> wrote:

>
>
> On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote:
>
>> Currently /usr/bin/python is provided by the python2 package. If python2
>> is not installed, the built image lacks a provider for it.
>> This results in failed scripts when using '/usr/bin/python' shebang.
>>
>> This patch adds a /usr/bin/python symlink for python3 to fix this issue.
>> For images containing python2 and python3, the do_rootfs task will fail
>> because now both of them provide the same file.
>> To allow this corner case, a DISTRO_FEATURE 'python2' has been introduced
>> which,
>> if set, will prevent python3 from creating the symlink.
>>
>> The link is created with relative path because using an absolute path
>> would fail
>> for native and nativesdk.
>
>
> What happens if and when python4 comes ?
> This might get us into same mess we had migrating python2 to 3
>
>
>>
>> Signed-off-by: Markus Volk <f_l_k@t-online.de>
>> ---
>>  meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
>>  meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
>>  2 files changed, 5 insertions(+)
>>
>> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json
>> b/meta/recipes-devtools/python/python3/python3-manifest.json
>> index 64203cf0fc..7b6f509a45 100644
>> --- a/meta/recipes-devtools/python/python3/python3-manifest.json
>> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json
>> @@ -203,6 +203,7 @@
>>          "files": [
>>              "${bindir}/python${PYTHON_MAJMIN}",
>>              "${bindir}/python${PYTHON_MAJMIN}.real",
>> +            "${bindir}/python",
>>              "${bindir}/python3",
>>              "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
>>              "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
>> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb
>> b/meta/recipes-devtools/python/python3_3.11.0.bb
>> index 92a1f69320..18b631f79d 100644
>> --- a/meta/recipes-devtools/python/python3_3.11.0.bb
>> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb
>> @@ -144,6 +144,7 @@ do_install:prepend() {
>>
>>  do_install:append:class-target() {
>>          oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
>> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>> ./python3 ${D}${bindir}/python', d)}
>>  }
>>
>>  do_install:append:class-native() {
>> @@ -156,6 +157,7 @@ do_install:append:class-native() {
>>          # (these often end up too long for the #! parser in the kernel
>> as the
>>          # buffer is 128 bytes long).
>>          ln -s python3-native/python3 ${D}${bindir}/nativepython3
>> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>> ./python3-native/python3 ${D}${bindir}/nativepython', d)}
>>
>>          # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000
>> of them
>>          # and the overhead in each recipe-sysroot-native isn't worth it,
>> particularly
>> @@ -213,6 +215,7 @@ do_install:append() {
>>  }
>>
>>  do_install:append:class-nativesdk () {
>> +    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>> ./python3 ${D}${bindir}/python', d)}
>>      # Make sure we use /usr/bin/env python
>>      for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
>>           sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = "
>> ${MLPREFIX}openssl ${MLPREFIX
>>
>>  # For historical reasons PN is empty and provided by python3-modules
>>  FILES:${PN} = ""
>> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2',
>> '', '${bindir}/python', d)}"
>>  RPROVIDES:${PN}-modules = "${PN}"
>>
>>  FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
>> --
>> 2.34.1
>>
>>
>>
>>
>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173642):
> https://lists.openembedded.org/g/openembedded-core/message/173642
> Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 7248 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 5270 bytes --]

On Sun, Nov 20, 2022 at 3:56 PM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> Python community has no plans for python4 whatsoever. Anything but gentle
> and steady evolution won’t be well received.
>
>
> https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/
>

Good news article although it does not say that python3 and python are
analogous and python package does not explicitly install python binary or
symlink either so it’s left up to how distros want to presume it. If we
want to do that then it would be prudent to see what other distributions
are doing in this regard. So we are not alone in case trouble arises

>
>
> Alex
>
> On Mon 21. Nov 2022 at 0.17, Khem Raj <raj.khem@gmail.com> wrote:
>
>>
>>
>> On Sun, Nov 20, 2022 at 9:52 AM Markus Volk <f_l_k@t-online.de> wrote:
>>
>>> Currently /usr/bin/python is provided by the python2 package. If python2
>>> is not installed, the built image lacks a provider for it.
>>> This results in failed scripts when using '/usr/bin/python' shebang.
>>>
>>> This patch adds a /usr/bin/python symlink for python3 to fix this issue.
>>> For images containing python2 and python3, the do_rootfs task will fail
>>> because now both of them provide the same file.
>>> To allow this corner case, a DISTRO_FEATURE 'python2' has been
>>> introduced which,
>>> if set, will prevent python3 from creating the symlink.
>>>
>>> The link is created with relative path because using an absolute path
>>> would fail
>>> for native and nativesdk.
>>
>>
>> What happens if and when python4 comes ?
>> This might get us into same mess we had migrating python2 to 3
>>
>>
>>>
>>> Signed-off-by: Markus Volk <f_l_k@t-online.de>
>>> ---
>>>  meta/recipes-devtools/python/python3/python3-manifest.json | 1 +
>>>  meta/recipes-devtools/python/python3_3.11.0.bb             | 4 ++++
>>>  2 files changed, 5 insertions(+)
>>>
>>> diff --git a/meta/recipes-devtools/python/python3/python3-manifest.json
>>> b/meta/recipes-devtools/python/python3/python3-manifest.json
>>> index 64203cf0fc..7b6f509a45 100644
>>> --- a/meta/recipes-devtools/python/python3/python3-manifest.json
>>> +++ b/meta/recipes-devtools/python/python3/python3-manifest.json
>>> @@ -203,6 +203,7 @@
>>>          "files": [
>>>              "${bindir}/python${PYTHON_MAJMIN}",
>>>              "${bindir}/python${PYTHON_MAJMIN}.real",
>>> +            "${bindir}/python",
>>>              "${bindir}/python3",
>>>              "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h",
>>>              "${libdir}/python${PYTHON_MAJMIN}/UserDict.py",
>>> diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb
>>> b/meta/recipes-devtools/python/python3_3.11.0.bb
>>> index 92a1f69320..18b631f79d 100644
>>> --- a/meta/recipes-devtools/python/python3_3.11.0.bb
>>> +++ b/meta/recipes-devtools/python/python3_3.11.0.bb
>>> @@ -144,6 +144,7 @@ do_install:prepend() {
>>>
>>>  do_install:append:class-target() {
>>>          oe_multilib_header python${PYTHON_MAJMIN}/pyconfig.h
>>> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>>> ./python3 ${D}${bindir}/python', d)}
>>>  }
>>>
>>>  do_install:append:class-native() {
>>> @@ -156,6 +157,7 @@ do_install:append:class-native() {
>>>          # (these often end up too long for the #! parser in the kernel
>>> as the
>>>          # buffer is 128 bytes long).
>>>          ln -s python3-native/python3 ${D}${bindir}/nativepython3
>>> +        ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>>> ./python3-native/python3 ${D}${bindir}/nativepython', d)}
>>>
>>>          # Remove the opt-1.pyc and opt-2.pyc files. There are over
>>> 3,000 of them
>>>          # and the overhead in each recipe-sysroot-native isn't worth
>>> it, particularly
>>> @@ -213,6 +215,7 @@ do_install:append() {
>>>  }
>>>
>>>  do_install:append:class-nativesdk () {
>>> +    ${@bb.utils.contains('DISTRO_FEATURES', 'python2', '', 'ln -sf
>>> ./python3 ${D}${bindir}/python', d)}
>>>      # Make sure we use /usr/bin/env python
>>>      for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
>>>           sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>>> @@ -376,6 +379,7 @@ RRECOMMENDS:${PN}-crypt:append:class-nativesdk = "
>>> ${MLPREFIX}openssl ${MLPREFIX
>>>
>>>  # For historical reasons PN is empty and provided by python3-modules
>>>  FILES:${PN} = ""
>>> +RPROVIDES:${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'python2',
>>> '', '${bindir}/python', d)}"
>>>  RPROVIDES:${PN}-modules = "${PN}"
>>>
>>>  FILES:${PN}-pydoc += "${bindir}/pydoc${PYTHON_MAJMIN} ${bindir}/pydoc3"
>>> --
>>> 2.34.1
>>>
>>>
>>>
>>>
>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>>
> View/Reply Online (#173642):
>> https://lists.openembedded.org/g/openembedded-core/message/173642
>> Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> alex.kanavin@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>

[-- Attachment #2: Type: text/html, Size: 8700 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@gmail.com> wrote:
>> Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.
>>
>> https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/
>
>
> Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises

Fedora has done it 3 years ago:
https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff

Alex

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

And here's debian:
https://wiki.debian.org/Python/FAQ#Python_2_support

On my Debian systems, indeed, /usr/bin/python is absent.

Alex

On Mon, 21 Nov 2022 at 18:18, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
>
> On Mon, 21 Nov 2022 at 16:17, Khem Raj <raj.khem@gmail.com> wrote:
> >> Python community has no plans for python4 whatsoever. Anything but gentle and steady evolution won’t be well received.
> >>
> >> https://www.techrepublic.com/article/programming-languages-why-python-4-0-will-probably-never-arrive-according-to-its-creator/
> >
> >
> > Good news article although it does not say that python3 and python are analogous and python package does not explicitly install python binary or symlink either so it’s left up to how distros want to presume it. If we want to do that then it would be prudent to see what other distributions are doing in this regard. So we are not alone in case trouble arises
>
> Fedora has done it 3 years ago:
> https://pagure.io/fedora-docs/release-notes/pull-request/410#request_diff
>
> Alex
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173658): https://lists.openembedded.org/g/openembedded-core/message/173658
> Mute This Topic: https://lists.openembedded.org/mt/95156228/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[f_l_k]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin 
<alex.kanavin@gmail.com> wrote:
> On my Debian systems, indeed, /usr/bin/python is absent.

Debian has a package for this
<https://packages.debian.org/bookworm/python-is-python3>


[-- Attachment #2: Type: text/html, Size: 485 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Ross Burton]

On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=t-online.de@lists.openembedded.org> wrote:
> On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>> On my Debian systems, indeed, /usr/bin/python is absent.
> 
> Debian has a package for this
> https://packages.debian.org/bookworm/python-is-python3

And that’s a perfectly good solution that I like.

Personally, I think people should forget that /usr/bin/python exists: the recommendation from Python is to call python2 or python3.  I can quote from PEP-0394:

“””
Depending on a distribution or system configuration, python may or may not be installed. If python is installed its target interpreter may refer to python2 or python3.”

…

    • Distributors may choose to set the behavior of the python command as follows:
        • python2,
        • python3,
        • not provide python command,
        • allow python to be configurable by an end user or a system administrator.
“””

We’ve picked option 3.  As per Python upstream, that’s absolutely fine.

If you have a serious need that /usr/bin/python exists, and is a symlink to python3, then could you not make a simple recipe that RDEPENDS on python3 and ships just a /usr/bin/python -> python3 symlink?  You can even put this in your layer to avoid having to debate it with the oe-core maintainers.

Ross

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1864 bytes --]

The serious need is that we need to patch all the scripts that ask for
python to add a 3 to it. And there will be more of these going forward, not
less. I’d rather just always have python available. Not a problem worth
deliberating over to be honest.

Alex

On Tue 22. Nov 2022 at 18.41, Ross Burton <Ross.Burton@arm.com> wrote:

> On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org <f_l_k=
> t-online.de@lists.openembedded.org> wrote:
> > On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin <
> alex.kanavin@gmail.com> wrote:
> >> On my Debian systems, indeed, /usr/bin/python is absent.
> >
> > Debian has a package for this
> > https://packages.debian.org/bookworm/python-is-python3
>
> And that’s a perfectly good solution that I like.
>
> Personally, I think people should forget that /usr/bin/python exists: the
> recommendation from Python is to call python2 or python3.  I can quote from
> PEP-0394:
>
> “””
> Depending on a distribution or system configuration, python may or may not
> be installed. If python is installed its target interpreter may refer to
> python2 or python3.”
>
> …
>
>     • Distributors may choose to set the behavior of the python command as
> follows:
>         • python2,
>         • python3,
>         • not provide python command,
>         • allow python to be configurable by an end user or a system
> administrator.
> “””
>
> We’ve picked option 3.  As per Python upstream, that’s absolutely fine.
>
> If you have a serious need that /usr/bin/python exists, and is a symlink
> to python3, then could you not make a simple recipe that RDEPENDS on
> python3 and ships just a /usr/bin/python -> python3 symlink?  You can even
> put this in your layer to avoid having to debate it with the oe-core
> maintainers.
>
> Ross

[-- Attachment #2: Type: text/html, Size: 2620 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Markus Volk]

[-- Attachment #1: Type: text/plain, Size: 3095 bytes --]

I use a bbappend for python3 in meta-wayland for almost a year now 
because i had issues installing python scripts with /usr/bin/python 
shebang e.g. here in sway:
<https://github.com/swaywm/sway/blob/master/contrib/inactive-windows-transparency.py>

Issue has been that, if adding this script to do_install(), package_qa 
failed because of the missing /usr/bin/python provider.

Ran into this quite a few times in the past and then created the 
symlink instead of patching shebangs

I just wanted to mention, that having a symlink would be an alternative 
to patching shebangs and as i was asked to send a patch, i did. But I'm 
fine with whatever is done in oe-core.

Basically I also think the best solution would be, upstream would 
deprecate /usr/bin/python usage. Unfortunaltey it's not, and so there 
will always be people using it.

Am Di, 22. Nov 2022 um 17:41:22 +0000 schrieb Ross Burton 
<ross.burton@arm.com>:
> On 22 Nov 2022, at 04:42, Markus Volk via lists.openembedded.org 
> <f_l_k=t-online.de@lists.openembedded.org 
> <mailto:f_l_k=t-online.de@lists.openembedded.org>> wrote:
>>  On Mon, Nov 21 2022 at 06:48:07 PM +0100, Alexander Kanavin 
>> <alex.kanavin@gmail.com <mailto:alex.kanavin@gmail.com>> wrote:
>>>  On my Debian systems, indeed, /usr/bin/python is absent.
>> 
>>  Debian has a package for this
>>  <https://packages.debian.org/bookworm/python-is-python3>
> 
> And that’s a perfectly good solution that I like.
> 
> Personally, I think people should forget that /usr/bin/python exists: 
> the recommendation from Python is to call python2 or python3.  I can 
> quote from PEP-0394:
> 
> “””
> Depending on a distribution or system configuration, python may or 
> may not be installed. If python is installed its target interpreter 
> may refer to python2 or python3.”
> 
> …
> 
>     • Distributors may choose to set the behavior of the python 
> command as follows:
>         • python2,
>         • python3,
>         • not provide python command,
>         • allow python to be configurable by an end user or a 
> system administrator.
> “””
> 
> We’ve picked option 3.  As per Python upstream, that’s absolutely 
> fine.
> 
> If you have a serious need that /usr/bin/python exists, and is a 
> symlink to python3, then could you not make a simple recipe that 
> RDEPENDS on python3 and ships just a /usr/bin/python -> python3 
> symlink?  You can even put this in your layer to avoid having to 
> debate it with the oe-core maintainers.
> 
> Ross
> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173694): 
> <https://lists.openembedded.org/g/openembedded-core/message/173694>
> Mute This Topic: <https://lists.openembedded.org/mt/95156228/3618223>
> Group Owner: openembedded-core+owner@lists.openembedded.org 
> <mailto:openembedded-core+owner@lists.openembedded.org>
> Unsubscribe: 
> <https://lists.openembedded.org/g/openembedded-core/unsub> 
> [f_l_k@t-online.de <mailto:f_l_k@t-online.de>]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


[-- Attachment #2: Type: text/html, Size: 3765 bytes --]

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Richard Purdie]

On Tue, 2022-11-22 at 19:03 +0100, Alexander Kanavin wrote:
> The serious need is that we need to patch all the scripts that ask
> for python to add a 3 to it. And there will be more of these going
> forward, not less. I’d rather just always have python available. Not
> a problem worth deliberating over to be honest.

I don't think it is that simple. 

a) If we take the change, we're no longer explicit in what we're using
which means users can be taken by surprise when a python2 script finds
python3 and could sometimes break.

b) Our current approach means that people have at least looked at and
decided something is python3 safe.

c) It means that anyone still using python2 components is left with no
way to make things work. Whilst nobody should be using python2 anymore,
I suspect some still are and this will be a huge problem to them.

d) If there ever is a python4, we'll have to redo a lot of the work
we're done to reach this point where things are fairly clean.


Rightly or wrongly, this patch will cause large amounts of pain for
some portion of our userbase and I'm not sure we have enough
justification to do that. That pain wouldn't likely be realised for
some time either :/.

Cheers,

Richard

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

On Tue, 22 Nov 2022 at 19:14, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> Rightly or wrongly, this patch will cause large amounts of pain for
> some portion of our userbase and I'm not sure we have enough
> justification to do that. That pain wouldn't likely be realised for
> some time either :/.

I have to point out that meta-python2 hasn't even received a
compatibility update for langsdale:
https://git.openembedded.org/meta-python2

While this might be 'too soon' to conclude that python2 is truly dead,
maybe a year (or two, or three) from now it won't be. Fedora has
already made the switch, Debian will follow, and honestly, I just
can't muster any sympathy for python2 users anymore. You can't push
back paying off technical debt forever and expect others to
accommodate you.

Let me propose this: a PACKAGECONFIG for the python recipe that adds
and installs the symlink in a dedicated package. We can keep it off
for now, but somewhere down the line we could revisit that against
established practice and what PEPs say then.

Alex

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexandre Belloni]

On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote:
> On Tue, 22 Nov 2022 at 19:14, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > Rightly or wrongly, this patch will cause large amounts of pain for
> > some portion of our userbase and I'm not sure we have enough
> > justification to do that. That pain wouldn't likely be realised for
> > some time either :/.
> 
> I have to point out that meta-python2 hasn't even received a
> compatibility update for langsdale:
> https://git.openembedded.org/meta-python2
> 
> While this might be 'too soon' to conclude that python2 is truly dead,
> maybe a year (or two, or three) from now it won't be. Fedora has
> already made the switch, Debian will follow, and honestly, I just
> can't muster any sympathy for python2 users anymore. You can't push
> back paying off technical debt forever and expect others to
> accommodate you.
> 
> Let me propose this: a PACKAGECONFIG for the python recipe that adds
> and installs the symlink in a dedicated package. We can keep it off
> for now, but somewhere down the line we could revisit that against
> established practice and what PEPs say then.
> 

I actually like the idea of having a python-is-python3 package so that
affected recipes could simply add it to their dependencies. That would
make it explicit that python is not python2.

> Alex

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173698): https://lists.openembedded.org/g/openembedded-core/message/173698
> Mute This Topic: https://lists.openembedded.org/mt/95156228/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

RE: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Alexandre Belloni via
> lists.openembedded.org
> Sent: den 22 november 2022 23:29
> To: Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: Richard Purdie <richard.purdie@linuxfoundation.org>; Ross Burton
> <Ross.Burton@arm.com>; Khem Raj <raj.khem@gmail.com>; Markus Volk
> <f_l_k@t-online.de>; openembedded-core@lists.openembedded.org
> Subject: Re: [oe-core][PATCH] python3: add ${bindir}/python symlink
> 
> On 22/11/2022 19:32:17+0100, Alexander Kanavin wrote:
> > On Tue, 22 Nov 2022 at 19:14, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > Rightly or wrongly, this patch will cause large amounts of pain for
> > > some portion of our userbase and I'm not sure we have enough
> > > justification to do that. That pain wouldn't likely be realised for
> > > some time either :/.
> >
> > I have to point out that meta-python2 hasn't even received a
> > compatibility update for langsdale:
> > https://git.openembedded.org/meta-python2
> >
> > While this might be 'too soon' to conclude that python2 is truly dead,
> > maybe a year (or two, or three) from now it won't be. Fedora has
> > already made the switch, Debian will follow, and honestly, I just
> > can't muster any sympathy for python2 users anymore. You can't push
> > back paying off technical debt forever and expect others to
> > accommodate you.
> >
> > Let me propose this: a PACKAGECONFIG for the python recipe that adds
> > and installs the symlink in a dedicated package. We can keep it off
> > for now, but somewhere down the line we could revisit that against
> > established practice and what PEPs say then.
> >
> 
> I actually like the idea of having a python-is-python3 package so that
> affected recipes could simply add it to their dependencies. That would
> make it explicit that python is not python2.

Please not that in the Debian case, nothing depends on python-is-python3. 
It is only provided so that users can manually install it if they have 
a Python script not provided by Debian that requires it.

And note that Debian also provides a python-is-python2 package, and 
obviously things would eventually fail if packages actually were to 
depend on either as this would open up for conflicts.

Thus all Python scripts in packages provided by Debian are required to 
explicitly specify either python2 or python3 on the shebang line.

> > Alex

//Peter

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Ross Burton]

On 22 Nov 2022, at 18:03, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
> 
> The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.

But as per PEP 0384, /usr/bin/python could be python2, or not there at all.  Software should use python3.

Ross

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Ross Burton]

On 22 Nov 2022, at 18:32, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> While this might be 'too soon' to conclude that python2 is truly dead,
> maybe a year (or two, or three) from now it won't be. Fedora has
> already made the switch, Debian will follow, and honestly, I just
> can't muster any sympathy for python2 users anymore. You can't push
> back paying off technical debt forever and expect others to
> accommodate you.

I absolutely agree with you here, which is why the one true binary should be /usr/bin/python3.

Ross

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Alexander Kanavin]

On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@arm.com> wrote:
> > The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.
>
> But as per PEP 0384, /usr/bin/python could be python2, or not there at all.  Software should use python3.

But would anyone pay attention to the PEP when writing new scripts? If
/usr/bin/python is available by default, as it is already on Fedora,
and probably will be on Debian (in testing/sid there is no choice
between 2 and 3 anymore - 2 is gone, and that clears the road to have
python->python3 installed by default), then it's not unreasonable to
have the script use that. I wouldn't want to know or care about
python's historical baggage. That's why I'm saying we're going to see
more upstreams that are just having #!/usr/bin/python and don't give a
damn, even when you point it out to them.

Alex

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Khem Raj]

On Wed, Nov 23, 2022 at 2:55 AM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
>
> On Wed, 23 Nov 2022 at 10:59, Ross Burton <Ross.Burton@arm.com> wrote:
> > > The serious need is that we need to patch all the scripts that ask for python to add a 3 to it. And there will be more of these going forward, not less. I’d rather just always have python available. Not a problem worth deliberating over to be honest.
> >
> > But as per PEP 0384, /usr/bin/python could be python2, or not there at all.  Software should use python3.
>
> But would anyone pay attention to the PEP when writing new scripts?

they should be pointed to PEP and I hope python tutors teach it from day 1.

If
> /usr/bin/python is available by default, as it is already on Fedora,

There perhaps is a reason for that in Fedora, but it is in conflict
with what PEP0384 is saying

> and probably will be on Debian (in testing/sid there is no choice
> between 2 and 3 anymore - 2 is gone, and that clears the road to have
> python->python3 installed by default), then it's not unreasonable to
> have the script use that. I wouldn't want to know or care about
> python's historical baggage. That's why I'm saying we're going to see
> more upstreams that are just having #!/usr/bin/python and don't give a
> damn, even when you point it out to them.
>

If you want to ignore the baggage then know that there is no
/usr/bin/python anymore.

> Alex

Re: [oe-core][PATCH] python3: add ${bindir}/python symlink

[Ross Burton]

> On 23 Nov 2022, at 16:34, Khem Raj via lists.openembedded.org <raj.khem=gmail.com@lists.openembedded.org> wrote:
> 
> If
>> /usr/bin/python is available by default, as it is already on Fedora,
> 
> There perhaps is a reason for that in Fedora, but it is in conflict
> with what PEP0384 is saying

It’s not in conflict: the PEP codifies that /usr/bin/python can be python3. Or python2. Or not there at all.  Basically, it’s up to the distribution, so people *writing Python* shouldn’t have any assumptions about it.

Ross