[PATCH v2 1/5] python3: correctly adjust include paths in sysconfigdata

[PATCH v2 1/5] python3: correctly adjust include paths in sysconfigdata

[Alexander Kanavin]

This was replacing /usr/include regardless of whether it
was at the beginning of the full path (correct, when building target
python3), or in the middle of it (not correct, when building
native or nativesdk python).

Through various reasons we haven't been bitten by this until now,
but latest setuptools does expose the problem with for example
nativesdk-python3-cffi and nativesdk-python3-cryptography which both
fail without this fix.

I am not aware of anything using INCLDIRSTOMAKE, and it is harder
to adjust correctly due to the value being a list; if something
is using it, we can look at it specifically.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/recipes-devtools/python/python3_3.11.0.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 6b6c983abf..92a1f69320 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -186,8 +186,8 @@ do_install:append() {
                 -e "s,^ 'LIBDIR'.*, 'LIBDIR': '${STAGING_LIBDIR}'\,,g" \
                 -e "s,^ 'INCLUDEDIR'.*, 'INCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
                 -e "s,^ 'CONFINCLUDEDIR'.*, 'CONFINCLUDEDIR': '${STAGING_INCDIR}'\,,g" \
-                -e "/^ 'INCLDIRSTOMAKE'/{N; s,/usr/include,${STAGING_INCDIR},g}" \
-                -e "/^ 'INCLUDEPY'/s,/usr/include,${STAGING_INCDIR},g" \
+                -e "s,^ 'INCLUDEPY'.*, 'INCLUDEPY': '${STAGING_INCDIR}/python${PYTHON_MAJMIN}'\,,g" \
+                -e "s,^ 'CONFINCLUDEPY'.*, 'CONFINCLUDEPY': '${STAGING_INCDIR}/python${PYTHON_MAJMIN}'\,,g" \
                 -e "s,${B},/build/path/unavailable/,g" \
                 $sysconfigfile
         cp $sysconfigfile ${D}${libdir}/python-sysconfigdata/_sysconfigdata.py
-- 
2.30.2

[PATCH v2 2/5] python3: use the standard shell version of python3-config

[Alexander Kanavin]

There is really no reason why we can't: it only needs to be available
in two versions (native/target), and the correct one can be picked
via PATH priority order.

This eliminates two un-upstreamable patches, one of which relies
on soon to be removed distutils.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../python3targetconfig.bbclass               | 17 ++++--
 ...shebang-overflow-on-python-config.py.patch | 14 ++---
 ...hell-version-of-python-config-that-w.patch | 36 ------------
 .../python/python3/python-config.patch        | 55 -------------------
 .../recipes-devtools/python/python3_3.11.0.bb | 18 +++++-
 5 files changed, 33 insertions(+), 107 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
 delete mode 100644 meta/recipes-devtools/python/python3/python-config.patch

diff --git a/meta/classes-recipe/python3targetconfig.bbclass b/meta/classes-recipe/python3targetconfig.bbclass
index 3f89e5e09e..02fdb0c03f 100644
--- a/meta/classes-recipe/python3targetconfig.bbclass
+++ b/meta/classes-recipe/python3targetconfig.bbclass
@@ -10,26 +10,31 @@ EXTRA_PYTHON_DEPENDS ?= ""
 EXTRA_PYTHON_DEPENDS:class-target = "python3"
 DEPENDS:append = " ${EXTRA_PYTHON_DEPENDS}"
 
-do_configure:prepend:class-target() {
+setup_target_config() {
         export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        export PATH=${STAGING_EXECPREFIXDIR}/python-target-config/:$PATH
+}
+
+do_configure:prepend:class-target() {
+        setup_target_config
 }
 
 do_compile:prepend:class-target() {
-        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        setup_target_config
 }
 
 do_install:prepend:class-target() {
-        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        setup_target_config
 }
 
 do_configure:prepend:class-nativesdk() {
-        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        setup_target_config
 }
 
 do_compile:prepend:class-nativesdk() {
-        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        setup_target_config
 }
 
 do_install:prepend:class-nativesdk() {
-        export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        setup_target_config
 }
diff --git a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
index 921da8de7c..db08435004 100644
--- a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
+++ b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -1,7 +1,7 @@
-From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From 78f482b91d94b44a02e02c4580166757119061ea Mon Sep 17 00:00:00 2001
 From: Paulo Neves <ptsneves@gmail.com>
 Date: Tue, 7 Jun 2022 16:16:41 +0200
-Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+Subject: [PATCH] Avoid shebang overflow on python-config.py
 
 The whole native path may be too big, leading to shebang
 overflow. Let's just use the env shebang.
@@ -11,15 +11,16 @@ Denial reason: [1]
 Upstream-Status: Denied [distribution]
 
 [1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
+
 ---
  Makefile.pre.in | 2 ++
  1 file changed, 2 insertions(+)
 
 diff --git a/Makefile.pre.in b/Makefile.pre.in
-index f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176 100644
+index 55c7c46..1f6500a 100644
 --- a/Makefile.pre.in
 +++ b/Makefile.pre.in
-@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
+@@ -2115,6 +2115,8 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
  	@ # Substitution happens here, as the completely-expanded BINDIR
  	@ # is not available in configure
  	sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
@@ -27,7 +28,4 @@ index f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb
 +	sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
  	@ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
  	LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
- 	@  # In OpenEmbedded, always use the python version of the script, the shell
--- 
-2.25.1
-
+ 	@ # On Darwin, always use the python version of the script, the shell
diff --git a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
deleted file mode 100644
index d98f243cb1..0000000000
--- a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-From 2406432449784243b7590009d42bd0e871253b2e Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Tue, 29 Jan 2019 15:03:01 +0100
-Subject: [PATCH] Do not use the shell version of python-config that was
- introduced in 3.4
-
-Revert instead to the original python version: it has our tweaks and
-outputs directories correctly.
-
-Upstream-Status: Inappropriate [oe-specific]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
-
----
- Makefile.pre.in | 9 +++------
- 1 file changed, 3 insertions(+), 6 deletions(-)
-
-diff --git a/Makefile.pre.in b/Makefile.pre.in
-index ee85f35..f0aedb7 100644
---- a/Makefile.pre.in
-+++ b/Makefile.pre.in
-@@ -1640,12 +1640,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
- 	sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
- 	@ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
- 	LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
--	@ # On Darwin, always use the python version of the script, the shell
--	@ # version doesn't use the compiler customizations that are provided
--	@ # in python (_osx_support.py).
--	@if test `uname -s` = Darwin; then \
--		cp python-config.py python-config; \
--	fi
-+	@  # In OpenEmbedded, always use the python version of the script, the shell
-+	@  # version is broken in multiple ways, and doesn't return correct directories
-+	cp python-config.py python-config
- 
- 
- # Install the include files
diff --git a/meta/recipes-devtools/python/python3/python-config.patch b/meta/recipes-devtools/python/python3/python-config.patch
deleted file mode 100644
index 4da399e46e..0000000000
--- a/meta/recipes-devtools/python/python3/python-config.patch
+++ /dev/null
@@ -1,55 +0,0 @@
-From 8632f25ac4e2c53a3c2c8a1b4fc97fc86e8aad5a Mon Sep 17 00:00:00 2001
-From: Tyler Hall <tylerwhall@gmail.com>
-Date: Sun, 4 May 2014 20:06:43 -0400
-Subject: [PATCH] python-config: Revert to using distutils.sysconfig
-
-The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in
-
-12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig
-
-affect the native runtime as well as cross building. Use the old, patched
-implementation which returns paths in the staging directory and for the target,
-as appropriate.
-
-Upstream-Status: Inappropriate [Embedded Specific]
-
-Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
-
----
- Misc/python-config.in | 12 +++++++-----
- 1 file changed, 7 insertions(+), 5 deletions(-)
-
-diff --git a/Misc/python-config.in b/Misc/python-config.in
-index ebd99da..0492e08 100644
---- a/Misc/python-config.in
-+++ b/Misc/python-config.in
-@@ -6,7 +6,9 @@
- import getopt
- import os
- import sys
--import sysconfig
-+import warnings
-+warnings.filterwarnings("ignore", category=DeprecationWarning)
-+from distutils import sysconfig
- 
- valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags',
-               'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir',
-@@ -35,14 +37,14 @@ if '--help' in opt_flags:
- 
- for opt in opt_flags:
-     if opt == '--prefix':
--        print(getvar('prefix'))
-+        print(sysconfig.PREFIX)
- 
-     elif opt == '--exec-prefix':
--        print(getvar('exec_prefix'))
-+        print(sysconfig.EXEC_PREFIX)
- 
-     elif opt in ('--includes', '--cflags'):
--        flags = ['-I' + sysconfig.get_path('include'),
--                 '-I' + sysconfig.get_path('platinclude')]
-+        flags = ['-I' + sysconfig.get_python_inc(),
-+                 '-I' + sysconfig.get_python_inc(plat_specific=True)]
-         if opt == '--cflags':
-             flags.extend(getvar('CFLAGS').split())
-         print(' '.join(flags))
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..4760895e9f 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -16,8 +16,6 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
            file://cgi_py.patch \
            file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \
            ${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \
-           file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \
-           file://python-config.patch \
            file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \
            file://0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch \
            file://0001-bpo-36852-proper-detection-of-mips-architecture-for-.patch \
@@ -172,6 +170,9 @@ do_install:append:class-native() {
         
         # disable the lookup in user's site-packages globally
         sed -i 's#ENABLE_USER_SITE = None#ENABLE_USER_SITE = False#' ${D}${libdir}/python${PYTHON_MAJMIN}/site.py
+
+        # python3-config needs to be in /usr/bin and not in a subdir of it to work properly
+        mv ${D}/${bindir}/${PN}/python*config ${D}/${bindir}/
 }
 
 do_install:append() {
@@ -220,6 +221,19 @@ do_install:append:class-nativesdk () {
     create_wrapper ${D}${bindir}/python${PYTHON_MAJMIN} TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' PYTHONNOUSERSITE='1'
 }
 
+SYSROOT_PREPROCESS_FUNCS:append:class-target = " provide_target_config_script"
+SYSROOT_PREPROCESS_FUNCS:append:class-nativesdk = " provide_target_config_script"
+
+# This is installed into /usr/python-target-config/ and not /usr/bin
+# because adding target sysroot's /usr/bin/ to PATH has unwanted side effects
+# in components erroneously picking up other target executables from it
+provide_target_config_script() {
+        install -d ${SYSROOT_DESTDIR}${prefix}/python-target-config/
+        install ${D}/${bindir}/python3-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
+        install ${D}/${bindir}/python${PYTHON_MAJMIN}-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
+}
+SYSROOT_DIRS += "${prefix}/python-target-config/"
+
 SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py"
 SSTATE_HASHEQUIV_FILEMAP = " \
     populate_sysroot:*/lib*/python3*/_sysconfigdata*.py:${TMPDIR} \
-- 
2.30.2

[PATCH v2 3/5] python3targetconfig.bbclass: use PYTHONPATH to point to the target config

[Alexander Kanavin]

There is no need to patch native python so that it looks in the target
sysroot; the same can be achieved with just an environment variable.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../python3targetconfig.bbclass               |  1 +
 .../classes-recipe/setuptools3_legacy.bbclass |  2 +-
 ...fig-append-STAGING_LIBDIR-python-sys.patch | 28 -------------------
 .../recipes-devtools/python/python3_3.11.0.bb |  1 -
 4 files changed, 2 insertions(+), 30 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch

diff --git a/meta/classes-recipe/python3targetconfig.bbclass b/meta/classes-recipe/python3targetconfig.bbclass
index 02fdb0c03f..22305fe176 100644
--- a/meta/classes-recipe/python3targetconfig.bbclass
+++ b/meta/classes-recipe/python3targetconfig.bbclass
@@ -12,6 +12,7 @@ DEPENDS:append = " ${EXTRA_PYTHON_DEPENDS}"
 
 setup_target_config() {
         export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+        export PYTHONPATH=${STAGING_LIBDIR}/python-sysconfigdata
         export PATH=${STAGING_EXECPREFIXDIR}/python-target-config/:$PATH
 }
 
diff --git a/meta/classes-recipe/setuptools3_legacy.bbclass b/meta/classes-recipe/setuptools3_legacy.bbclass
index 21748f922a..57de956d0a 100644
--- a/meta/classes-recipe/setuptools3_legacy.bbclass
+++ b/meta/classes-recipe/setuptools3_legacy.bbclass
@@ -49,7 +49,7 @@ setuptools3_legacy_do_install() {
         install -d ${D}${PYTHON_SITEPACKAGES_DIR}
         STAGING_INCDIR=${STAGING_INCDIR} \
         STAGING_LIBDIR=${STAGING_LIBDIR} \
-        PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
+        PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR}:$PYTHONPATH \
         ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
         build --build-base=${B} install --skip-build ${SETUPTOOLS_INSTALL_ARGS} || \
         bbfatal_log "'${PYTHON_PN} setup.py install ${SETUPTOOLS_INSTALL_ARGS}' execution failed."
diff --git a/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
deleted file mode 100644
index 45a37ed1a9..0000000000
--- a/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
+++ /dev/null
@@ -1,28 +0,0 @@
-From 9c6b9f46179c8f9c9391767e2b02f268a1ee7a9c Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Thu, 31 Jan 2019 16:46:30 +0100
-Subject: [PATCH] distutils/sysconfig: append
- STAGING_LIBDIR/python-sysconfigdata to sys.path
-
-So that target configuration can be used when running native python
-
-Upstream-Status: Inappropriate [oe-core specific]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
-
----
- Lib/sysconfig.py | 2 ++
- 1 file changed, 2 insertions(+)
-
-diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
-index ff399e2..95844cf 100644
---- a/Lib/sysconfig.py
-+++ b/Lib/sysconfig.py
-@@ -528,6 +528,8 @@ def _init_posix(vars):
-     """Initialize the module as appropriate for POSIX systems."""
-     # _sysconfigdata is generated at build time, see _generate_posix_vars()
-     name = _get_sysconfigdata_name()
-+    if 'STAGING_LIBDIR' in os.environ:
-+        sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
-     _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
-     build_time_vars = _temp.build_time_vars
-     vars.update(build_time_vars)
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 4760895e9f..94f01af83a 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -36,7 +36,6 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
 
 SRC_URI:append:class-native = " \
            file://0001-Lib-sysconfig.py-use-prefix-value-from-build-configu.patch \
-           file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
            file://12-distutils-prefix-is-inside-staging-area.patch \
            file://0001-Don-t-search-system-for-headers-libraries.patch \
            "
-- 
2.30.2

[PATCH v2 4/5] vala: install vapigen-wrapper into /usr/bin/crosscripts and stage only that

[Alexander Kanavin]

Staging the whole /usr/bin is not correct, as it pulls in also
all the vala's cross binaries, which may be discovered by other recipes
and things will go wrong then.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/recipes-devtools/vala/vala.inc | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/vala/vala.inc b/meta/recipes-devtools/vala/vala.inc
index 974baa33f5..d3daee37dc 100644
--- a/meta/recipes-devtools/vala/vala.inc
+++ b/meta/recipes-devtools/vala/vala.inc
@@ -42,20 +42,20 @@ EXTRA_OECONF += " --disable-valadoc"
 # Vapigen wrapper needs to be available system-wide, because it will be used
 # to build vapi files from all other packages with vala support
 do_install:append:class-target() {
-        install -d ${D}${bindir}/
-        install ${B}/vapigen-wrapper ${D}${bindir}/
+        install -d ${D}${bindir_crossscripts}/
+        install ${B}/vapigen-wrapper ${D}${bindir_crossscripts}/
 }
 
 # Put vapigen wrapper into target sysroot so that it can be used when building
 # vapi files.
-SYSROOT_DIRS:append:class-target = " ${bindir}"
+SYSROOT_DIRS += "${bindir_crossscripts}"
 
 SYSROOT_PREPROCESS_FUNCS:append:class-target = " vapigen_sysroot_preprocess"
 vapigen_sysroot_preprocess() {
         # Tweak the vapigen name in the vapigen pkgconfig file, so that it picks
         # up our wrapper.
         sed -i \
-           -e "s|vapigen=.*|vapigen=${bindir}/vapigen-wrapper|" \
+           -e "s|vapigen=.*|vapigen=${bindir_crossscripts}/vapigen-wrapper|" \
            ${SYSROOT_DESTDIR}${libdir}/pkgconfig/vapigen-${SHRT_VER}.pc
 }
 
@@ -64,5 +64,5 @@ SSTATE_SCAN_FILES += "vapigen-wrapper"
 PACKAGE_PREPROCESS_FUNCS += "vala_package_preprocess"
 
 vala_package_preprocess () {
-	sed -i -e 's:${RECIPE_SYSROOT}::g;' ${PKGD}${bindir}/vapigen-wrapper
+	sed -i -e 's:${RECIPE_SYSROOT}::g;' ${PKGD}${bindir_crossscripts}/vapigen-wrapper
 }
-- 
2.30.2

[PATCH v2 5/5] sanity.bbclass: do not check for presence of distutils

[Alexander Kanavin]

This will be removed in 3.12 and has been deprecated for a while.

If anything breaks because of its absence on the host, this would
expose the breakage so it can be fixed.

icu source specifically does not refer to distutils anywhere.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/classes-global/sanity.bbclass | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/meta/classes-global/sanity.bbclass b/meta/classes-global/sanity.bbclass
index 606444cae1..38f4a4b22f 100644
--- a/meta/classes-global/sanity.bbclass
+++ b/meta/classes-global/sanity.bbclass
@@ -625,11 +625,9 @@ def check_sanity_version_change(status, d):
     # never again until the sanity version or host distrubution id/version changes.
 
     # Check the python install is complete. Examples that are often removed in
-    # minimal installations: glib-2.0-natives requries # xml.parsers.expat and icu
-    # requires distutils.sysconfig.
+    # minimal installations: glib-2.0-natives requries # xml.parsers.expat
     try:
         import xml.parsers.expat
-        import distutils.sysconfig
     except ImportError as e:
         status.addresult('Your Python 3 is not a full install. Please install the module %s (see the Getting Started guide for further information).\n' % e.name)
 
-- 
2.30.2

Re: [OE-core] [PATCH v2 5/5] sanity.bbclass: do not check for presence of distutils

[Martin Jansa]

On Wed, Nov 9, 2022 at 9:21 PM Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>
> This will be removed in 3.12 and has been deprecated for a while.
>
> If anything breaks because of its absence on the host, this would
> expose the breakage so it can be fixed.

I've sent a fix for glib-2.0-native yesterday:
https://lists.openembedded.org/g/openembedded-core/message/196136
which is now queued in master-next

> icu source specifically does not refer to distutils anywhere.

yes, that was removed in 65-1 release with
https://github.com/unicode-org/icu/commit/b4d41b0561b6e8de38b99850ce0e4be8ef536bb1

FWIW: in gentoo with python-3.12.1 I can still use distutils (provided
by setuptools):
$ python3.12
Python 3.12.2 (main, Feb 17 2024, 22:28:14) [GCC 13.2.1 20240210] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils
>>>

so I've noticed this issue only after ubuntu-24.04 switched the
default python3 from 3.11 to 3.12 and my Dockerfile was installing
only python3-distutils, not python3-setuptools:

# python3-distutils is needed by few components:
# icu-native, ovmf-native:
# x86_64-linux/icu-native/64.2-r0/temp/log.do_configure.28031:ModuleNotFoundError:
No module named 'distutils.sysconfig'
# x86_64-linux/ovmf-native/edk2-stable201905-r0/temp/log.do_compile.67451:Python
reported: "No module named 'distutils.util"
# in Ubuntu the python3 package doesn't pull python3-distutils:
# python3 -m distutils.sysconfig
# /usr/bin/python3: No module named distutils.sysconfig

# This issue with icu-native with Yocto 3.0 Zeus was discussed here:
# https://www.yoctoproject.org/pipermail/yocto/2019-October/047169.html
# https://www.mail-archive.com/yocto@yoctoproject.org/msg46284.html
# icu-native/65.1 and newer (in Yocto 3.1 Dunfell) doesn't need
python3-distutils
# anymore, it was removed here:
# https://github.com/unicode-org/icu/commit/b4d41b0561b6e8de38b99850ce0e4be8ef536bb1

# Re-reported upstream in:
# http://lists.openembedded.org/pipermail/openembedded-core/2020-March/293984.html

RUN apt-get install -y python3-distutils

Not sure what to do with python-3.12 in kirkstone, icu is fine there
(70.1), ovmf-native should be fine as well (distutils removed in
edk2-stable202108-rc0:
https://github.com/tianocore/edk2/commit/fc50df0d8eb5331b6641daeedd7f05ae75014ece
https://github.com/tianocore/edk2/commit/0b1b0a9674e27c858f05436ed92250f4498245cf
kirkstone has edk2-stable202202)

So we might be able to drop it as well with only that glib-2.0 fix
backported to kirkstone.

+Steve if he agrees with this plan for kirkstone or would prefer not
to support python 3.12 there.

Cheers,

>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  meta/classes-global/sanity.bbclass | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/meta/classes-global/sanity.bbclass b/meta/classes-global/sanity.bbclass
> index 606444cae1..38f4a4b22f 100644
> --- a/meta/classes-global/sanity.bbclass
> +++ b/meta/classes-global/sanity.bbclass
> @@ -625,11 +625,9 @@ def check_sanity_version_change(status, d):
>      # never again until the sanity version or host distrubution id/version changes.
>
>      # Check the python install is complete. Examples that are often removed in
> -    # minimal installations: glib-2.0-natives requries # xml.parsers.expat and icu
> -    # requires distutils.sysconfig.
> +    # minimal installations: glib-2.0-natives requries # xml.parsers.expat
>      try:
>          import xml.parsers.expat
> -        import distutils.sysconfig
>      except ImportError as e:
>          status.addresult('Your Python 3 is not a full install. Please install the module %s (see the Getting Started guide for further information).\n' % e.name)
>
> --
> 2.30.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#173054): https://lists.openembedded.org/g/openembedded-core/message/173054
> Mute This Topic: https://lists.openembedded.org/mt/94921928/3617156
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Martin.Jansa@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>