* [PATCH v2 2/5] python3: use the standard shell version of python3-config
2022-11-09 20:21 [PATCH v2 1/5] python3: correctly adjust include paths in sysconfigdata Alexander Kanavin
@ 2022-11-09 20:21 ` Alexander Kanavin
2022-11-09 20:21 ` [PATCH v2 3/5] python3targetconfig.bbclass: use PYTHONPATH to point to the target config Alexander Kanavin
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Alexander Kanavin @ 2022-11-09 20:21 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
There is really no reason why we can't: it only needs to be available
in two versions (native/target), and the correct one can be picked
via PATH priority order.
This eliminates two un-upstreamable patches, one of which relies
on soon to be removed distutils.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
.../python3targetconfig.bbclass | 17 ++++--
...shebang-overflow-on-python-config.py.patch | 14 ++---
...hell-version-of-python-config-that-w.patch | 36 ------------
.../python/python3/python-config.patch | 55 -------------------
.../recipes-devtools/python/python3_3.11.0.bb | 18 +++++-
5 files changed, 33 insertions(+), 107 deletions(-)
delete mode 100644 meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
delete mode 100644 meta/recipes-devtools/python/python3/python-config.patch
diff --git a/meta/classes-recipe/python3targetconfig.bbclass b/meta/classes-recipe/python3targetconfig.bbclass
index 3f89e5e09e..02fdb0c03f 100644
--- a/meta/classes-recipe/python3targetconfig.bbclass
+++ b/meta/classes-recipe/python3targetconfig.bbclass
@@ -10,26 +10,31 @@ EXTRA_PYTHON_DEPENDS ?= ""
EXTRA_PYTHON_DEPENDS:class-target = "python3"
DEPENDS:append = " ${EXTRA_PYTHON_DEPENDS}"
-do_configure:prepend:class-target() {
+setup_target_config() {
export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ export PATH=${STAGING_EXECPREFIXDIR}/python-target-config/:$PATH
+}
+
+do_configure:prepend:class-target() {
+ setup_target_config
}
do_compile:prepend:class-target() {
- export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ setup_target_config
}
do_install:prepend:class-target() {
- export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ setup_target_config
}
do_configure:prepend:class-nativesdk() {
- export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ setup_target_config
}
do_compile:prepend:class-nativesdk() {
- export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ setup_target_config
}
do_install:prepend:class-nativesdk() {
- export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ setup_target_config
}
diff --git a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
index 921da8de7c..db08435004 100644
--- a/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
+++ b/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
@@ -1,7 +1,7 @@
-From f0c9dec63d452a7cd1e15ea653f4aced281f021c Mon Sep 17 00:00:00 2001
+From 78f482b91d94b44a02e02c4580166757119061ea Mon Sep 17 00:00:00 2001
From: Paulo Neves <ptsneves@gmail.com>
Date: Tue, 7 Jun 2022 16:16:41 +0200
-Subject: [PATCH 1/1] Avoid shebang overflow on python-config.py
+Subject: [PATCH] Avoid shebang overflow on python-config.py
The whole native path may be too big, leading to shebang
overflow. Let's just use the env shebang.
@@ -11,15 +11,16 @@ Denial reason: [1]
Upstream-Status: Denied [distribution]
[1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
+
---
Makefile.pre.in | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Makefile.pre.in b/Makefile.pre.in
-index f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb31113176 100644
+index 55c7c46..1f6500a 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
-@@ -1638,6 +1638,8 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
+@@ -2115,6 +2115,8 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
@ # Substitution happens here, as the completely-expanded BINDIR
@ # is not available in configure
sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
@@ -27,7 +28,4 @@ index f0aedb76cb58999427804255da56fa53284d7032..dd88e43114730f7681715777cc76dabb
+ sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
@ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
- @ # In OpenEmbedded, always use the python version of the script, the shell
---
-2.25.1
-
+ @ # On Darwin, always use the python version of the script, the shell
diff --git a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch b/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
deleted file mode 100644
index d98f243cb1..0000000000
--- a/meta/recipes-devtools/python/python3/0001-Do-not-use-the-shell-version-of-python-config-that-w.patch
+++ /dev/null
@@ -1,36 +0,0 @@
-From 2406432449784243b7590009d42bd0e871253b2e Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Tue, 29 Jan 2019 15:03:01 +0100
-Subject: [PATCH] Do not use the shell version of python-config that was
- introduced in 3.4
-
-Revert instead to the original python version: it has our tweaks and
-outputs directories correctly.
-
-Upstream-Status: Inappropriate [oe-specific]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
-
----
- Makefile.pre.in | 9 +++------
- 1 file changed, 3 insertions(+), 6 deletions(-)
-
-diff --git a/Makefile.pre.in b/Makefile.pre.in
-index ee85f35..f0aedb7 100644
---- a/Makefile.pre.in
-+++ b/Makefile.pre.in
-@@ -1640,12 +1640,9 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
- sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
- @ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
- LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
-- @ # On Darwin, always use the python version of the script, the shell
-- @ # version doesn't use the compiler customizations that are provided
-- @ # in python (_osx_support.py).
-- @if test `uname -s` = Darwin; then \
-- cp python-config.py python-config; \
-- fi
-+ @ # In OpenEmbedded, always use the python version of the script, the shell
-+ @ # version is broken in multiple ways, and doesn't return correct directories
-+ cp python-config.py python-config
-
-
- # Install the include files
diff --git a/meta/recipes-devtools/python/python3/python-config.patch b/meta/recipes-devtools/python/python3/python-config.patch
deleted file mode 100644
index 4da399e46e..0000000000
--- a/meta/recipes-devtools/python/python3/python-config.patch
+++ /dev/null
@@ -1,55 +0,0 @@
-From 8632f25ac4e2c53a3c2c8a1b4fc97fc86e8aad5a Mon Sep 17 00:00:00 2001
-From: Tyler Hall <tylerwhall@gmail.com>
-Date: Sun, 4 May 2014 20:06:43 -0400
-Subject: [PATCH] python-config: Revert to using distutils.sysconfig
-
-The newer sysconfig module shares some code with distutils.sysconfig, but the same modifications as in
-
-12-distutils-prefix-is-inside-staging-area.patch makes distutils.sysconfig
-
-affect the native runtime as well as cross building. Use the old, patched
-implementation which returns paths in the staging directory and for the target,
-as appropriate.
-
-Upstream-Status: Inappropriate [Embedded Specific]
-
-Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
-
----
- Misc/python-config.in | 12 +++++++-----
- 1 file changed, 7 insertions(+), 5 deletions(-)
-
-diff --git a/Misc/python-config.in b/Misc/python-config.in
-index ebd99da..0492e08 100644
---- a/Misc/python-config.in
-+++ b/Misc/python-config.in
-@@ -6,7 +6,9 @@
- import getopt
- import os
- import sys
--import sysconfig
-+import warnings
-+warnings.filterwarnings("ignore", category=DeprecationWarning)
-+from distutils import sysconfig
-
- valid_opts = ['prefix', 'exec-prefix', 'includes', 'libs', 'cflags',
- 'ldflags', 'extension-suffix', 'help', 'abiflags', 'configdir',
-@@ -35,14 +37,14 @@ if '--help' in opt_flags:
-
- for opt in opt_flags:
- if opt == '--prefix':
-- print(getvar('prefix'))
-+ print(sysconfig.PREFIX)
-
- elif opt == '--exec-prefix':
-- print(getvar('exec_prefix'))
-+ print(sysconfig.EXEC_PREFIX)
-
- elif opt in ('--includes', '--cflags'):
-- flags = ['-I' + sysconfig.get_path('include'),
-- '-I' + sysconfig.get_path('platinclude')]
-+ flags = ['-I' + sysconfig.get_python_inc(),
-+ '-I' + sysconfig.get_python_inc(plat_specific=True)]
- if opt == '--cflags':
- flags.extend(getvar('CFLAGS').split())
- print(' '.join(flags))
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 92a1f69320..4760895e9f 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -16,8 +16,6 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
file://cgi_py.patch \
file://0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch \
${@bb.utils.contains('PACKAGECONFIG', 'tk', '', 'file://avoid_warning_about_tkinter.patch', d)} \
- file://0001-Do-not-use-the-shell-version-of-python-config-that-w.patch \
- file://python-config.patch \
file://0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch \
file://0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch \
file://0001-bpo-36852-proper-detection-of-mips-architecture-for-.patch \
@@ -172,6 +170,9 @@ do_install:append:class-native() {
# disable the lookup in user's site-packages globally
sed -i 's#ENABLE_USER_SITE = None#ENABLE_USER_SITE = False#' ${D}${libdir}/python${PYTHON_MAJMIN}/site.py
+
+ # python3-config needs to be in /usr/bin and not in a subdir of it to work properly
+ mv ${D}/${bindir}/${PN}/python*config ${D}/${bindir}/
}
do_install:append() {
@@ -220,6 +221,19 @@ do_install:append:class-nativesdk () {
create_wrapper ${D}${bindir}/python${PYTHON_MAJMIN} TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' PYTHONNOUSERSITE='1'
}
+SYSROOT_PREPROCESS_FUNCS:append:class-target = " provide_target_config_script"
+SYSROOT_PREPROCESS_FUNCS:append:class-nativesdk = " provide_target_config_script"
+
+# This is installed into /usr/python-target-config/ and not /usr/bin
+# because adding target sysroot's /usr/bin/ to PATH has unwanted side effects
+# in components erroneously picking up other target executables from it
+provide_target_config_script() {
+ install -d ${SYSROOT_DESTDIR}${prefix}/python-target-config/
+ install ${D}/${bindir}/python3-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
+ install ${D}/${bindir}/python${PYTHON_MAJMIN}-config ${SYSROOT_DESTDIR}/${prefix}/python-target-config/
+}
+SYSROOT_DIRS += "${prefix}/python-target-config/"
+
SSTATE_SCAN_FILES += "Makefile _sysconfigdata.py"
SSTATE_HASHEQUIV_FILEMAP = " \
populate_sysroot:*/lib*/python3*/_sysconfigdata*.py:${TMPDIR} \
--
2.30.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH v2 3/5] python3targetconfig.bbclass: use PYTHONPATH to point to the target config
2022-11-09 20:21 [PATCH v2 1/5] python3: correctly adjust include paths in sysconfigdata Alexander Kanavin
2022-11-09 20:21 ` [PATCH v2 2/5] python3: use the standard shell version of python3-config Alexander Kanavin
@ 2022-11-09 20:21 ` Alexander Kanavin
2022-11-09 20:21 ` [PATCH v2 4/5] vala: install vapigen-wrapper into /usr/bin/crosscripts and stage only that Alexander Kanavin
2022-11-09 20:21 ` [PATCH v2 5/5] sanity.bbclass: do not check for presence of distutils Alexander Kanavin
3 siblings, 0 replies; 6+ messages in thread
From: Alexander Kanavin @ 2022-11-09 20:21 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
There is no need to patch native python so that it looks in the target
sysroot; the same can be achieved with just an environment variable.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
.../python3targetconfig.bbclass | 1 +
.../classes-recipe/setuptools3_legacy.bbclass | 2 +-
...fig-append-STAGING_LIBDIR-python-sys.patch | 28 -------------------
.../recipes-devtools/python/python3_3.11.0.bb | 1 -
4 files changed, 2 insertions(+), 30 deletions(-)
delete mode 100644 meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
diff --git a/meta/classes-recipe/python3targetconfig.bbclass b/meta/classes-recipe/python3targetconfig.bbclass
index 02fdb0c03f..22305fe176 100644
--- a/meta/classes-recipe/python3targetconfig.bbclass
+++ b/meta/classes-recipe/python3targetconfig.bbclass
@@ -12,6 +12,7 @@ DEPENDS:append = " ${EXTRA_PYTHON_DEPENDS}"
setup_target_config() {
export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
+ export PYTHONPATH=${STAGING_LIBDIR}/python-sysconfigdata
export PATH=${STAGING_EXECPREFIXDIR}/python-target-config/:$PATH
}
diff --git a/meta/classes-recipe/setuptools3_legacy.bbclass b/meta/classes-recipe/setuptools3_legacy.bbclass
index 21748f922a..57de956d0a 100644
--- a/meta/classes-recipe/setuptools3_legacy.bbclass
+++ b/meta/classes-recipe/setuptools3_legacy.bbclass
@@ -49,7 +49,7 @@ setuptools3_legacy_do_install() {
install -d ${D}${PYTHON_SITEPACKAGES_DIR}
STAGING_INCDIR=${STAGING_INCDIR} \
STAGING_LIBDIR=${STAGING_LIBDIR} \
- PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
+ PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR}:$PYTHONPATH \
${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
build --build-base=${B} install --skip-build ${SETUPTOOLS_INSTALL_ARGS} || \
bbfatal_log "'${PYTHON_PN} setup.py install ${SETUPTOOLS_INSTALL_ARGS}' execution failed."
diff --git a/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch b/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
deleted file mode 100644
index 45a37ed1a9..0000000000
--- a/meta/recipes-devtools/python/python3/0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch
+++ /dev/null
@@ -1,28 +0,0 @@
-From 9c6b9f46179c8f9c9391767e2b02f268a1ee7a9c Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Thu, 31 Jan 2019 16:46:30 +0100
-Subject: [PATCH] distutils/sysconfig: append
- STAGING_LIBDIR/python-sysconfigdata to sys.path
-
-So that target configuration can be used when running native python
-
-Upstream-Status: Inappropriate [oe-core specific]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
-
----
- Lib/sysconfig.py | 2 ++
- 1 file changed, 2 insertions(+)
-
-diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
-index ff399e2..95844cf 100644
---- a/Lib/sysconfig.py
-+++ b/Lib/sysconfig.py
-@@ -528,6 +528,8 @@ def _init_posix(vars):
- """Initialize the module as appropriate for POSIX systems."""
- # _sysconfigdata is generated at build time, see _generate_posix_vars()
- name = _get_sysconfigdata_name()
-+ if 'STAGING_LIBDIR' in os.environ:
-+ sys.path.append(os.environ['STAGING_LIBDIR']+'/python-sysconfigdata')
- _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
- build_time_vars = _temp.build_time_vars
- vars.update(build_time_vars)
diff --git a/meta/recipes-devtools/python/python3_3.11.0.bb b/meta/recipes-devtools/python/python3_3.11.0.bb
index 4760895e9f..94f01af83a 100644
--- a/meta/recipes-devtools/python/python3_3.11.0.bb
+++ b/meta/recipes-devtools/python/python3_3.11.0.bb
@@ -36,7 +36,6 @@ SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
SRC_URI:append:class-native = " \
file://0001-Lib-sysconfig.py-use-prefix-value-from-build-configu.patch \
- file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
file://12-distutils-prefix-is-inside-staging-area.patch \
file://0001-Don-t-search-system-for-headers-libraries.patch \
"
--
2.30.2
^ permalink raw reply related [flat|nested] 6+ messages in thread