All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/19] Python PEP-517: build wheels and installw with pip
@ 2022-02-21  1:27 Tim Orling
  2022-02-21  1:27 ` [PATCH 01/19] python3-wheel: move 0.37.0 from meta-python Tim Orling
                   ` (19 more replies)
  0 siblings, 20 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream Python is moving to pyproject.toml rather than setup.py and
wheels rather than eggs. The main place this is documented is in
PEP-517[1]. Installing packages with setup.py install is deprecated.

We have a choice with the 'kirkstone' LTS release of introducing this
series (and fixing any fall out during the stabilization period) or
living with deprecated Python packaging for the duration of the LTS (at
a minimum 2 years). This seems to be a compelling enough reason to bring
in this series, even if it results in a slight slip of the 'feature
freeze', which is officially tomorrow, Monday 21 Feb, 2022.

This series bootstraps python3-setuptools-native,
python3-flit-core-native, python3-wheel-native and python3-pip-native,
simply unzipping the wheel into PYTHON_SITEPACKAGES_DIR. This allows all
other recipes to be installed with pip, as intended by upstream.

Three new classes are introduced:
(1) flit_core which builds PEP-517 compliant packages which use
    flit_core.buildapi as their build-backend.
(2) setuptools_build_meta which builds PEP-517 compliant packages which use
    setuptools_build_meta  as their build-backend.
(3) pip_install_wheel, which simply installs wheels built by other
    classes.

The original setuptools3.bbclass do_compile is modified to
'setup.py bdist_wheel' (from python3-wheel-native) rather than the deprecated
'setup.py build' mechanism. 

The original setuptools3.bbclass do_install is modified to
use the pip_install_wheel.bbclass to install wheels with pip rather than
the deprecated 'setup.py install' mechanism. 

For pip install, we disable dependency checking (--no-deps) because it
proves to be brittle in our environment without providing full wheel
cache that seems like overhead that we do not need. We also disable
fetching wheels from pypi (--no-index). This is no different than our
normal workflow of requiring dependencies to be defined in a recipe.

Missing from this series is documentation of the new features, classes
and variables (this will be done under separate cover).

Also desired and previously discussed in YP meetings is a helper
function to warn that a recipe which currently inherits setuptools3
but has a pyproject.toml and defines a build-system.build-backend should
instead use of the new PEP-517 classes. This function is still in progress.

During testing, recipes which use python3-setuptools-scm are failing to
determine the version of the python package, For several recipes, the fix
was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes
are still going to fail for the short term, until the root cause can be
determined.

Another PEP-517 build-backend, poetry.core.masonry.api has been
identified in recipes in meta-python, but not in oe-core. The
python3-poetry-core recipe is WIP, but brings in a number of
dependencies that are perhaps not worth it to bring into oe-core at this
time. Rather we can support these recipes in meta-python?

[YOCTO #14638]

[1] https://www.python.org/dev/peps/pep-0517/

The following changes since commit 0539b563c6f41fef383429a13c28bf918231d404:

  libhandy: Use upstream regex to check version of upgrade. (2022-02-20 16:45:25 +0000)

are available in the Git repository at:

  git://git.yoctoproject.org/git/poky-contrib timo/bootstrap-wheels
  http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels

Tim Orling (19):
  python3-wheel: move 0.37.0 from meta-python
  python3-flit-core: add recipe for 3.6.0
  python3-flit-core: simplify
  flit_core.bbclass: add helper for newer python packaging
  python3-wheel: inherit flit_core
  pip_install_wheel.bbclass: add helper class
  setuptools_build_meta.bbclass: add helper class
  python3-pip: inherit setuptools_build_meta
  python3-attrs: inherit setuptools_build_meta
  python3-git: inherit setuptools_build_meta
  python3-pytest: inherit setuptools_build_meta
  python3-setuptools-scm: inherit setuptools_build_meta
  python3-zipp: inherit setuptools_build_meta
  python3-iniconfig: inherit setuptools_build_meta
  python3-py: inherit setuptools_build_meta
  python3-pluggy: inherit setuptools_build_meta
  python3-setuptools: inherit setuptools_base_meta
  setuptools3.bbclass: refactor for wheels
  python3-more-itertools: set PIP_INSTALL_PACKAGE

 meta/classes/flit_core.bbclass                |  16 +++
 meta/classes/pip_install_wheel.bbclass        |  18 ++++
 meta/classes/setuptools3.bbclass              |  20 ++--
 meta/classes/setuptools_build_meta.bbclass    |  18 ++++
 .../python/python3-attrs_21.4.0.bb            |   2 +-
 .../python/python3-flit-core_3.6.0.bb         |  41 +++++++
 .../python/python3-git_3.1.26.bb              |   2 +-
 .../python/python3-iniconfig_1.1.1.bb         |   4 +-
 .../python/python3-more-itertools_8.12.0.bb   |   2 +
 .../python/python3-pip_22.0.3.bb              |  19 +++-
 .../python/python3-pluggy_1.0.0.bb            |   2 +-
 .../python/python3-py_1.11.0.bb               |   2 +-
 .../python/python3-pytest_7.0.1.bb            |   4 +-
 .../python/python3-setuptools-scm_6.4.2.bb    |   2 +-
 .../python/python3-setuptools_59.5.0.bb       |  19 +++-
 ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++
 .../python/python3-wheel_0.37.1.bb            |  24 +++++
 .../python/python3-zipp_3.7.0.bb              |   2 +-
 18 files changed, 269 insertions(+), 28 deletions(-)
 create mode 100644 meta/classes/flit_core.bbclass
 create mode 100644 meta/classes/pip_install_wheel.bbclass
 create mode 100644 meta/classes/setuptools_build_meta.bbclass
 create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
 create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb

-- 
2.30.2



^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH 01/19] python3-wheel: move 0.37.0 from meta-python
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 02/19] python3-flit-core: add recipe for 3.6.0 Tim Orling
                   ` (18 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

This is one of the "new build tools" which are part of pypa (Python
Packaging Authority) toolchain.

Wheels are the official delivery mechanism for Python packages, replacing
the now deprecated Eggs (egg-info).

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb

diff --git a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
new file mode 100644
index 00000000000..6c7a31db9e7
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
@@ -0,0 +1,12 @@
+SUMMARY = "The official binary distribution format for Python "
+HOMEPAGE = "https://github.com/pypa/wheel"
+SECTION = "devel/python"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a0148f94f483f3e1489"
+
+SRC_URI[sha256sum] = "e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4"
+
+inherit pypi setuptools3
+
+BBCLASSEXTEND = "native"
+
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 02/19] python3-flit-core: add recipe for 3.6.0
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
  2022-02-21  1:27 ` [PATCH 01/19] python3-wheel: move 0.37.0 from meta-python Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 03/19] python3-flit-core: simplify Tim Orling
                   ` (17 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

This is the core of one of the "new build system" tools, although not
officially part of the pypa (Python Packaging Authority) repositories,
it is an increasingly common build tool (e.g. typing_extensions and
tomli) as declared in pyproject.toml for said packages.

This package provides a very simple bootstrapping method that builds
the source tarball (build_sdist) and a wheel (build_wheel).

Wheels are the official vehicle for delivering Python packages now.
Eggs (egg-info) are deprecated and will increasingly go away.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../python/python3-flit-core_3.6.0.bb         | 35 +++++++++++++++++++
 1 file changed, 35 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb

diff --git a/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb b/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
new file mode 100644
index 00000000000..4221076bc73
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
@@ -0,0 +1,35 @@
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=41eb78fa8a872983a882c694a8305f08"
+
+SRC_URI[sha256sum] = "b1464e006df4df4c8eeb37671c0e0ce66e1d04e4a36d91b702f180a25fde3c11"
+
+inherit python3native python3-dir pypi
+
+DEPENDS += "python3 python3-pip-native"
+
+# We need the full flit tarball
+PYPI_PACKAGE = "flit"
+
+do_compile () {
+    nativepython3 flit_core/build_dists.py
+}
+
+do_install () {
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+    PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
+    nativepython3 -m pip install -vvvv --no-index --target ${D}${PYTHON_SITEPACKAGES_DIR} ./flit_core/dist/flit_core-${PV}-py3-none-any.whl
+}
+
+FILES:${PN} += "\
+    ${PYTHON_SITEPACKAGES_DIR}/flit_core/* \
+    ${PYTHON_SITEPACKAGES_DIR}/flit_core-${PV}.dist-info/* \
+"
+
+PACKAGES =+ "${PN}-tests"
+
+FILES:${PN}-tests += "\
+    ${PYTHON_SITEPACKAGES_DIR}/flit_core/tests/* \
+"
+
+BBCLASSEXTEND = "native nativesdk"
+
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 03/19] python3-flit-core: simplify
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
  2022-02-21  1:27 ` [PATCH 01/19] python3-wheel: move 0.37.0 from meta-python Tim Orling
  2022-02-21  1:27 ` [PATCH 02/19] python3-flit-core: add recipe for 3.6.0 Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 04/19] flit_core.bbclass: add helper for newer python packaging Tim Orling
                   ` (16 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a build_dists.py which can be used to bootstrap
building of the sdist and wheel.

Bootstrap -native by simply unzipping the wheel to
PYTHON_SITEPACKAGES_DIR.

Use pip to install the wheel for target.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../python/python3-flit-core_3.6.0.bb                | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb b/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
index 4221076bc73..d0b88c3fa50 100644
--- a/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
+++ b/meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
@@ -3,9 +3,10 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=41eb78fa8a872983a882c694a8305f08"
 
 SRC_URI[sha256sum] = "b1464e006df4df4c8eeb37671c0e0ce66e1d04e4a36d91b702f180a25fde3c11"
 
-inherit python3native python3-dir pypi
+inherit python3native python3-dir pypi setuptools3-base
 
-DEPENDS += "python3 python3-pip-native"
+DEPENDS:append:class-target = " python3-pip-native"
+DEPENDS:append:class-native = " unzip-native"
 
 # We need the full flit tarball
 PYPI_PACKAGE = "flit"
@@ -17,7 +18,12 @@ do_compile () {
 do_install () {
     install -d ${D}${PYTHON_SITEPACKAGES_DIR}
     PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
-    nativepython3 -m pip install -vvvv --no-index --target ${D}${PYTHON_SITEPACKAGES_DIR} ./flit_core/dist/flit_core-${PV}-py3-none-any.whl
+    nativepython3 -m pip install -vvvv --no-deps --no-index --target ${D}${PYTHON_SITEPACKAGES_DIR} ./flit_core/dist/flit_core-${PV}-py3-none-any.whl
+}
+
+do_install:class-native () {
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+    unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ./flit_core/dist/flit_core-${PV}-py3-none-any.whl
 }
 
 FILES:${PN} += "\
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 04/19] flit_core.bbclass: add helper for newer python packaging
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (2 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 03/19] python3-flit-core: simplify Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 05/19] python3-wheel: inherit flit_core Tim Orling
                   ` (15 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Some python packages now use pyproject.toml and declare
flit_core.buildapi as the build engine

Use pip_install_wheel class to install.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/classes/flit_core.bbclass | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 meta/classes/flit_core.bbclass

diff --git a/meta/classes/flit_core.bbclass b/meta/classes/flit_core.bbclass
new file mode 100644
index 00000000000..0f2eec85d0b
--- /dev/null
+++ b/meta/classes/flit_core.bbclass
@@ -0,0 +1,16 @@
+inherit pip_install_wheel python3native python3-dir
+
+DEPENDS += "python3 python3-flit-core-native python3-pip-native"
+
+do_configure () {
+    mkdir -p ${S}/dist
+    cat > ${S}/build-it.py << EOF
+from flit_core import buildapi
+buildapi.build_wheel('./dist')
+EOF
+}
+
+do_compile () {
+    ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} ${S}/build-it.py
+}
+
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 05/19] python3-wheel: inherit flit_core
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (3 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 04/19] flit_core.bbclass: add helper for newer python packaging Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 06/19] pip_install_wheel.bbclass: add helper class Tim Orling
                   ` (14 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Backport pyproject.toml from flit-backend branch.
Inherit flit_core class to build
Inherit pip_install_wheel to install wheels for target

We need to bootstrap python3-wheel-native in order to have bdist_wheel
available to python3-setuptools-native and the refactored
setuptools3.bbclass. Simply unzip the wheel into
PYTHON_SITEPACKAGES_DIR for class-native.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++
 .../python/python3-wheel_0.37.1.bb            |  16 ++-
 2 files changed, 114 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch

diff --git a/meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch b/meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
new file mode 100644
index 00000000000..023de0e6a88
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
@@ -0,0 +1,100 @@
+From f00dd220346773bc088d403847ee7f06f2b4c30a Mon Sep 17 00:00:00 2001
+From: Tim Orling <tim.orling@konsulko.com>
+Date: Fri, 18 Feb 2022 11:09:16 -0800
+Subject: [PATCH] Backport pyproject.toml from flit-backend branch
+
+This allows us to bootstrap wheels and PEP-517 packaging.
+
+Upstream-Status: Backport from flit-backend branch
+https://raw.githubusercontent.com/pypa/wheel/4f6ba78fede38a8d9e35a14e38377a121033afb3/pyproject.toml
+
+Signed-off-by: Tim Orling <tim.orling@konsulko.com>
+---
+ pyproject.toml | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++
+ 1 file changed, 78 insertions(+)
+ create mode 100644 pyproject.toml
+
+diff --git a/pyproject.toml b/pyproject.toml
+new file mode 100644
+index 0000000..749b8de
+--- /dev/null
++++ b/pyproject.toml
+@@ -0,0 +1,78 @@
++[build-system]
++requires = ["flit_core >=3.2,<4"]
++build-backend = "flit_core.buildapi"
++
++[project]
++name = "wheel"
++description = "A built-package format for Python"
++readme = "README.rst"
++classifiers = [
++    "Development Status :: 5 - Production/Stable",
++    "Intended Audience :: Developers",
++    "Topic :: System :: Archiving :: Packaging",
++    "License :: OSI Approved :: MIT License",
++    "Programming Language :: Python",
++    "Programming Language :: Python :: 3 :: Only",
++    "Programming Language :: Python :: 3.7",
++    "Programming Language :: Python :: 3.8",
++    "Programming Language :: Python :: 3.9",
++    "Programming Language :: Python :: 3.10"
++]
++authors = [{name = "Daniel Holth", email = "dholth@fastmail.fm"}]
++maintainers = [{name = "Alex Grönholm", email = "alex.gronholm@nextday.fi"}]
++keywords = ["wheel", "packaging"]
++license = {file = "LICENSE.txt"}
++requires-python = ">=3.7"
++dependencies = [
++    "setuptools >= 45.2.0"
++]
++dynamic = ["version"]
++
++[project.urls]
++Documentation = "https://wheel.readthedocs.io/"
++Changelog = "https://wheel.readthedocs.io/en/stable/news.html"
++"Issue Tracker" = "https://github.com/pypa/wheel/issues"
++
++[project.scripts]
++wheel = "wheel.cli:main"
++
++[project.entry-points."distutils.commands"]
++bdist_wheel = "wheel.bdist_wheel:bdist_wheel"
++
++[project.optional-dependencies]
++test = [
++    "pytest >= 3.0.0"
++]
++
++[tool.flit.sdist]
++exclude = [
++    ".cirrus.yml",
++    ".github/*",
++    ".gitignore",
++    ".pre-commit-config.yaml",
++    ".readthedocs.yml"
++]
++
++[tool.black]
++target-version = ['py37']
++extend-exclude = '''
++^/src/wheel/vendored/
++'''
++
++[tool.isort]
++src_paths = ["src"]
++profile = "black"
++skip_gitignore = true
++
++[tool.flake8]
++max-line-length = 88
++
++[tool.pytest.ini_options]
++testpaths = "tests"
++
++[tool.coverage.run]
++source = ["wheel"]
++omit = ["*/vendored/*"]
++
++[tool.coverage.report]
++show_missing = true
diff --git a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
index 6c7a31db9e7..c7354c9d982 100644
--- a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
+++ b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb
@@ -6,7 +6,19 @@ LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=10;endline=10;md5=8227180126797a01
 
 SRC_URI[sha256sum] = "e9a504e793efbca1b8e0e9cb979a249cf4a0a7b5b8c9e8b65a5e39d49529c1c4"
 
-inherit pypi setuptools3
+inherit flit_core pypi setuptools3-base
 
-BBCLASSEXTEND = "native"
+SRC_URI += " file://0001-Backport-pyproject.toml-from-flit-backend-branch.patch"
+
+DEPENDS:remove:class-native = "python3-pip-native"
+
+do_install:class-native () {
+    # We need to bootstrap python3-wheel-native
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+    PYPA_WHEEL="${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl"
+    unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \
+    bbfatal_log "Failed to install"
+}
+
+BBCLASSEXTEND = "native nativesdk"
 
-- 
2.30.2


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 06/19] pip_install_wheel.bbclass: add helper class
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (4 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 05/19] python3-wheel: inherit flit_core Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 07/19] setuptools_build_meta.bbclass: " Tim Orling
                   ` (13 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Provide a helper class to use pip to install wheels built by either
bdist_wheel or a PEP-517 backend.

By passings --no-deps and --no-index we avoid finicky dependency
checking (pip expects wheels in its cache) and avoid trying to fetch
wheels from pypi.org. This is basically the same behavior we have now,
the dependencies should be declared in the recipe.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/classes/pip_install_wheel.bbclass | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 meta/classes/pip_install_wheel.bbclass

diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
new file mode 100644
index 00000000000..e6a6801227f
--- /dev/null
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -0,0 +1,18 @@
+DEPENDS:append = " python3-pip-native"
+
+PIP_INSTALL_PACKAGE ?= "${PYPI_PACKAGE}"
+PIP_INSTALL_DIST_PATH ?= "${B}/dist"
+PYPA_WHEEL ??= "${PIP_INSTALL_DIST_PATH}/${PIP_INSTALL_PACKAGE}-${PV}-*.whl"
+
+pip_install_wheel_do_install:prepend () {
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+}
+
+export PYPA_WHEEL
+
+pip_install_wheel_do_install () {
+    nativepython3 -m pip install -vvvv --no-deps --no-index --target ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} ||
+    bbfatal_log "Failed to pip install wheel. Check the logs."
+}
+
+EXPORT_FUNCTIONS do_install
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 07/19] setuptools_build_meta.bbclass: add helper class
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (5 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 06/19] pip_install_wheel.bbclass: add helper class Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 08/19] python3-pip: inherit setuptools_build_meta Tim Orling
                   ` (12 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

This class uses the PEP 517 compliant setuptools.buil_meta to build
wheels.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/classes/setuptools_build_meta.bbclass | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 meta/classes/setuptools_build_meta.bbclass

diff --git a/meta/classes/setuptools_build_meta.bbclass b/meta/classes/setuptools_build_meta.bbclass
new file mode 100644
index 00000000000..b1441e65dd5
--- /dev/null
+++ b/meta/classes/setuptools_build_meta.bbclass
@@ -0,0 +1,18 @@
+inherit pip_install_wheel setuptools3-base
+
+DEPENDS += "python3 python3-setuptools-native python3-wheel-native"
+
+setuptools_build_meta_do_configure () {
+    mkdir -p ${S}/dist
+    cat > ${S}/build-it.py << EOF
+from setuptools import build_meta
+wheel = build_meta.build_wheel('./dist')
+print(wheel)
+EOF
+}
+
+setuptools_build_meta_do_compile () {
+    nativepython3 ${S}/build-it.py
+}
+
+EXPORT_FUNCTIONS do_configure do_compile
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 08/19] python3-pip: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (6 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 07/19] setuptools_build_meta.bbclass: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 09/19] python3-attrs: " Tim Orling
                   ` (11 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares setuptools.build_meta
as the backend for PEP-517 packaging.

We bootstrap the -native installation by simply unzipping the wheel to
PYTHON_SITEPACKAGES_DIR, so that all other recipes can use pip to
install wheels (as intended by upstream).

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../python/python3-pip_22.0.3.bb              | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-pip_22.0.3.bb b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
index 889a6bf8adc..482a210ee99 100644
--- a/meta/recipes-devtools/python/python3-pip_22.0.3.bb
+++ b/meta/recipes-devtools/python/python3-pip_22.0.3.bb
@@ -4,17 +4,32 @@ SECTION = "devel/python"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=63ec52baf95163b597008bb46db68030"
 
+inherit pypi setuptools_build_meta
+
 DEPENDS += "python3 python3-setuptools-native"
 
-inherit pypi setuptools3
+# To avoid a dependency loop; we bootstrap -native
+DEPENDS:remove:class-native = "python3-pip-native"
+DEPENDS:append:class-native = " unzip-native"
 
 SRC_URI += "file://0001-change-shebang-to-python3.patch"
 
 SRC_URI[sha256sum] = "f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0"
 
+PYPA_WHEEL ?= "${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl"
+
+do_install:class-native() {
+    # Bootstrap to prevent dependency loop in python3-pip-native
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+    unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \
+    bbfatal_log "Failed to unzip wheel: ${PYPA_WHEEL}. Check the logs."
+}
+
 do_install:append() {
     # Install as pip3 and leave pip2 as default
-    rm ${D}/${bindir}/pip
+    if [ -f ${D}/${bindir}/pip ]; then
+        rm ${D}/${bindir}/pip
+    fi
 }
 
 RDEPENDS:${PN} = "\
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 09/19] python3-attrs: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (7 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 08/19] python3-pip: inherit setuptools_build_meta Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 10/19] python3-git: " Tim Orling
                   ` (10 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-attrs_21.4.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-attrs_21.4.0.bb b/meta/recipes-devtools/python/python3-attrs_21.4.0.bb
index d1dde29d376..20e4541f81c 100644
--- a/meta/recipes-devtools/python/python3-attrs_21.4.0.bb
+++ b/meta/recipes-devtools/python/python3-attrs_21.4.0.bb
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=d4ab25949a73fe7d4fdee93bcbdbf8ff"
 
 SRC_URI[sha256sum] = "626ba8234211db98e869df76230a137c4c40a12d72445c45d5f5b716f076e2fd"
 
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 RDEPENDS:${PN}:class-target += " \
     ${PYTHON_PN}-crypt \
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 10/19] python3-git: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (8 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 09/19] python3-attrs: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 11/19] python3-pytest: " Tim Orling
                   ` (9 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-git_3.1.26.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-git_3.1.26.bb b/meta/recipes-devtools/python/python3-git_3.1.26.bb
index b6171a0e4d1..8f4b13cc15d 100644
--- a/meta/recipes-devtools/python/python3-git_3.1.26.bb
+++ b/meta/recipes-devtools/python/python3-git_3.1.26.bb
@@ -10,7 +10,7 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=8b8d26c37c1d5a04f9b0186edbebc183"
 
 PYPI_PACKAGE = "GitPython"
 
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 SRC_URI[sha256sum] = "fc8868f63a2e6d268fb25f481995ba185a85a66fcad126f039323ff6635669ee"
 
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 11/19] python3-pytest: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (9 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 10/19] python3-git: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 12/19] python3-setuptools-scm: " Tim Orling
                   ` (8 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-pytest_7.0.1.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-pytest_7.0.1.bb b/meta/recipes-devtools/python/python3-pytest_7.0.1.bb
index 25014064890..ea6ba80300a 100644
--- a/meta/recipes-devtools/python/python3-pytest_7.0.1.bb
+++ b/meta/recipes-devtools/python/python3-pytest_7.0.1.bb
@@ -7,9 +7,9 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=bd27e41b6550fe0fc45356d1d81ee37c"
 
 SRC_URI[sha256sum] = "e30905a0c131d3d94b89624a1cc5afec3e0ba2fbdb151867d8e0ebd49850f171"
 
-DEPENDS = "python3-setuptools-scm-native"
+DEPENDS += "python3-setuptools-scm-native"
 
-inherit update-alternatives pypi setuptools3
+inherit update-alternatives pypi setuptools_build_meta
 
 RDEPENDS:${PN}:class-target += " \
     ${PYTHON_PN}-atomicwrites \
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 12/19] python3-setuptools-scm: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (10 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 11/19] python3-pytest: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 13/19] python3-zipp: " Tim Orling
                   ` (7 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb b/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb
index 10dad5a37be..182ccecbbfc 100644
--- a/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb
+++ b/meta/recipes-devtools/python/python3-setuptools-scm_6.4.2.bb
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=8227180126797a0148
 SRC_URI[sha256sum] = "6833ac65c6ed9711a4d5d2266f8024cfa07c533a0e55f4c12f6eff280a5a9e30"
 
 PYPI_PACKAGE = "setuptools_scm"
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 UPSTREAM_CHECK_REGEX = "setuptools_scm-(?P<pver>.*)\.tar"
 
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 13/19] python3-zipp: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (11 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 12/19] python3-setuptools-scm: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 14/19] python3-iniconfig: " Tim Orling
                   ` (6 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-zipp_3.7.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-zipp_3.7.0.bb b/meta/recipes-devtools/python/python3-zipp_3.7.0.bb
index 9ce987c8702..d0f936faf7c 100644
--- a/meta/recipes-devtools/python/python3-zipp_3.7.0.bb
+++ b/meta/recipes-devtools/python/python3-zipp_3.7.0.bb
@@ -7,7 +7,7 @@ SRC_URI[sha256sum] = "9f50f446828eb9d45b267433fd3e9da8d801f614129124863f9c51ebce
 
 DEPENDS += "${PYTHON_PN}-setuptools-scm-native"
 
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 DEPENDS += "${PYTHON_PN}-toml-native"
 
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 14/19] python3-iniconfig: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (12 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 13/19] python3-zipp: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 15/19] python3-py: " Tim Orling
                   ` (5 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which decalres the
setuptools.build_meta backend for PEP-517 packaging.

DEPENDS on python3-setuptools-scm-native (as declared in the
pyproject.toml build-system.requires).

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb b/meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb
index 6b3359712f5..5338980e699 100644
--- a/meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb
+++ b/meta/recipes-devtools/python/python3-iniconfig_1.1.1.bb
@@ -7,4 +7,6 @@ LIC_FILES_CHKSUM = "file://LICENSE;md5=a6bb0320b04a0a503f12f69fea479de9"
 SRC_URI[md5sum] = "0b7f3be87481211c183eae095bcea6f1"
 SRC_URI[sha256sum] = "bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32"
 
-inherit pypi setuptools3
+DEPENDS += "python3-setuptools-scm-native"
+
+inherit pypi setuptools_build_meta
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 15/19] python3-py: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (13 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 14/19] python3-iniconfig: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 16/19] python3-pluggy: " Tim Orling
                   ` (4 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstreama provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-py_1.11.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-py_1.11.0.bb b/meta/recipes-devtools/python/python3-py_1.11.0.bb
index f8be3935e5e..02027248f46 100644
--- a/meta/recipes-devtools/python/python3-py_1.11.0.bb
+++ b/meta/recipes-devtools/python/python3-py_1.11.0.bb
@@ -7,7 +7,7 @@ SRC_URI[sha256sum] = "51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4
 
 DEPENDS += "${PYTHON_PN}-setuptools-scm-native"
 
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 BBCLASSEXTEND = "native nativesdk"
 
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 16/19] python3-pluggy: inherit setuptools_build_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (14 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 15/19] python3-py: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 17/19] python3-setuptools: inherit setuptools_base_meta Tim Orling
                   ` (3 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which while it does not declare the
setuptools.build_meta backend is compatible with it.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-pluggy_1.0.0.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb b/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb
index 14a7ccb0b0d..73bbcb02697 100644
--- a/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb
+++ b/meta/recipes-devtools/python/python3-pluggy_1.0.0.bb
@@ -10,7 +10,7 @@ RDEPENDS:${PN} += "${PYTHON_PN}-importlib-metadata \
                    ${PYTHON_PN}-more-itertools \
 "
 
-inherit pypi ptest setuptools3
+inherit pypi ptest setuptools_build_meta
 
 SRC_URI += " \
 	file://run-ptest \
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 17/19] python3-setuptools: inherit setuptools_base_meta
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (15 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 16/19] python3-pluggy: " Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 18/19] setuptools3.bbclass: refactor for wheels Tim Orling
                   ` (2 subsequent siblings)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Upstream provides a pyproject.toml which declares the
setuptools.build_meta backend for PEP-517 packaging.

We need to bootstrap python3-setuptools-native, simply installing by
unzipping the built wheel. This avoids a dependency loop.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 .../python/python3-setuptools_59.5.0.bb       | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
index 33ca928c3e5..2155101d305 100644
--- a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
+++ b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb
@@ -4,7 +4,7 @@ SECTION = "devel/python"
 LICENSE = "MIT"
 LIC_FILES_CHKSUM = "file://LICENSE;beginline=1;endline=19;md5=7a7126e068206290f3fe9f8d6c713ea6"
 
-inherit pypi setuptools3
+inherit pypi setuptools_build_meta
 
 SRC_URI:append:class-native = " file://0001-conditionally-do-not-fetch-code-by-easy_install.patch"
 
@@ -17,6 +17,19 @@ SRC_URI[sha256sum] = "d144f85102f999444d06f9c0e8c737fd0194f10f2f7e5fdb77573f6e2f
 
 DEPENDS += "${PYTHON_PN}"
 
+# Avoid dependency loop; we bootstrap -native
+DEPENDS:remove:class-native = "python3-pip-native python3-setuptools-native"
+DEPENDS:append:class-native = " unzip-native"
+
+PYPA_WHEEL ?= "${B}/dist/${PYPI_PACKAGE}-${PV}-*.whl"
+
+do_install:class-native() {
+    # Bootstrap to prevent dependency loop in python3-pip-native
+    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
+    unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PYPA_WHEEL} || \
+    bbfatal_log "Failed to unzip wheel: ${PYPA_WHEEL}. Check the logs."
+}
+
 RDEPENDS:${PN} = "\
     ${PYTHON_PN}-2to3 \
     ${PYTHON_PN}-compile \
@@ -38,10 +51,6 @@ RDEPENDS:${PN} = "\
     ${PYTHON_PN}-xml \
 "
 
-do_install:prepend() {
-    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-}
-
 BBCLASSEXTEND = "native nativesdk"
 
 # The pkg-resources module can be used by itself, without the package downloader
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 18/19] setuptools3.bbclass: refactor for wheels
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (16 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 17/19] python3-setuptools: inherit setuptools_base_meta Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21  1:27 ` [PATCH 19/19] python3-more-itertools: set PIP_INSTALL_PACKAGE Tim Orling
  2022-02-21 10:52 ` [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Depend on python3-wheel-native so that we can build with 'setup.py
bdist_wheel'.

Use pip_install_wheel class to install the built wheels with pip, as
intended by upstream Python.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/classes/setuptools3.bbclass | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/meta/classes/setuptools3.bbclass b/meta/classes/setuptools3.bbclass
index fd8499d26c7..12561340b07 100644
--- a/meta/classes/setuptools3.bbclass
+++ b/meta/classes/setuptools3.bbclass
@@ -1,6 +1,7 @@
-inherit setuptools3-base
+inherit setuptools3-base pip_install_wheel
 
-B = "${WORKDIR}/build"
+# bdist_wheel builds in ./dist
+#B = "${WORKDIR}/build"
 
 SETUPTOOLS_BUILD_ARGS ?= ""
 SETUPTOOLS_INSTALL_ARGS ?= "--root=${D} \
@@ -23,20 +24,15 @@ setuptools3_do_compile() {
         STAGING_INCDIR=${STAGING_INCDIR} \
         STAGING_LIBDIR=${STAGING_LIBDIR} \
         ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
-        build --build-base=${B} ${SETUPTOOLS_BUILD_ARGS} || \
-        bbfatal_log "'${PYTHON_PN} setup.py build ${SETUPTOOLS_BUILD_ARGS}' execution failed."
+        bdist_wheel ${SETUPTOOLS_BUILD_ARGS} || \
+        bbfatal_log "'${PYTHON_PN} setup.py bdist_wheel ${SETUPTOOLS_BUILD_ARGS}' execution failed."
 }
 setuptools3_do_compile[vardepsexclude] = "MACHINE"
 
 setuptools3_do_install() {
         cd ${SETUPTOOLS_SETUP_PATH}
-        install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-        STAGING_INCDIR=${STAGING_INCDIR} \
-        STAGING_LIBDIR=${STAGING_LIBDIR} \
-        PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \
-        ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py \
-        build --build-base=${B} install --skip-build ${SETUPTOOLS_INSTALL_ARGS} || \
-        bbfatal_log "'${PYTHON_PN} setup.py install ${SETUPTOOLS_INSTALL_ARGS}' execution failed."
+
+        pip_install_wheel_do_install
 
         # support filenames with *spaces*
         find ${D} -name "*.py" -exec grep -q ${D} {} \; \
@@ -64,5 +60,5 @@ setuptools3_do_install[vardepsexclude] = "MACHINE"
 EXPORT_FUNCTIONS do_configure do_compile do_install
 
 export LDSHARED="${CCLD} -shared"
-DEPENDS += "python3-setuptools-native"
+DEPENDS += "python3-setuptools-native python3-wheel-native"
 
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH 19/19] python3-more-itertools: set PIP_INSTALL_PACKAGE
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (17 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 18/19] setuptools3.bbclass: refactor for wheels Tim Orling
@ 2022-02-21  1:27 ` Tim Orling
  2022-02-21 10:52 ` [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)
  19 siblings, 0 replies; 22+ messages in thread
From: Tim Orling @ 2022-02-21  1:27 UTC (permalink / raw)
  To: openembedded-core

Define PIP_INSTALL_PACKAGE as more_itertools to match the name of the
wheel.

[YOCTO #14638]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb b/meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb
index c1e34b205da..5788af3f71e 100644
--- a/meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb
+++ b/meta/recipes-devtools/python/python3-more-itertools_8.12.0.bb
@@ -7,6 +7,8 @@ SRC_URI[sha256sum] = "7dc6ad46f05f545f900dd59e8dfb4e84a4827b97b3cfecb175ea0c7d24
 
 inherit pypi setuptools3 ptest
 
+PIP_INSTALL_PACKAGE = "more_itertools"
+
 SRC_URI += " \
 	file://run-ptest \
 "
-- 
2.30.2



^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip
  2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
                   ` (18 preceding siblings ...)
  2022-02-21  1:27 ` [PATCH 19/19] python3-more-itertools: set PIP_INSTALL_PACKAGE Tim Orling
@ 2022-02-21 10:52 ` Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)
  2022-02-21 11:03   ` Richard Purdie
  19 siblings, 1 reply; 22+ messages in thread
From: Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco) @ 2022-02-21 10:52 UTC (permalink / raw)
  To: Tim Orling, openembedded-core

Hi,

Seems that python changes cause build fail https://swatbot.yoctoproject.org/collection/6991/

Regards,
Oleksiy

________________________________________
From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> on behalf of Tim Orling <ticotimo@gmail.com>
Sent: Monday, February 21, 2022 03:27
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip

Upstream Python is moving to pyproject.toml rather than setup.py and
wheels rather than eggs. The main place this is documented is in
PEP-517[1]. Installing packages with setup.py install is deprecated.

We have a choice with the 'kirkstone' LTS release of introducing this
series (and fixing any fall out during the stabilization period) or
living with deprecated Python packaging for the duration of the LTS (at
a minimum 2 years). This seems to be a compelling enough reason to bring
in this series, even if it results in a slight slip of the 'feature
freeze', which is officially tomorrow, Monday 21 Feb, 2022.

This series bootstraps python3-setuptools-native,
python3-flit-core-native, python3-wheel-native and python3-pip-native,
simply unzipping the wheel into PYTHON_SITEPACKAGES_DIR. This allows all
other recipes to be installed with pip, as intended by upstream.

Three new classes are introduced:
(1) flit_core which builds PEP-517 compliant packages which use
    flit_core.buildapi as their build-backend.
(2) setuptools_build_meta which builds PEP-517 compliant packages which use
    setuptools_build_meta  as their build-backend.
(3) pip_install_wheel, which simply installs wheels built by other
    classes.

The original setuptools3.bbclass do_compile is modified to
'setup.py bdist_wheel' (from python3-wheel-native) rather than the deprecated
'setup.py build' mechanism.

The original setuptools3.bbclass do_install is modified to
use the pip_install_wheel.bbclass to install wheels with pip rather than
the deprecated 'setup.py install' mechanism.

For pip install, we disable dependency checking (--no-deps) because it
proves to be brittle in our environment without providing full wheel
cache that seems like overhead that we do not need. We also disable
fetching wheels from pypi (--no-index). This is no different than our
normal workflow of requiring dependencies to be defined in a recipe.

Missing from this series is documentation of the new features, classes
and variables (this will be done under separate cover).

Also desired and previously discussed in YP meetings is a helper
function to warn that a recipe which currently inherits setuptools3
but has a pyproject.toml and defines a build-system.build-backend should
instead use of the new PEP-517 classes. This function is still in progress.

During testing, recipes which use python3-setuptools-scm are failing to
determine the version of the python package, For several recipes, the fix
was to use pyproject.toml and PEP-517 build backends. To be sure, other recipes
are still going to fail for the short term, until the root cause can be
determined.

Another PEP-517 build-backend, poetry.core.masonry.api has been
identified in recipes in meta-python, but not in oe-core. The
python3-poetry-core recipe is WIP, but brings in a number of
dependencies that are perhaps not worth it to bring into oe-core at this
time. Rather we can support these recipes in meta-python?

[YOCTO #14638]

[1] https://www.python.org/dev/peps/pep-0517/

The following changes since commit 0539b563c6f41fef383429a13c28bf918231d404:

  libhandy: Use upstream regex to check version of upgrade. (2022-02-20 16:45:25 +0000)

are available in the Git repository at:

  git://git.yoctoproject.org/git/poky-contrib timo/bootstrap-wheels
  http://git.yoctoproject.org/cgit.cgi/git/poky-contrib/log/?h=timo/bootstrap-wheels

Tim Orling (19):
  python3-wheel: move 0.37.0 from meta-python
  python3-flit-core: add recipe for 3.6.0
  python3-flit-core: simplify
  flit_core.bbclass: add helper for newer python packaging
  python3-wheel: inherit flit_core
  pip_install_wheel.bbclass: add helper class
  setuptools_build_meta.bbclass: add helper class
  python3-pip: inherit setuptools_build_meta
  python3-attrs: inherit setuptools_build_meta
  python3-git: inherit setuptools_build_meta
  python3-pytest: inherit setuptools_build_meta
  python3-setuptools-scm: inherit setuptools_build_meta
  python3-zipp: inherit setuptools_build_meta
  python3-iniconfig: inherit setuptools_build_meta
  python3-py: inherit setuptools_build_meta
  python3-pluggy: inherit setuptools_build_meta
  python3-setuptools: inherit setuptools_base_meta
  setuptools3.bbclass: refactor for wheels
  python3-more-itertools: set PIP_INSTALL_PACKAGE

 meta/classes/flit_core.bbclass                |  16 +++
 meta/classes/pip_install_wheel.bbclass        |  18 ++++
 meta/classes/setuptools3.bbclass              |  20 ++--
 meta/classes/setuptools_build_meta.bbclass    |  18 ++++
 .../python/python3-attrs_21.4.0.bb            |   2 +-
 .../python/python3-flit-core_3.6.0.bb         |  41 +++++++
 .../python/python3-git_3.1.26.bb              |   2 +-
 .../python/python3-iniconfig_1.1.1.bb         |   4 +-
 .../python/python3-more-itertools_8.12.0.bb   |   2 +
 .../python/python3-pip_22.0.3.bb              |  19 +++-
 .../python/python3-pluggy_1.0.0.bb            |   2 +-
 .../python/python3-py_1.11.0.bb               |   2 +-
 .../python/python3-pytest_7.0.1.bb            |   4 +-
 .../python/python3-setuptools-scm_6.4.2.bb    |   2 +-
 .../python/python3-setuptools_59.5.0.bb       |  19 +++-
 ...roject.toml-from-flit-backend-branch.patch | 100 ++++++++++++++++++
 .../python/python3-wheel_0.37.1.bb            |  24 +++++
 .../python/python3-zipp_3.7.0.bb              |   2 +-
 18 files changed, 269 insertions(+), 28 deletions(-)
 create mode 100644 meta/classes/flit_core.bbclass
 create mode 100644 meta/classes/pip_install_wheel.bbclass
 create mode 100644 meta/classes/setuptools_build_meta.bbclass
 create mode 100644 meta/recipes-devtools/python/python3-flit-core_3.6.0.bb
 create mode 100644 meta/recipes-devtools/python/python3-wheel/0001-Backport-pyproject.toml-from-flit-backend-branch.patch
 create mode 100644 meta/recipes-devtools/python/python3-wheel_0.37.1.bb

--
2.30.2



^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip
  2022-02-21 10:52 ` [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)
@ 2022-02-21 11:03   ` Richard Purdie
  0 siblings, 0 replies; 22+ messages in thread
From: Richard Purdie @ 2022-02-21 11:03 UTC (permalink / raw)
  To: oobitots, Tim Orling, openembedded-core

On Mon, 2022-02-21 at 10:52 +0000, Oleksiy Obitotskyy via lists.openembedded.org
wrote:
> Hi,
> 
> Seems that python changes cause build fail https://swatbot.yoctoproject.org/collection/6991/

FWIW that was with the other patches in your branch too. I was curious to see
how it looked but it isn't quite there yet :/

Cheers,

Richard



^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2022-02-21 11:03 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-21  1:27 [PATCH 00/19] Python PEP-517: build wheels and installw with pip Tim Orling
2022-02-21  1:27 ` [PATCH 01/19] python3-wheel: move 0.37.0 from meta-python Tim Orling
2022-02-21  1:27 ` [PATCH 02/19] python3-flit-core: add recipe for 3.6.0 Tim Orling
2022-02-21  1:27 ` [PATCH 03/19] python3-flit-core: simplify Tim Orling
2022-02-21  1:27 ` [PATCH 04/19] flit_core.bbclass: add helper for newer python packaging Tim Orling
2022-02-21  1:27 ` [PATCH 05/19] python3-wheel: inherit flit_core Tim Orling
2022-02-21  1:27 ` [PATCH 06/19] pip_install_wheel.bbclass: add helper class Tim Orling
2022-02-21  1:27 ` [PATCH 07/19] setuptools_build_meta.bbclass: " Tim Orling
2022-02-21  1:27 ` [PATCH 08/19] python3-pip: inherit setuptools_build_meta Tim Orling
2022-02-21  1:27 ` [PATCH 09/19] python3-attrs: " Tim Orling
2022-02-21  1:27 ` [PATCH 10/19] python3-git: " Tim Orling
2022-02-21  1:27 ` [PATCH 11/19] python3-pytest: " Tim Orling
2022-02-21  1:27 ` [PATCH 12/19] python3-setuptools-scm: " Tim Orling
2022-02-21  1:27 ` [PATCH 13/19] python3-zipp: " Tim Orling
2022-02-21  1:27 ` [PATCH 14/19] python3-iniconfig: " Tim Orling
2022-02-21  1:27 ` [PATCH 15/19] python3-py: " Tim Orling
2022-02-21  1:27 ` [PATCH 16/19] python3-pluggy: " Tim Orling
2022-02-21  1:27 ` [PATCH 17/19] python3-setuptools: inherit setuptools_base_meta Tim Orling
2022-02-21  1:27 ` [PATCH 18/19] setuptools3.bbclass: refactor for wheels Tim Orling
2022-02-21  1:27 ` [PATCH 19/19] python3-more-itertools: set PIP_INSTALL_PACKAGE Tim Orling
2022-02-21 10:52 ` [OE-core] [PATCH 00/19] Python PEP-517: build wheels and installw with pip Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)
2022-02-21 11:03   ` Richard Purdie

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.