[PATCH 1/4] python3-pyproject-hooks: add new recipe

[PATCH 1/4] python3-pyproject-hooks: add new recipe

[Ross Burton]

This used to be known as python3-pep517 and is a key library used to
build modern Python code.

It is currently vendored into python3-picobuild but I plan to remove
that.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/conf/distro/include/maintainers.inc             |  1 +
 .../python/python3-pyproject-hooks_1.0.0.bb          | 12 ++++++++++++
 2 files changed, 13 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 508d10e091a..83966a250b1 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -662,6 +662,7 @@ RECIPE_MAINTAINER:pn-python3-pygments = "Oleksandr Kravchuk <open.source@oleksan
 RECIPE_MAINTAINER:pn-python3-pygobject = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-pyopenssl = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-pyparsing = "Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>"
+RECIPE_MAINTAINER:pn-python3-pyproject-hooks = "Ross Burton <ross.burton@arm.com>"
 RECIPE_MAINTAINER:pn-python3-pysocks = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-pytest = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-pytest-runner = "Tim Orling <tim.orling@konsulko.com>"
diff --git a/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb b/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb
new file mode 100644
index 00000000000..10d8225c3d5
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb
@@ -0,0 +1,12 @@
+SUMMARY = "A low-level library for calling build-backends in pyproject.toml-based projects"
+HOMEPAGE = "https://github.com/pypa/pyproject-hooks"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=aad69c93f605003e3342b174d9b0708c"
+
+SRC_URI[sha256sum] = "f271b298b97f5955d53fb12b72c1fb1948c22c1a6b70b315c54cedaca0264ef5"
+
+inherit pypi python_flit_core
+
+PYPI_PACKAGE = "pyproject_hooks"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.34.1

[PATCH 2/4] python3-build: add 0.10.0

[Ross Burton]

This is the PEP517 build frontend blessed by the Python Packaging
Authority, and now that Python 3.11 has been released and enough
packages have ported to flit, it can be used without a huge bootstraping
problem.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/conf/distro/include/maintainers.inc           |  1 +
 .../python/python3-build_0.10.0.bb                 | 14 ++++++++++++++
 2 files changed, 15 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3-build_0.10.0.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 83966a250b1..7254c749c51 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -600,6 +600,7 @@ RECIPE_MAINTAINER:pn-python3-atomicwrites = "Tim Orling <tim.orling@konsulko.com
 RECIPE_MAINTAINER:pn-python3-attrs = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-babel = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-bcrypt = "Tim Orling <tim.orling@konsulko.com>"
+RECIPE_MAINTAINER:pn-python3-build = "Ross Burton <ross.burton@arm.com>"
 RECIPE_MAINTAINER:pn-python3-certifi = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-cffi = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-chardet = "Tim Orling <tim.orling@konsulko.com>"
diff --git a/meta/recipes-devtools/python/python3-build_0.10.0.bb b/meta/recipes-devtools/python/python3-build_0.10.0.bb
new file mode 100644
index 00000000000..bebc00a437d
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-build_0.10.0.bb
@@ -0,0 +1,14 @@
+SUMMARY = "A simple, correct PEP517 package builder"
+HOMEPAGE = "https://github.com/pypa/build"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=310439af287b0fb4780b2ad6907c256c"
+
+SRC_URI[sha256sum] = "d5b71264afdb5951d6704482aac78de887c80691c52b88a9ad195983ca2c9269"
+
+inherit pypi python_flit_core
+
+DEPENDS += "python3-packaging-native python3-pyproject-hooks-native"
+
+RDEPENDS:${PN} += "python3-packaging python3-pyproject-hooks"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.34.1

[PATCH 3/4] python_pep517: use python3-build, not picobuild

[Ross Burton]

Now that the bootstrap sequence for pypa/build is just
python3-packaging, and python3-pyproject-hooks, we can use build instead
of picobuild.

The only change visible outside of this class is that the variable
PEP517_PICOBUILD_OPTS is now PEP517_BUILD_OPTS, but I'm only aware of one
recipe that used that variable.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/classes-recipe/python_pep517.bbclass             |  7 ++++---
 meta/recipes-devtools/python/python3-build_0.10.0.bb  | 11 ++++++++++-
 .../python/python3-flit-core_3.8.0.bb                 |  2 +-
 .../python/python3-installer_0.6.0.bb                 |  2 +-
 .../recipes-devtools/python/python3-packaging_22.0.bb |  7 +++++++
 .../python/python3-pyproject-hooks_1.0.0.bb           |  7 +++++++
 6 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/meta/classes-recipe/python_pep517.bbclass b/meta/classes-recipe/python_pep517.bbclass
index 202dde0bc31..a1659c5f628 100644
--- a/meta/classes-recipe/python_pep517.bbclass
+++ b/meta/classes-recipe/python_pep517.bbclass
@@ -10,7 +10,7 @@
 # This class will build a wheel in do_compile, and use pypa/installer to install
 # it in do_install.
 
-DEPENDS:append = " python3-picobuild-native python3-installer-native"
+DEPENDS:append = " python3-build-native python3-installer-native"
 
 # Where to execute the build process from
 PEP517_SOURCE_PATH ?= "${S}"
@@ -18,7 +18,8 @@ PEP517_SOURCE_PATH ?= "${S}"
 # The directory where wheels will be written
 PEP517_WHEEL_PATH ?= "${WORKDIR}/dist"
 
-PEP517_PICOBUILD_OPTS ?= ""
+# Other options to pass to build
+PEP517_BUILD_OPTS ?= ""
 
 # The interpreter to use for installed scripts
 PEP517_INSTALL_PYTHON = "python3"
@@ -36,7 +37,7 @@ python_pep517_do_configure () {
 # When we have Python 3.11 we can parse pyproject.toml to determine the build
 # API entry point directly
 python_pep517_do_compile () {
-    nativepython3 -m picobuild --source ${PEP517_SOURCE_PATH} --dest ${PEP517_WHEEL_PATH} --wheel ${PEP517_PICOBUILD_OPTS}
+    nativepython3 -m build --no-isolation --wheel --outdir ${PEP517_WHEEL_PATH} ${PEP517_SOURCE_PATH} ${PEP517_BUILD_OPTS}
 }
 do_compile[cleandirs] += "${PEP517_WHEEL_PATH}"
 
diff --git a/meta/recipes-devtools/python/python3-build_0.10.0.bb b/meta/recipes-devtools/python/python3-build_0.10.0.bb
index bebc00a437d..770a32023d8 100644
--- a/meta/recipes-devtools/python/python3-build_0.10.0.bb
+++ b/meta/recipes-devtools/python/python3-build_0.10.0.bb
@@ -7,7 +7,16 @@ SRC_URI[sha256sum] = "d5b71264afdb5951d6704482aac78de887c80691c52b88a9ad195983ca
 
 inherit pypi python_flit_core
 
-DEPENDS += "python3-packaging-native python3-pyproject-hooks-native"
+DEPENDS += "python3-pyproject-hooks-native"
+
+DEPENDS:remove:class-native = "python3-build-native"
+
+# Skip dependencies as we're doing a minimal build to bootstrap
+PEP517_BUILD_OPTS:class-native = "--skip-dependency-check"
+
+do_compile:prepend:class-native() {
+    export PYTHONPATH="${S}/src"
+}
 
 RDEPENDS:${PN} += "python3-packaging python3-pyproject-hooks"
 
diff --git a/meta/recipes-devtools/python/python3-flit-core_3.8.0.bb b/meta/recipes-devtools/python/python3-flit-core_3.8.0.bb
index 3d815d53163..2e5b1e80695 100644
--- a/meta/recipes-devtools/python/python3-flit-core_3.8.0.bb
+++ b/meta/recipes-devtools/python/python3-flit-core_3.8.0.bb
@@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "d0f2a8f4bd45dc794befbf5839ecc0fd3830d65a57bd52b5997542fac5
 inherit pypi python_flit_core
 
 # Need to install by hand as there's a dependency loop
-DEPENDS:remove:class-native = " python3-picobuild-native python3-installer-native"
+DEPENDS:remove:class-native = " python3-build-native python3-installer-native"
 DEPENDS:append:class-native = " unzip-native"
 
 # We need the full flit tarball
diff --git a/meta/recipes-devtools/python/python3-installer_0.6.0.bb b/meta/recipes-devtools/python/python3-installer_0.6.0.bb
index e728e47cecb..a2130a4507a 100644
--- a/meta/recipes-devtools/python/python3-installer_0.6.0.bb
+++ b/meta/recipes-devtools/python/python3-installer_0.6.0.bb
@@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "f3bd36cd261b440a88a1190b1becca0578fee90b4b62decc796932fdd5
 inherit pypi python_flit_core
 
 # Bootstrap the native build
-DEPENDS:remove:class-native = "python3-picobuild-native python3-installer-native"
+DEPENDS:remove:class-native = "python3-build-native python3-installer-native"
 
 INSTALL_WHEEL_COMPILE_BYTECODE:class-native = "--no-compile-bytecode"
 
diff --git a/meta/recipes-devtools/python/python3-packaging_22.0.bb b/meta/recipes-devtools/python/python3-packaging_22.0.bb
index bcf077d2c71..22f066416b7 100644
--- a/meta/recipes-devtools/python/python3-packaging_22.0.bb
+++ b/meta/recipes-devtools/python/python3-packaging_22.0.bb
@@ -8,3 +8,10 @@ SRC_URI[sha256sum] = "2198ec20bd4c017b8f9717e00f0c8714076fc2fd93816750ab48e2c41d
 inherit pypi python_flit_core
 
 BBCLASSEXTEND = "native nativesdk"
+
+# Bootstrap the native build
+DEPENDS:remove:class-native = "python3-build-native"
+
+do_compile:class-native () {
+    python_flit_core_do_manual_build
+}
diff --git a/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb b/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb
index 10d8225c3d5..9aca5918ec1 100644
--- a/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb
+++ b/meta/recipes-devtools/python/python3-pyproject-hooks_1.0.0.bb
@@ -10,3 +10,10 @@ inherit pypi python_flit_core
 PYPI_PACKAGE = "pyproject_hooks"
 
 BBCLASSEXTEND = "native nativesdk"
+
+# Bootstrap the native build
+DEPENDS:remove:class-native = "python3-build-native"
+
+do_compile:class-native () {
+    python_flit_core_do_manual_build
+}
-- 
2.34.1

[PATCH 4/4] python3-picobuild: remove

[Ross Burton]

We've moved to using python3-build, so this can be removed.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/conf/distro/include/maintainers.inc      |  1 -
 .../python/python3-picobuild_0.2.bb           | 23 -------------------
 2 files changed, 24 deletions(-)
 delete mode 100644 meta/recipes-devtools/python/python3-picobuild_0.2.bb

diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 7254c749c51..fbcb025c954 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -647,7 +647,6 @@ RECIPE_MAINTAINER:pn-python3-packaging = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-pathlib2 = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-pathspec = "Ross Burton <ross.burton@arm.com>"
 RECIPE_MAINTAINER:pn-python3-pbr = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
-RECIPE_MAINTAINER:pn-python3-picobuild = "Ross Burton <ross.burton@arm.com>"
 RECIPE_MAINTAINER:pn-python3-pip = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
 RECIPE_MAINTAINER:pn-python3-pluggy = "Tim Orling <tim.orling@konsulko.com>"
 RECIPE_MAINTAINER:pn-python3-ply = "Tim Orling <tim.orling@konsulko.com>"
diff --git a/meta/recipes-devtools/python/python3-picobuild_0.2.bb b/meta/recipes-devtools/python/python3-picobuild_0.2.bb
deleted file mode 100644
index 99e4cb87618..00000000000
--- a/meta/recipes-devtools/python/python3-picobuild_0.2.bb
+++ /dev/null
@@ -1,23 +0,0 @@
-SUMMARY = "Library and tool for installing Python wheels"
-HOMEPAGE = "https://gitlab.com/rossburton/picobuild"
-LICENSE = "MIT & ( Apache-2.0 | BSD-2-Clause)"
-
-LIC_FILES_CHKSUM = "file://LICENSE;md5=6c811a9fbdf5641ff0b0d43fbbb440e5 \
-                    file://picobuild/vendored/packaging/LICENSE.APACHE;md5=2ee41112a44fe7014dce33e26468ba93 \
-                    file://picobuild/vendored/packaging/LICENSE.BSD;md5=7bef9bf4a8e4263634d0597e7ba100b8 \
-                    file://picobuild/vendored/pep517/LICENSE;md5=aad69c93f605003e3342b174d9b0708c \
-                    file://picobuild/vendored/pyparsing/LICENSE;md5=657a566233888513e1f07ba13e2f47f1 \
-                    file://picobuild/vendored/tomli/LICENSE;md5=aaaaf0879d17df0110d1aa8c8c9f46f5 \
-                    "
-
-SRC_URI = "git://gitlab.com/rossburton/picobuild.git;protocol=https;branch=main"
-SRCREV = "ed3b16ce48d91df181e5f5d77b9bbc2577b3fd9d"
-PV .= "+git${SRCPV}"
-
-S = "${WORKDIR}/git"
-
-inherit python_flit_core
-
-DEPENDS:remove:class-native = "python3-picobuild-native"
-
-BBCLASSEXTEND = "native nativesdk"
-- 
2.34.1

Re: [OE-core] [PATCH 4/4] python3-picobuild: remove

[Khem Raj]

I think meta-python has
meta-python/recipes-devtools/python/python3-pytest-forked_1.4.0.bb
which needs some TLC after this.

There is another failure with opengl-es-cts

| ModuleNotFoundError: No module named 'build.common'

See https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/2348/steps/14/logs/stdio

On Thu, Jan 12, 2023 at 3:23 AM Ross Burton <ross.burton@arm.com> wrote:
>
> We've moved to using python3-build, so this can be removed.
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/conf/distro/include/maintainers.inc      |  1 -
>  .../python/python3-picobuild_0.2.bb           | 23 -------------------
>  2 files changed, 24 deletions(-)
>  delete mode 100644 meta/recipes-devtools/python/python3-picobuild_0.2.bb
>
> diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
> index 7254c749c51..fbcb025c954 100644
> --- a/meta/conf/distro/include/maintainers.inc
> +++ b/meta/conf/distro/include/maintainers.inc
> @@ -647,7 +647,6 @@ RECIPE_MAINTAINER:pn-python3-packaging = "Tim Orling <tim.orling@konsulko.com>"
>  RECIPE_MAINTAINER:pn-python3-pathlib2 = "Tim Orling <tim.orling@konsulko.com>"
>  RECIPE_MAINTAINER:pn-python3-pathspec = "Ross Burton <ross.burton@arm.com>"
>  RECIPE_MAINTAINER:pn-python3-pbr = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
> -RECIPE_MAINTAINER:pn-python3-picobuild = "Ross Burton <ross.burton@arm.com>"
>  RECIPE_MAINTAINER:pn-python3-pip = "Zang Ruochen <zangrc.fnst@fujitsu.com>"
>  RECIPE_MAINTAINER:pn-python3-pluggy = "Tim Orling <tim.orling@konsulko.com>"
>  RECIPE_MAINTAINER:pn-python3-ply = "Tim Orling <tim.orling@konsulko.com>"
> diff --git a/meta/recipes-devtools/python/python3-picobuild_0.2.bb b/meta/recipes-devtools/python/python3-picobuild_0.2.bb
> deleted file mode 100644
> index 99e4cb87618..00000000000
> --- a/meta/recipes-devtools/python/python3-picobuild_0.2.bb
> +++ /dev/null
> @@ -1,23 +0,0 @@
> -SUMMARY = "Library and tool for installing Python wheels"
> -HOMEPAGE = "https://gitlab.com/rossburton/picobuild"
> -LICENSE = "MIT & ( Apache-2.0 | BSD-2-Clause)"
> -
> -LIC_FILES_CHKSUM = "file://LICENSE;md5=6c811a9fbdf5641ff0b0d43fbbb440e5 \
> -                    file://picobuild/vendored/packaging/LICENSE.APACHE;md5=2ee41112a44fe7014dce33e26468ba93 \
> -                    file://picobuild/vendored/packaging/LICENSE.BSD;md5=7bef9bf4a8e4263634d0597e7ba100b8 \
> -                    file://picobuild/vendored/pep517/LICENSE;md5=aad69c93f605003e3342b174d9b0708c \
> -                    file://picobuild/vendored/pyparsing/LICENSE;md5=657a566233888513e1f07ba13e2f47f1 \
> -                    file://picobuild/vendored/tomli/LICENSE;md5=aaaaf0879d17df0110d1aa8c8c9f46f5 \
> -                    "
> -
> -SRC_URI = "git://gitlab.com/rossburton/picobuild.git;protocol=https;branch=main"
> -SRCREV = "ed3b16ce48d91df181e5f5d77b9bbc2577b3fd9d"
> -PV .= "+git${SRCPV}"
> -
> -S = "${WORKDIR}/git"
> -
> -inherit python_flit_core
> -
> -DEPENDS:remove:class-native = "python3-picobuild-native"
> -
> -BBCLASSEXTEND = "native nativesdk"
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#175794): https://lists.openembedded.org/g/openembedded-core/message/175794
> Mute This Topic: https://lists.openembedded.org/mt/96220394/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [PATCH 4/4] python3-picobuild: remove

[Ross Burton]

On 13 Jan 2023, at 07:51, Khem Raj <raj.khem@gmail.com> wrote:
> 
> I think meta-python has
> meta-python/recipes-devtools/python/python3-pytest-forked_1.4.0.bb
> which needs some TLC after this.
> 
> There is another failure with opengl-es-cts
> 
> | ModuleNotFoundError: No module named 'build.common'
> 
> See https://autobuilder.yoctoproject.org/typhoon/#/builders/88/builds/2348/steps/14/logs/stdio

I have the patch for pytest-forked, failed to send.  I’ve not seen the opengl-es-cts one before, I’ll have a look.

Ross

Re: [OE-core] [PATCH 4/4] python3-picobuild: remove

[Ross Burton]

On 13 Jan 2023, at 10:36, Ross Burton via lists.openembedded.org <ross.burton=arm.com@lists.openembedded.org> wrote:
> I have the patch for pytest-forked, failed to send.  I’ve not seen the opengl-es-cts one before, I’ll have a look.

Patches sent to oe-devel.  Note that the pyforked one needs to merge after the picobuild->build migration.

Ross