[meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Zoltán Böszörményi]

Some python modules (e.g. SciPy, scikit-image) use meson-python
(a.k.a. mesonpy) in pyproject.toml:

    [build-system]
    build-backend = 'mesonpy'

This class, together with python3-meson-python and its dependencies
will allow building such modules.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 ...python_setuptools_build_meta_mesonpy.bbclass | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass

diff --git a/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
new file mode 100644
index 000000000..1d3fdbd7c
--- /dev/null
+++ b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
@@ -0,0 +1,17 @@
+#
+# Copyright OpenEmbedded Contributors
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit setuptools3-base pkgconfig meson-common python3targetconfig python_pep517
+
+DEPENDS += "python3-setuptools-native python3-wheel-native ${@'' if d.getVar('PN') == 'python3-meson-python-native' else 'python3-meson-python-native'}"
+
+PEP517_BUILD_OPTS = '--config-setting=setup-args="${MESONOPTS} ${MESON_SOURCEPATH} ${B} ${MESON_CROSS_FILE} ${EXTRA_OEMESON}"'
+
+export MESONPY_BUILD = "${B}"
+
+# Python pyx -> c -> so build leaves absolute build paths in the code
+INSANE_SKIP:${PN} += "buildpaths"
+INSANE_SKIP:${PN}-src += "buildpaths"
-- 
2.39.2

[meta-python][PATCH 2/4] python3-patchelf: New recipe

[Zoltán Böszörményi]

Dummy module to indicate the presence of patchelf.
Used by python3-meson-python.

The CMakeLists.txt file is a crippled copy from this patchelf
python module's sources, removing almost everything, and
adding a dummy install target, so do_install() doesn't fail.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 .../python/python3-patchelf/CMakeLists.txt    |  7 +++++
 .../python/python3-patchelf/patchelf.patch    | 11 +++++++
 .../python/python3-patchelf_0.17.2.1.bb       | 31 +++++++++++++++++++
 3 files changed, 49 insertions(+)
 create mode 100644 meta-python/recipes-devtools/python/python3-patchelf/CMakeLists.txt
 create mode 100644 meta-python/recipes-devtools/python/python3-patchelf/patchelf.patch
 create mode 100644 meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb

diff --git a/meta-python/recipes-devtools/python/python3-patchelf/CMakeLists.txt b/meta-python/recipes-devtools/python/python3-patchelf/CMakeLists.txt
new file mode 100644
index 000000000..335bb503f
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-patchelf/CMakeLists.txt
@@ -0,0 +1,7 @@
+cmake_minimum_required(VERSION 3.6)
+
+project(PatchelfPythonDistributions)
+
+install(CODE "
+ message(STATUS \"Install patchelf project\")
+")
diff --git a/meta-python/recipes-devtools/python/python3-patchelf/patchelf.patch b/meta-python/recipes-devtools/python/python3-patchelf/patchelf.patch
new file mode 100644
index 000000000..96dbf2180
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-patchelf/patchelf.patch
@@ -0,0 +1,11 @@
+--- patchelf-0.17.2.1/setup.py.old	2023-03-13 10:12:55.951514413 +0100
++++ patchelf-0.17.2.1/setup.py	2023-03-13 10:13:18.877751074 +0100
+@@ -95,8 +95,6 @@
+     version=get_version(),
+     cmdclass=cmdclass,
+ 
+-    scripts=["bin/patchelf"],
+-
+     url="https://github.com/NixOS/patchelf",
+     project_urls={
+         "Source Code": "https://github.com/mayeut/patchelf-pypi",
diff --git a/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
new file mode 100644
index 000000000..630bd808a
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
@@ -0,0 +1,31 @@
+# This is a dummy package needed by python3-meson-python
+SUMMARY = "A small utility to modify the dynamic linker and RPATH of ELF executables."
+LICENSE = "GPL-3.0-or-later & Apache-2.0"
+LIC_FILES_CHKSUM = " \
+    file://LICENSE;md5=86d3f3a95c324c9479bd8986968f4327 \
+    file://COPYING;md5=d32239bcb673463ab874e80d47fae504 \
+"
+
+DEPENDS = "python3-scikit-build-native"
+
+PYPI_PACKAGE = "patchelf"
+PYPI_ARCHIVE_NAME_PREFIX = "pypi-"
+
+inherit pypi python_setuptools_build_meta
+SRC_URI[sha256sum] = "a6eb0dd452ce4127d0d5e1eb26515e39186fa609364274bc1b0b77539cfa7031"
+
+SRC_URI += " \
+    file://CMakeLists.txt \
+    file://patchelf.patch \
+"
+
+addtask do_patchbuild after do_patch before do_configure
+
+do_patchbuild () {
+    rm -f ${S}/CMakeLists.txt
+    cp ${WORKDIR}/CMakeLists.txt ${S}/
+}
+
+RDEPENDS:${PN} = "patchelf"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.39.2

[meta-python][PATCH 3/4] python3-pyproject-metadata: New recipe

[Zoltán Böszörményi]

Dependency of python3-meson-python.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 .../python/python3-pyproject-metadata_0.7.1.bb         | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 meta-python/recipes-devtools/python/python3-pyproject-metadata_0.7.1.bb

diff --git a/meta-python/recipes-devtools/python/python3-pyproject-metadata_0.7.1.bb b/meta-python/recipes-devtools/python/python3-pyproject-metadata_0.7.1.bb
new file mode 100644
index 000000000..32938421d
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-pyproject-metadata_0.7.1.bb
@@ -0,0 +1,10 @@
+SUMMARY = "PEP 621 metadata parsing"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=310439af287b0fb4780b2ad6907c256c"
+
+PYPI_PACKAGE = "pyproject-metadata"
+
+inherit pypi python_setuptools_build_meta
+SRC_URI[sha256sum] = "0a94f18b108b9b21f3a26a3d541f056c34edcb17dc872a144a15618fed7aef67"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.39.2

[meta-python][PATCH 4/4] python3-meson-python: New recipe

[Zoltán Böszörményi]

Used by inherit python_setuptools_build_meta_mesonpy.

Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
---
 .../remove-hardcoded-setup-args.patch         | 123 ++++++++++++++++++
 .../use-installed-mesonpy.patch               |   9 ++
 .../python3-meson-python_0.13.0.pre0.bb       |  26 ++++
 3 files changed, 158 insertions(+)
 create mode 100644 meta-python/recipes-devtools/python/python3-meson-python/remove-hardcoded-setup-args.patch
 create mode 100644 meta-python/recipes-devtools/python/python3-meson-python/use-installed-mesonpy.patch
 create mode 100644 meta-python/recipes-devtools/python/python3-meson-python_0.13.0.pre0.bb

diff --git a/meta-python/recipes-devtools/python/python3-meson-python/remove-hardcoded-setup-args.patch b/meta-python/recipes-devtools/python/python3-meson-python/remove-hardcoded-setup-args.patch
new file mode 100644
index 000000000..3edda85dc
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-meson-python/remove-hardcoded-setup-args.patch
@@ -0,0 +1,123 @@
+--- meson_python-0.13.0.pre0/mesonpy/__init__.py.old	1970-01-01 01:00:00.000000000 +0100
++++ meson_python-0.13.0.pre0/mesonpy/__init__.py	2023-03-13 21:26:52.263117416 +0100
+@@ -669,8 +669,6 @@
+         self._build_dir = pathlib.Path(build_dir).absolute() if build_dir else (self._working_dir / 'build')
+         self._editable_verbose = editable_verbose
+         self._install_dir = self._working_dir / 'install'
+-        self._meson_native_file = self._source_dir / '.mesonpy-native-file.ini'
+-        self._meson_cross_file = self._source_dir / '.mesonpy-cross-file.ini'
+         self._meson_args: MesonArgs = collections.defaultdict(list)
+         self._env = os.environ.copy()
+ 
+@@ -679,32 +677,6 @@
+         if ninja_path is not None:
+             self._env.setdefault('NINJA', str(ninja_path))
+ 
+-        # setuptools-like ARCHFLAGS environment variable support
+-        if sysconfig.get_platform().startswith('macosx-'):
+-            archflags = self._env.get('ARCHFLAGS')
+-            if archflags is not None:
+-                arch, *other = filter(None, (x.strip() for x in archflags.split('-arch')))
+-                if other:
+-                    raise ConfigError(f'Multi-architecture builds are not supported but $ARCHFLAGS={archflags!r}')
+-                macver, _, nativearch = platform.mac_ver()
+-                if arch != nativearch:
+-                    x = self._env.setdefault('_PYTHON_HOST_PLATFORM', f'macosx-{macver}-{arch}')
+-                    if not x.endswith(arch):
+-                        raise ConfigError(f'$ARCHFLAGS={archflags!r} and $_PYTHON_HOST_PLATFORM={x!r} do not agree')
+-                    family = 'aarch64' if arch == 'arm64' else arch
+-                    cross_file_data = textwrap.dedent(f'''
+-                        [binaries]
+-                        c = ['cc', '-arch', {arch!r}]
+-                        cpp = ['c++', '-arch', {arch!r}]
+-                        [host_machine]
+-                        system = 'Darwin'
+-                        cpu = {arch!r}
+-                        cpu_family = {family!r}
+-                        endian = 'little'
+-                    ''')
+-                    self._meson_cross_file.write_text(cross_file_data)
+-                    self._meson_args['setup'].extend(('--cross-file', os.fspath(self._meson_cross_file)))
+-
+         # load config -- PEP 621 support is optional
+         self._config = tomllib.loads(self._source_dir.joinpath('pyproject.toml').read_text())
+         self._pep621 = 'project' in self._config
+@@ -749,19 +721,6 @@
+             [binaries]
+             python = '{sys.executable}'
+         ''')
+-        native_file_mismatch = (
+-            not self._meson_native_file.exists()
+-            or self._meson_native_file.read_text() != native_file_data
+-        )
+-        if native_file_mismatch:
+-            try:
+-                self._meson_native_file.write_text(native_file_data)
+-            except OSError:
+-                # if there are permission errors or something else in the source
+-                # directory, put the native file in the working directory instead
+-                # (this won't survive multiple calls -- Meson will have to be reconfigured)
+-                self._meson_native_file = self._working_dir / '.mesonpy-native-file.ini'
+-                self._meson_native_file.write_text(native_file_data)
+ 
+         # Don't reconfigure if build directory doesn't have meson-private/coredata.data
+         # (means something went wrong)
+@@ -784,7 +743,7 @@
+     def _proc(self, *args: str) -> None:
+         """Invoke a subprocess."""
+         print('{cyan}{bold}+ {}{reset}'.format(' '.join(args), **_STYLES))
+-        r = subprocess.run(list(args), env=self._env, cwd=self._build_dir)
++        r = subprocess.run(' '.join(list(args)).split(), env=self._env, cwd=self._build_dir)
+         if r.returncode != 0:
+             raise SystemExit(r.returncode)
+ 
+@@ -800,22 +759,6 @@
+         """
+         sys_paths = mesonpy._introspection.SYSCONFIG_PATHS
+         setup_args = [
+-            f'--prefix={sys.base_prefix}',
+-            os.fspath(self._source_dir),
+-            os.fspath(self._build_dir),
+-            f'--native-file={os.fspath(self._meson_native_file)}',
+-            # TODO: Allow configuring these arguments
+-            '-Ddebug=false',
+-            '-Doptimization=2',
+-
+-            # XXX: This should not be needed, but Meson is using the wrong paths
+-            #      in some scenarios, like on macOS.
+-            #      https://github.com/mesonbuild/meson-python/pull/87#discussion_r1047041306
+-            '--python.purelibdir',
+-            sys_paths['purelib'],
+-            '--python.platlibdir',
+-            sys_paths['platlib'],
+-
+             # user args
+             *self._meson_args['setup'],
+         ]
+@@ -905,8 +848,7 @@
+         editable_verbose: bool = False,
+     ) -> Iterator[Project]:
+         """Creates a project instance pointing to a temporary working directory."""
+-        with tempfile.TemporaryDirectory(prefix='.mesonpy-', dir=os.fspath(source_dir)) as tmpdir:
+-            yield cls(source_dir, tmpdir, build_dir, meson_args, editable_verbose)
++        yield cls(source_dir, build_dir, build_dir, meson_args, editable_verbose)
+ 
+     @functools.lru_cache()
+     def _info(self, name: str) -> Dict[str, Any]:
+@@ -1105,15 +1047,7 @@
+         for key, value in config_settings.items()
+     }
+ 
+-    builddir_value = config_settings.get('builddir', {})
+-    if len(builddir_value) > 0:
+-        if len(builddir_value) != 1:
+-            raise ConfigError('Only one value for configuration entry "builddir" can be specified')
+-        builddir = builddir_value[0]
+-        if not isinstance(builddir, str):
+-            raise ConfigError(f'Configuration entry "builddir" should be a string not {type(builddir)}')
+-    else:
+-        builddir = None
++    builddir = os.environ.get('MESONPY_BUILD')
+ 
+     def _validate_string_collection(key: str) -> None:
+         assert isinstance(config_settings, Mapping)
diff --git a/meta-python/recipes-devtools/python/python3-meson-python/use-installed-mesonpy.patch b/meta-python/recipes-devtools/python/python3-meson-python/use-installed-mesonpy.patch
new file mode 100644
index 000000000..64c2c304c
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-meson-python/use-installed-mesonpy.patch
@@ -0,0 +1,9 @@
+--- meson_python-0.13.0.pre0/pyproject.toml.old	2023-03-14 08:55:19.092987968 +0100
++++ meson_python-0.13.0.pre0/pyproject.toml	2023-03-14 08:55:40.108672980 +0100
+@@ -1,6 +1,5 @@
+ [build-system]
+ build-backend = 'mesonpy'
+-backend-path = ['.']
+ requires = [
+   'meson>=0.63.3',
+   'pyproject-metadata>=0.7.1',
diff --git a/meta-python/recipes-devtools/python/python3-meson-python_0.13.0.pre0.bb b/meta-python/recipes-devtools/python/python3-meson-python_0.13.0.pre0.bb
new file mode 100644
index 000000000..21a4a6e5f
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-meson-python_0.13.0.pre0.bb
@@ -0,0 +1,26 @@
+SUMMARY = "Meson Python build backend (PEP 517)"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://LICENSE;md5=d580b27e67cc0892a5b005b0be114b60"
+
+DEPENDS = " \
+	meson-native ninja-native python3-ninja-native \
+	python3-pyproject-metadata-native \
+	python3-patchelf-native \
+"
+
+PYPI_PACKAGE = "meson_python"
+
+inherit pypi python_setuptools_build_meta_mesonpy
+SRC_URI[sha256sum] = "8d537a0304709c31c11ffa34872a62a4c06a6a6c24fc862b7fb4306f3e881b95"
+
+SRC_URI:append:class-native = "file://remove-hardcoded-setup-args.patch"
+SRC_URI:append:class-nativesdk = "file://use-installed-mesonpy.patch"
+SRC_URI:append:class-target = "file://use-installed-mesonpy.patch"
+
+RDEPENDS:${PN} = " \
+	meson ninja python3-ninja \
+	python3-pyproject-metadata \
+	python3-patchelf \
+"
+
+BBCLASSEXTEND = "native nativesdk"
-- 
2.39.2

Re: [oe] [meta-python][PATCH 2/4] python3-patchelf: New recipe

[Ross Burton]

On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:
> +++ b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
> @@ -0,0 +1,31 @@
> +# This is a dummy package needed by python3-meson-python

I note the words “dummy package”

> +DEPENDS = "python3-scikit-build-native"

Where is this dependency from?

> +PYPI_PACKAGE = "patchelf"
> +PYPI_ARCHIVE_NAME_PREFIX = "pypi-"
> +
> +inherit pypi python_setuptools_build_meta

Why is it downloading stuff?

Why is any of this needed?  Can’t we just comment out the requirement for patchelf, or write a proper stub package?

Ross

Re: [oe] [meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Ross Burton]

On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:
> 
> Some python modules (e.g. SciPy, scikit-image) use meson-python
> (a.k.a. mesonpy) in pyproject.toml:
> 
>    [build-system]
>    build-backend = 'mesonpy'
> 
> This class, together with python3-meson-python and its dependencies
> will allow building such modules.

So the pep517 build thing provides dependencies (which we ignore), build isolation (which we disable), and automatic use of the right tooling (which we have to specify manually).

So what does the meson-python integration bring?  Can we just run meson directly?

> +DEPENDS += "python3-setuptools-native python3-wheel-native ${@'' if d.getVar('PN') == 'python3-meson-python-native' else 'python3-meson-python-native’}"

FWIW, all the other pep517 classes have a “neat” depends without logic and expect the recipe (python3-meson-python-native in this case) to remove itself from the DEPENDS.

Ross

Re: [oe] [meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Böszörményi Zoltán]

2023. 03. 15. 13:46 keltezéssel, Ross Burton írta:
> On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:
>> Some python modules (e.g. SciPy, scikit-image) use meson-python
>> (a.k.a. mesonpy) in pyproject.toml:
>>
>>     [build-system]
>>     build-backend = 'mesonpy'
>>
>> This class, together with python3-meson-python and its dependencies
>> will allow building such modules.
> So the pep517 build thing provides dependencies (which we ignore),

Does this mean recipes using this class would specify the
build dependencies themselves instead of having them
brought in via the class?

> build isolation (which we disable), and automatic use of the right tooling (which we have to specify manually).

Not sure what this means, please elaborate.
Please instruct me what clean up in this class to get this acceptible.

meta-scipy has a quite old SciPy version and the latest version uses this thing.
I have a working scikit-image 0.20.0 recipe also using this.

> So what does the meson-python integration bring?  Can we just run meson directly?

Unfortunately, no.

meson itself does not provide a PEP517 compliant build backend.
meson-python (mesonpy, https://pypi.org/project/meson-python/) does that as a wrapper over 
meson.

I tried just using meson, but while it the .so parts were built,
none of the .py files were properly owned and no egg metadata
was generated. While the first issue is fixable in the recipe,
the second isn't.

This resulted in that the presence of the mesonpy and its version
were not detected by a dependee module's build system e.g.
scikit-image. I would expect any project using build-backend = 'mesonpy'
would fail without the detecting the meson-python module.

>> +DEPENDS += "python3-setuptools-native python3-wheel-native ${@'' if d.getVar('PN') == 'python3-meson-python-native' else 'python3-meson-python-native’}"
> FWIW, all the other pep517 classes have a “neat” depends without logic and expect the recipe (python3-meson-python-native in this case) to remove itself from the DEPENDS.

Thanks for the pointer, I will make this change.

Re: [oe] [meta-python][PATCH 2/4] python3-patchelf: New recipe

[Böszörményi Zoltán]

2023. 03. 15. 13:42 keltezéssel, Ross Burton írta:
> On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:
>> +++ b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
>> @@ -0,0 +1,31 @@
>> +# This is a dummy package needed by python3-meson-python
> I note the words “dummy package”

See the sources of https://pypi.org/project/patchelf/
There is none, only the build system.

The build system would have just built patchelf, and
installed into ${bindir} which would conflict with the
existing patchelf recipe.

>> +DEPENDS = "python3-scikit-build-native"
> Where is this dependency from?

 From its build system. The meta build complains.
The scikit-build recipe I previously sent was already merged,
so it may be used.

>> +PYPI_PACKAGE = "patchelf"
>> +PYPI_ARCHIVE_NAME_PREFIX = "pypi-"
>> +
>> +inherit pypi python_setuptools_build_meta
> Why is it downloading stuff?
>
> Why is any of this needed?  Can’t we just comment out the requirement for patchelf, or write a proper stub package?

It means less patching in modules depending on this
particular module, like meson-python.

Do you prefer patching out this dependency from meson-python
and any other that may depend on it?

Re: [oe] [meta-python][PATCH 2/4] python3-patchelf: New recipe

[Böszörményi Zoltán]

2023. 03. 15. 15:18 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> 2023. 03. 15. 13:42 keltezéssel, Ross Burton írta:
>> On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org 
>> <zboszor=gmail.com@lists.openembedded.org> wrote:
>>> +++ b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
>>> @@ -0,0 +1,31 @@
>>> +# This is a dummy package needed by python3-meson-python
>> I note the words “dummy package”
>
> See the sources of https://pypi.org/project/patchelf/
> There is none, only the build system.
>
> The build system would have just built patchelf, and
> installed into ${bindir} which would conflict with the
> existing patchelf recipe.
>
>>> +DEPENDS = "python3-scikit-build-native"
>> Where is this dependency from?
>
> From its build system.

Actually, it's in mesonpy/__init__.py:

class _depstr:
     """Namespace that holds the requirement strings for dependencies we *might*
     need at runtime. Having them in one place makes it easier to update.
     """
     patchelf = 'patchelf >= 0.11.0'
     ninja = f'ninja >= {_NINJA_REQUIRED_VERSION}'

> The meta build complains.

meson-python uses itself as the build backend to built itself,
so the build really does complain if the patchelf dummy
module is not present. ¯\_(ツ)_/¯

> The scikit-build recipe I previously sent was already merged,
> so it may be used.
>
>>> +PYPI_PACKAGE = "patchelf"
>>> +PYPI_ARCHIVE_NAME_PREFIX = "pypi-"
>>> +
>>> +inherit pypi python_setuptools_build_meta
>> Why is it downloading stuff?
>>
>> Why is any of this needed?  Can’t we just comment out the requirement for patchelf, or 
>> write a proper stub package?
>
> It means less patching in modules depending on this
> particular module, like meson-python.
>
> Do you prefer patching out this dependency from meson-python
> and any other that may depend on it?
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#101562): https://lists.openembedded.org/g/openembedded-devel/message/101562
> Mute This Topic: https://lists.openembedded.org/mt/97622994/3617728
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [oe] [meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Ross Burton]

On 15 Mar 2023, at 14:16, Böszörményi Zoltán <zboszor@gmail.com> wrote:
> 
> 2023. 03. 15. 13:46 keltezéssel, Ross Burton írta:
>> On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org <zboszor=gmail.com@lists.openembedded.org> wrote:
>>> Some python modules (e.g. SciPy, scikit-image) use meson-python
>>> (a.k.a. mesonpy) in pyproject.toml:
>>> 
>>>    [build-system]
>>>    build-backend = 'mesonpy'
>>> 
>>> This class, together with python3-meson-python and its dependencies
>>> will allow building such modules.
>> So the pep517 build thing provides dependencies (which we ignore),
> 
> Does this mean recipes using this class would specify the
> build dependencies themselves instead of having them
> brought in via the class?

Yes. That’s explicit and intentionally the desired behaviour.  Dependencies are declared in the recipe.

>> build isolation (which we disable), and automatic use of the right tooling (which we have to specify manually).
> 
> Not sure what this means, please elaborate.

The python ‘build’ tool can do builds inside a minimal virtual environment. We deliberately don’t do this: dependencies are expected to be in the sysroot via DEPENDS, and they should be used. build won’t be able to fetch anything else during do_compile anyway, as the network is disabled.

build also reads the pyproject.toml to determine what build tool to use.  As we need dependencies up front this isn’t possible.

> Please instruct me what clean up in this class to get this acceptible.
> 
> meta-scipy has a quite old SciPy version and the latest version uses this thing.
> I have a working scikit-image 0.20.0 recipe also using this.
> 
>> So what does the meson-python integration bring?  Can we just run meson directly?
> 
> Unfortunately, no.
> 
> meson itself does not provide a PEP517 compliant build backend.
> meson-python (mesonpy, https://pypi.org/project/meson-python/) does that as a wrapper over meson.

Right, but if we’re just calling meson that isn’t relevant.

> I tried just using meson, but while it the .so parts were built,
> none of the .py files were properly owned and no egg metadata
> was generated. While the first issue is fixable in the recipe,
> the second isn’t.

So the point of meson-python is to write the package metadata, right.

It seems very overcomplicated for such a simple job, but fine.

I’ll give this another review tomorrow, but fyi you can remove the setuptools dependency.

Ross

Re: [oe] [meta-python][PATCH 2/4] python3-patchelf: New recipe

[Böszörményi Zoltán]

2023. 03. 15. 18:49 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
> 2023. 03. 15. 15:18 keltezéssel, Zoltan Boszormenyi via lists.openembedded.org írta:
>> 2023. 03. 15. 13:42 keltezéssel, Ross Burton írta:
>>> On 15 Mar 2023, at 08:04, Zoltan Boszormenyi via lists.openembedded.org 
>>> <zboszor=gmail.com@lists.openembedded.org> wrote:
>>>> +++ b/meta-python/recipes-devtools/python/python3-patchelf_0.17.2.1.bb
>>>> @@ -0,0 +1,31 @@
>>>> +# This is a dummy package needed by python3-meson-python
>>> I note the words “dummy package”
>>
>> See the sources of https://pypi.org/project/patchelf/
>> There is none, only the build system.
>>
>> The build system would have just built patchelf, and
>> installed into ${bindir} which would conflict with the
>> existing patchelf recipe.
>>
>>>> +DEPENDS = "python3-scikit-build-native"
>>> Where is this dependency from?
>>
>> From its build system.

patchelf's pyproject.toml references it, in turn
the scikit-build uses cmake to build things.
The easiest way was to replace the original CMakeLists.txt
with a crippled one to only print a line when installing.

The below part is the patchelf dependency in mesonpy,
sorry for my confusion.

> Actually, it's in mesonpy/__init__.py:
>
> class _depstr:
>     """Namespace that holds the requirement strings for dependencies we *might*
>     need at runtime. Having them in one place makes it easier to update.
>     """
>     patchelf = 'patchelf >= 0.11.0'
>     ninja = f'ninja >= {_NINJA_REQUIRED_VERSION}'
>
>> The meta build complains.
>
> meson-python uses itself as the build backend to built itself,
> so the build really does complain if the patchelf dummy
> module is not present. ¯\_(ツ)_/¯
>
>> The scikit-build recipe I previously sent was already merged,
>> so it may be used.
>>
>>>> +PYPI_PACKAGE = "patchelf"
>>>> +PYPI_ARCHIVE_NAME_PREFIX = "pypi-"
>>>> +
>>>> +inherit pypi python_setuptools_build_meta
>>> Why is it downloading stuff?
>>>
>>> Why is any of this needed?  Can’t we just comment out the requirement for patchelf, or 
>>> write a proper stub package?
>>
>> It means less patching in modules depending on this
>> particular module, like meson-python.
>>
>> Do you prefer patching out this dependency from meson-python
>> and any other that may depend on it?
>>
>>
>>
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#101565): https://lists.openembedded.org/g/openembedded-devel/message/101565
> Mute This Topic: https://lists.openembedded.org/mt/97622994/3617728
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [zboszor@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [oe] [meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 2533 bytes --]

On Wed, Mar 15, 2023 at 1:04 AM Zoltan Boszormenyi <zboszor@gmail.com>
wrote:

> Some python modules (e.g. SciPy, scikit-image) use meson-python
> (a.k.a. mesonpy) in pyproject.toml:
>
>     [build-system]
>     build-backend = 'mesonpy'
>
> This class, together with python3-meson-python and its dependencies
> will allow building such modules.
>
> Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
> ---
>  ...python_setuptools_build_meta_mesonpy.bbclass | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>  create mode 100644
> meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
>

The name of the class is overly long. If you look at the other classes in
oe-core, they basically use the build-backend value,
prefixed with the "python_" namespace, so I would propose just
"python_mesonpy.bbclass"

The python_setuptools_build_meta.bbclass was named as such because the
build-backend is "setuptools.build_meta"
such as in https://github.com/jaraco/zipp/blob/main/pyproject.toml#L3


> diff --git
> a/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
> b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
> new file mode 100644
> index 000000000..1d3fdbd7c
> --- /dev/null
> +++ b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
> @@ -0,0 +1,17 @@
> +#
> +# Copyright OpenEmbedded Contributors
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +inherit setuptools3-base pkgconfig meson-common python3targetconfig
> python_pep517
> +
> +DEPENDS += "python3-setuptools-native python3-wheel-native ${@'' if
> d.getVar('PN') == 'python3-meson-python-native' else
> 'python3-meson-python-native'}"
> +
> +PEP517_BUILD_OPTS = '--config-setting=setup-args="${MESONOPTS}
> ${MESON_SOURCEPATH} ${B} ${MESON_CROSS_FILE} ${EXTRA_OEMESON}"'
> +
> +export MESONPY_BUILD = "${B}"
> +
> +# Python pyx -> c -> so build leaves absolute build paths in the code
> +INSANE_SKIP:${PN} += "buildpaths"
> +INSANE_SKIP:${PN}-src += "buildpaths"
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#101529):
> https://lists.openembedded.org/g/openembedded-devel/message/101529
> Mute This Topic: https://lists.openembedded.org/mt/97622993/924729
> Group Owner: openembedded-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [
> ticotimo@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 3852 bytes --]

Re: [oe] [meta-python][PATCH 1/4] python_setuptools_build_meta_mesonpy.bbclass: New class

[Böszörményi Zoltán]

2023. 03. 16. 16:51 keltezéssel, Tim Orling írta:
>
>
> On Wed, Mar 15, 2023 at 1:04 AM Zoltan Boszormenyi <zboszor@gmail.com> wrote:
>
>     Some python modules (e.g. SciPy, scikit-image) use meson-python
>     (a.k.a. mesonpy) in pyproject.toml:
>
>         [build-system]
>         build-backend = 'mesonpy'
>
>     This class, together with python3-meson-python and its dependencies
>     will allow building such modules.
>
>     Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
>     ---
>      ...python_setuptools_build_meta_mesonpy.bbclass | 17 +++++++++++++++++
>      1 file changed, 17 insertions(+)
>      create mode 100644 meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
>
>
> The name of the class is overly long. If you look at the other classes in oe-core, they 
> basically use the build-backend value,
> prefixed with the "python_" namespace, so I would propose just "python_mesonpy.bbclass"

I agree. I will shorten the name of the class.

>
> The python_setuptools_build_meta.bbclass was named as such because the build-backend is 
> "setuptools.build_meta"
> such as in https://github.com/jaraco/zipp/blob/main/pyproject.toml#L3
>
>
>     diff --git a/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
>     b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
>     new file mode 100644
>     index 000000000..1d3fdbd7c
>     --- /dev/null
>     +++ b/meta-python/classes/python_setuptools_build_meta_mesonpy.bbclass
>     @@ -0,0 +1,17 @@
>     +#
>     +# Copyright OpenEmbedded Contributors
>     +#
>     +# SPDX-License-Identifier: MIT
>     +#
>     +
>     +inherit setuptools3-base pkgconfig meson-common python3targetconfig python_pep517
>     +
>     +DEPENDS += "python3-setuptools-native python3-wheel-native ${@'' if d.getVar('PN')
>     == 'python3-meson-python-native' else 'python3-meson-python-native'}"
>     +
>     +PEP517_BUILD_OPTS = '--config-setting=setup-args="${MESONOPTS} ${MESON_SOURCEPATH}
>     ${B} ${MESON_CROSS_FILE} ${EXTRA_OEMESON}"'
>     +
>     +export MESONPY_BUILD = "${B}"
>     +
>     +# Python pyx -> c -> so build leaves absolute build paths in the code
>     +INSANE_SKIP:${PN} += "buildpaths"
>     +INSANE_SKIP:${PN}-src += "buildpaths"
>     -- 
>     2.39.2
>
>
>     -=-=-=-=-=-=-=-=-=-=-=-
>     Links: You receive all messages sent to this group.
>     View/Reply Online (#101529):
>     https://lists.openembedded.org/g/openembedded-devel/message/101529
>     Mute This Topic: https://lists.openembedded.org/mt/97622993/924729
>     Group Owner: openembedded-devel+owner@lists.openembedded.org
>     <mailto:openembedded-devel%2Bowner@lists.openembedded.org>
>     Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub
>     [ticotimo@gmail.com]
>     -=-=-=-=-=-=-=-=-=-=-=-
>