All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation
@ 2022-08-05 14:36 Guillaume W. Bres
  2022-08-05 14:36 ` [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0 Guillaume W. Bres
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Guillaume W. Bres @ 2022-08-05 14:36 UTC (permalink / raw)
  To: buildroot; +Cc: Guillaume W. Bres

python-scipy is moving to a meson build infrastructure,
and requires fortran cross-compilation

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
---
 package/pkg-meson.mk                   | 2 ++
 support/misc/cross-compilation.conf.in | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/package/pkg-meson.mk b/package/pkg-meson.mk
index 156b005fdd..32d41170f3 100644
--- a/package/pkg-meson.mk
+++ b/package/pkg-meson.mk
@@ -76,10 +76,12 @@ define PKG_MESON_CROSSCONFIG_SED
         -e "s%@TARGET_CC@%$(TARGET_CC)%g" \
         -e "s%@TARGET_CXX@%$(TARGET_CXX)%g" \
         -e "s%@TARGET_AR@%$(TARGET_AR)%g" \
+        -e "s%@TARGET_FC@%$(TARGET_FC)%g" \
         -e "s%@TARGET_STRIP@%$(TARGET_STRIP)%g" \
         -e "s%@TARGET_ARCH@%$(PKG_MESON_TARGET_CPU_FAMILY)%g" \
         -e "s%@TARGET_CPU@%$(GCC_TARGET_CPU)%g" \
         -e "s%@TARGET_ENDIAN@%$(call qstrip,$(call LOWERCASE,$(BR2_ENDIAN)))%g" \
+        -e "s%@TARGET_FCFLAGS@%$(call make-sq-comma-list,$($(strip $(2))))%g" \
         -e "s%@TARGET_CFLAGS@%$(call make-sq-comma-list,$($(strip $(1))))%g" \
         -e "s%@TARGET_LDFLAGS@%$(call make-sq-comma-list,$($(strip $(3))))%g" \
         -e "s%@TARGET_CXXFLAGS@%$(call make-sq-comma-list,$($(strip $(2))))%g" \
diff --git a/support/misc/cross-compilation.conf.in b/support/misc/cross-compilation.conf.in
index 235e5a8495..7053df472b 100644
--- a/support/misc/cross-compilation.conf.in
+++ b/support/misc/cross-compilation.conf.in
@@ -9,6 +9,7 @@ cpp = '@TARGET_CXX@'
 ar = '@TARGET_AR@'
 strip = '@TARGET_STRIP@'
 cmake = '@BR2_CMAKE@'
+fortran = '@TARGET_FC@'
 pkgconfig = '@PKGCONF_HOST_BINARY@'
 g-ir-compiler = '@STAGING_DIR@/usr/bin/g-ir-compiler'
 g-ir-scanner = '@STAGING_DIR@/usr/bin/g-ir-scanner'
@@ -18,6 +19,8 @@ c_args = [@TARGET_CFLAGS@]
 c_link_args = [@TARGET_LDFLAGS@]
 cpp_args = [@TARGET_CXXFLAGS@]
 cpp_link_args = [@TARGET_LDFLAGS@]
+fortran_args = [@TARGET_FCFLAGS@]
+fortran_link_args = [@TARGET_LDFLAGS@]
 wrap_mode = 'nodownload'
 cmake_prefix_path = '@STAGING_DIR@/usr/lib/cmake'
 
-- 
2.30.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-05 14:36 [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Guillaume W. Bres
@ 2022-08-05 14:36 ` Guillaume W. Bres
  2022-08-05 14:46   ` Guillaume Bres
  2022-08-06 22:13 ` [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Thomas Petazzoni via buildroot
  2023-06-26 21:34 ` James Hilliard
  2 siblings, 1 reply; 12+ messages in thread
From: Guillaume W. Bres @ 2022-08-05 14:36 UTC (permalink / raw)
  To: buildroot; +Cc: Guillaume W. Bres

SciPy is moving to a meson build infrastructure.
The current meson build scripts deduce all internal
locations based on the host-python location, which is incorrect
when dealing with "npyrandom" and "npymath", that are included
in code that gets cross-compiled.

https://github.com/scipy/scipy/issues/14812

Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
---
As expected, moving to meson already makes things much easier for us,
as meson is capable of deducing a cross-compilation context and can take
advantage of pkg-config too. Also, the distutils approach will get removed before 1.12.0.

The npy core lib "problem" is acknowledged by the scipy dev team and we can expect improved
behavior in the next releases. Ralf Gommers from the team provided
valuable input for this patch serie, thank you!

Run the runtime test with
./support/testing/run-tests -d dl -o test -k test.package.test_python_scipy.TestPythonPy3SciPy
---
 ...uild-fix-numpy-core-include-location.patch | 33 +++++++++++++++
 package/python-scipy/python-scipy.hash        |  2 +-
 package/python-scipy/python-scipy.mk          | 42 +++++++++----------
 3 files changed, 53 insertions(+), 24 deletions(-)
 create mode 100644 package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch

diff --git a/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
new file mode 100644
index 0000000000..3b5ee78f3e
--- /dev/null
+++ b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
@@ -0,0 +1,33 @@
+From 3045d4ebc83dda5922175758adc74018aebccdd1 Mon Sep 17 00:00:00 2001
+From: "Guillaume W. Bres" <guillaume.bressaix@gmail.com>
+Date: Fri, 5 Aug 2022 16:02:21 +0200
+Subject: [PATCH] scipy/meson.build: fix numpy core include location
+
+The numpy core location is deduced from host-python,
+while we will include lib npyrandom and npymath when cross compiling.
+These two must be previously cross-compiled.
+
+Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
+---
+ scipy/meson.build | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/scipy/meson.build b/scipy/meson.build
+index 76bb3606e..f65601aee 100644
+--- a/scipy/meson.build
++++ b/scipy/meson.build
+@@ -52,9 +52,9 @@ inc_f2py = include_directories(incdir_f2py)
+ fortranobject_c = incdir_f2py / 'fortranobject.c'
+ 
+ cc = meson.get_compiler('c')
+-npymath_path = incdir_numpy / '..' / 'lib'
++npymath_path = @NUMPY_SYSROOT_INCDIR@ / '..' / 'lib'
+ npymath_lib = cc.find_library('npymath', dirs: npymath_path)
+-npyrandom_path = incdir_numpy / '..' / '..' / 'random' / 'lib'
++npyrandom_path = @NUMPY_SYSROOT_INCDIR@ / '..' / '..' / 'random' / 'lib'
+ # Note: `required: false` can be removed once numpy 1.19 is the minimum version
+ npyrandom_lib = cc.find_library('npyrandom', dirs: npyrandom_path, required: false)
+ 
+-- 
+2.30.2
+
diff --git a/package/python-scipy/python-scipy.hash b/package/python-scipy/python-scipy.hash
index 0274679c43..6637faf5b4 100644
--- a/package/python-scipy/python-scipy.hash
+++ b/package/python-scipy/python-scipy.hash
@@ -1,5 +1,5 @@
 # Locally generated
-sha256  0140dd24d14ce194fdb3df496c2190f0896ca84ac98bf5d69c3b3fb7bde1a1e9  scipy-1.8.1.tar.xz
+sha256  c0dfd7d2429452e7e94904c6a3af63cbaa3cf51b348bd9d35b42db7e9ad42791  scipy-1.9.0.tar.gz
 sha256  96599f8ec69a2cc609265eed13fc75090aacb28742260e55ce623788a33da19c  LICENSE.txt
 sha256  37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35  scipy/linalg/src/lapack_deprecations/LICENSE
 sha256  606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609  scipy/ndimage/LICENSE.txt
diff --git a/package/python-scipy/python-scipy.mk b/package/python-scipy/python-scipy.mk
index b5006a2e15..cb0b078e0b 100644
--- a/package/python-scipy/python-scipy.mk
+++ b/package/python-scipy/python-scipy.mk
@@ -4,8 +4,8 @@
 #
 ################################################################################
 
-PYTHON_SCIPY_VERSION = 1.8.1
-PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.xz
+PYTHON_SCIPY_VERSION = 1.9.0
+PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.gz
 PYTHON_SCIPY_SITE = https://github.com/scipy/scipy/releases/download/v$(PYTHON_SCIPY_VERSION)
 PYTHON_SCIPY_LICENSE = \
 	BSD-3-Clause, \
@@ -23,6 +23,7 @@ PYTHON_SCIPY_LICENSE_FILES = \
 	scipy/sparse/linalg/_eigen/arpack/ARPACK/COPYING \
 	scipy/spatial/qhull_src/COPYING.txt
 PYTHON_SCIPY_DEPENDENCIES += \
+	host-pkgconf \
 	host-python-numpy \
 	host-python-pythran \
 	zlib \
@@ -31,26 +32,21 @@ PYTHON_SCIPY_DEPENDENCIES += \
 	python-pybind
 PYTHON_SCIPY_INSTALL_STAGING = YES
 
-PYTHON_SCIPY_SETUP_TYPE = setuptools
-PYTHON_SCIPY_BUILD_OPTS = config_fc --fcompiler=gnu95
-
-PYTHON_SCIPY_LDFLAGS = $(TARGET_LDFLAGS) -shared \
-	-L$(PYTHON3_PATH)/site-packages/numpy/core/lib
-# -lnpyrandom localization
-PYTHON_SCIPY_LDFLAGS += \
-	-L$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/numpy/random/lib
-
-PYTHON_SCIPY_ENV = \
-	F90=$(TARGET_FC) \
-	LDFLAGS="$(PYTHON_SCIPY_LDFLAGS)"
-
-# Provide system configuration options to numpy distutils extensions, telling
-# to find all include files and libraries in staging directory.
-define PYTHON_SCIPY_CONFIGURE_CMDS
-	-rm -f $(@D)/site.cfg
-	echo "[DEFAULT]" >> $(@D)/site.cfg
-	echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
-	echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
+# the meson.build infrastructure, currently uses host-python
+# for code generation, f2py invokation, but also to determine
+# all internal dependencies location. It is incorrect regarging
+# libnpyrandom and libnpymath because we need the cross-compiled version
+# for these two
+# sed -i 's/@NUMPY_SYSROOT_INCDIR@/$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy/' $(@D)/scipy/meson.build
+define PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
+	sed -i \
+	    "s:@NUMPY_SYSROOT_INCDIR@:'$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy\/core\/include\/':" \
+	        $(@D)/scipy/meson.build
 endef
+PYTHON_SCIPY_PRE_CONFIGURE_HOOKS += PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
+
+PYTHON_SCIPY_CONF_OPTS = \
+	-Dblas=lapack \
+	-Dlapack=lapack
 
-$(eval $(python-package))
+$(eval $(meson-package))
-- 
2.30.2

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-05 14:36 ` [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0 Guillaume W. Bres
@ 2022-08-05 14:46   ` Guillaume Bres
  2022-08-08 20:58     ` Arnout Vandecappelle
  0 siblings, 1 reply; 12+ messages in thread
From: Guillaume Bres @ 2022-08-05 14:46 UTC (permalink / raw)
  To: buildroot; +Cc: ralf.gommers, Thomas Petazzoni


[-- Attachment #1.1: Type: text/plain, Size: 7795 bytes --]

Hello Thomas,

Thanks to Ralf Gommers from the Scipy dev team, we understand what was
wrong in the previous submissions, we have a fix, see my v3 submission.
I wanted to mark this one as ready for review, but decided not to.. Build
passes for standard platforms, but the runtime test fails.

$scipy/scipy/meson.build l70: the pybind11 include path is resolved to some
$(STAGING_DIR)/include location in the runtime test env,
which "does not exist". Scipy uses a "python import pybind" command to
deduce the pybind11 lib location, which is totally fine, but somehow leads
to faulty behavior in the qemu environment

Guillaume W. Bres
Software engineer
<guillaume.bressaix@gmail.com>


Le ven. 5 août 2022 à 16:37, Guillaume W. Bres <guillaume.bressaix@gmail.com>
a écrit :

> SciPy is moving to a meson build infrastructure.
> The current meson build scripts deduce all internal
> locations based on the host-python location, which is incorrect
> when dealing with "npyrandom" and "npymath", that are included
> in code that gets cross-compiled.
>
> https://github.com/scipy/scipy/issues/14812
>
> Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
> ---
> As expected, moving to meson already makes things much easier for us,
> as meson is capable of deducing a cross-compilation context and can take
> advantage of pkg-config too. Also, the distutils approach will get removed
> before 1.12.0.
>
> The npy core lib "problem" is acknowledged by the scipy dev team and we
> can expect improved
> behavior in the next releases. Ralf Gommers from the team provided
> valuable input for this patch serie, thank you!
>
> Run the runtime test with
> ./support/testing/run-tests -d dl -o test -k
> test.package.test_python_scipy.TestPythonPy3SciPy
> ---
>  ...uild-fix-numpy-core-include-location.patch | 33 +++++++++++++++
>  package/python-scipy/python-scipy.hash        |  2 +-
>  package/python-scipy/python-scipy.mk          | 42 +++++++++----------
>  3 files changed, 53 insertions(+), 24 deletions(-)
>  create mode 100644
> package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
>
> diff --git
> a/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
> b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
> new file mode 100644
> index 0000000000..3b5ee78f3e
> --- /dev/null
> +++
> b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
> @@ -0,0 +1,33 @@
> +From 3045d4ebc83dda5922175758adc74018aebccdd1 Mon Sep 17 00:00:00 2001
> +From: "Guillaume W. Bres" <guillaume.bressaix@gmail.com>
> +Date: Fri, 5 Aug 2022 16:02:21 +0200
> +Subject: [PATCH] scipy/meson.build: fix numpy core include location
> +
> +The numpy core location is deduced from host-python,
> +while we will include lib npyrandom and npymath when cross compiling.
> +These two must be previously cross-compiled.
> +
> +Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
> +---
> + scipy/meson.build | 4 ++--
> + 1 file changed, 2 insertions(+), 2 deletions(-)
> +
> +diff --git a/scipy/meson.build b/scipy/meson.build
> +index 76bb3606e..f65601aee 100644
> +--- a/scipy/meson.build
> ++++ b/scipy/meson.build
> +@@ -52,9 +52,9 @@ inc_f2py = include_directories(incdir_f2py)
> + fortranobject_c = incdir_f2py / 'fortranobject.c'
> +
> + cc = meson.get_compiler('c')
> +-npymath_path = incdir_numpy / '..' / 'lib'
> ++npymath_path = @NUMPY_SYSROOT_INCDIR@ / '..' / 'lib'
> + npymath_lib = cc.find_library('npymath', dirs: npymath_path)
> +-npyrandom_path = incdir_numpy / '..' / '..' / 'random' / 'lib'
> ++npyrandom_path = @NUMPY_SYSROOT_INCDIR@ / '..' / '..' / 'random' / 'lib'
> + # Note: `required: false` can be removed once numpy 1.19 is the minimum
> version
> + npyrandom_lib = cc.find_library('npyrandom', dirs: npyrandom_path,
> required: false)
> +
> +--
> +2.30.2
> +
> diff --git a/package/python-scipy/python-scipy.hash
> b/package/python-scipy/python-scipy.hash
> index 0274679c43..6637faf5b4 100644
> --- a/package/python-scipy/python-scipy.hash
> +++ b/package/python-scipy/python-scipy.hash
> @@ -1,5 +1,5 @@
>  # Locally generated
> -sha256  0140dd24d14ce194fdb3df496c2190f0896ca84ac98bf5d69c3b3fb7bde1a1e9
> scipy-1.8.1.tar.xz
> +sha256  c0dfd7d2429452e7e94904c6a3af63cbaa3cf51b348bd9d35b42db7e9ad42791
> scipy-1.9.0.tar.gz
>  sha256  96599f8ec69a2cc609265eed13fc75090aacb28742260e55ce623788a33da19c
> LICENSE.txt
>  sha256  37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35
> scipy/linalg/src/lapack_deprecations/LICENSE
>  sha256  606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609
> scipy/ndimage/LICENSE.txt
> diff --git a/package/python-scipy/python-scipy.mk b/package/python-scipy/
> python-scipy.mk
> index b5006a2e15..cb0b078e0b 100644
> --- a/package/python-scipy/python-scipy.mk
> +++ b/package/python-scipy/python-scipy.mk
> @@ -4,8 +4,8 @@
>  #
>
>  ################################################################################
>
> -PYTHON_SCIPY_VERSION = 1.8.1
> -PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.xz
> +PYTHON_SCIPY_VERSION = 1.9.0
> +PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.gz
>  PYTHON_SCIPY_SITE =
> https://github.com/scipy/scipy/releases/download/v$(PYTHON_SCIPY_VERSION)
>  PYTHON_SCIPY_LICENSE = \
>         BSD-3-Clause, \
> @@ -23,6 +23,7 @@ PYTHON_SCIPY_LICENSE_FILES = \
>         scipy/sparse/linalg/_eigen/arpack/ARPACK/COPYING \
>         scipy/spatial/qhull_src/COPYING.txt
>  PYTHON_SCIPY_DEPENDENCIES += \
> +       host-pkgconf \
>         host-python-numpy \
>         host-python-pythran \
>         zlib \
> @@ -31,26 +32,21 @@ PYTHON_SCIPY_DEPENDENCIES += \
>         python-pybind
>  PYTHON_SCIPY_INSTALL_STAGING = YES
>
> -PYTHON_SCIPY_SETUP_TYPE = setuptools
> -PYTHON_SCIPY_BUILD_OPTS = config_fc --fcompiler=gnu95
> -
> -PYTHON_SCIPY_LDFLAGS = $(TARGET_LDFLAGS) -shared \
> -       -L$(PYTHON3_PATH)/site-packages/numpy/core/lib
> -# -lnpyrandom localization
> -PYTHON_SCIPY_LDFLAGS += \
> -
>  -L$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/numpy/random/lib
> -
> -PYTHON_SCIPY_ENV = \
> -       F90=$(TARGET_FC) \
> -       LDFLAGS="$(PYTHON_SCIPY_LDFLAGS)"
> -
> -# Provide system configuration options to numpy distutils extensions,
> telling
> -# to find all include files and libraries in staging directory.
> -define PYTHON_SCIPY_CONFIGURE_CMDS
> -       -rm -f $(@D)/site.cfg
> -       echo "[DEFAULT]" >> $(@D)/site.cfg
> -       echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
> -       echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
> +# the meson.build infrastructure, currently uses host-python
> +# for code generation, f2py invokation, but also to determine
> +# all internal dependencies location. It is incorrect regarging
> +# libnpyrandom and libnpymath because we need the cross-compiled version
> +# for these two
> +# sed -i 's/@NUMPY_SYSROOT_INCDIR@/$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy/'
> $(@D)/scipy/meson.build
> +define PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
> +       sed -i \
> +           "s:@NUMPY_SYSROOT_INCDIR@:'$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy\/core\/include\/':"
> \
> +               $(@D)/scipy/meson.build
>  endef
> +PYTHON_SCIPY_PRE_CONFIGURE_HOOKS +=
> PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
> +
> +PYTHON_SCIPY_CONF_OPTS = \
> +       -Dblas=lapack \
> +       -Dlapack=lapack
>
> -$(eval $(python-package))
> +$(eval $(meson-package))
> --
> 2.30.2
>
>

[-- Attachment #1.2: Type: text/html, Size: 9999 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation
  2022-08-05 14:36 [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Guillaume W. Bres
  2022-08-05 14:36 ` [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0 Guillaume W. Bres
@ 2022-08-06 22:13 ` Thomas Petazzoni via buildroot
  2022-08-08 19:47   ` Yann E. MORIN
  2023-06-26 21:34 ` James Hilliard
  2 siblings, 1 reply; 12+ messages in thread
From: Thomas Petazzoni via buildroot @ 2022-08-06 22:13 UTC (permalink / raw)
  To: Guillaume W. Bres; +Cc: buildroot, Yann E. MORIN

Hello,

+Yann in Cc, as there is an open issue, see below.

On Fri,  5 Aug 2022 16:36:24 +0200
"Guillaume W. Bres" <guillaume.bressaix@gmail.com> wrote:

> python-scipy is moving to a meson build infrastructure,
> and requires fortran cross-compilation
> 
> Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
> ---
>  package/pkg-meson.mk                   | 2 ++
>  support/misc/cross-compilation.conf.in | 3 +++
>  2 files changed, 5 insertions(+)

I applied to master... but I went too fast, there were several
incorrect things in this commit.

+        -e "s%@TARGET_FCFLAGS@%$(call make-sq-comma-list,$($(strip $(2))))%g" \

this is wrong: $(2) in PKG_MESON_CROSSCONFIG_SED contains the CXXFLAGS,
not the FCFLAGS. So I changed that to $(4), and passed FCFLAGS as a
fourth argument to this macro.

Also, you were missing setting FCFLAGS in the
PKG_MESON_INSTALL_CROSS_CONF macro.

I fixed this in a follow-up commit:
https://git.buildroot.org/buildroot/commit/?id=cd933d676f210789730ecb25b23d902620d72072.
I hope I got it right.

Now, on to the open issue which explains why I have added Yann in Cc.
We are currently setting the C++ compiler and Fortran compiler path in
Meson's cross-compilation.conf unconditionally... even when the
toolchain has no C++ or Fortran support. So it means we're pointing to
compilers that don't exist.

Ideally, the cross-compilation.conf should be generated with no
reference to a C++ or Fortran compiler, or a /bin/false reference, and
ditto for the CXXFLAGS/FCFLAGS when there is no C++ or Fortran support.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation
  2022-08-06 22:13 ` [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Thomas Petazzoni via buildroot
@ 2022-08-08 19:47   ` Yann E. MORIN
  2022-08-09 16:25     ` Yann E. MORIN
  0 siblings, 1 reply; 12+ messages in thread
From: Yann E. MORIN @ 2022-08-08 19:47 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: buildroot, Guillaume W. Bres

Thomas, All,

On 2022-08-07 00:13 +0200, Thomas Petazzoni spake thusly:
[--SNIP--]
> Now, on to the open issue which explains why I have added Yann in Cc.
> We are currently setting the C++ compiler and Fortran compiler path in
> Meson's cross-compilation.conf unconditionally... even when the
> toolchain has no C++ or Fortran support. So it means we're pointing to
> compilers that don't exist.

I think this is fine. They don't exist, so they can't be run, so meson
will not inadvertently be able to build a C++/Fortran/... program and we
will notice the issue.

> Ideally, the cross-compilation.conf should be generated with no
> reference to a C++ or Fortran compiler, or a /bin/false reference, and
> ditto for the CXXFLAGS/FCFLAGS when there is no C++ or Fortran support.

In the past, we used to use false as a C++ compiler (but not for meson
specifically), and it turned out that that caused quite some
hard-to-understand issues, see 4cd1ab15886a (core: alternate solution to
disable C++).

I'll ask the meson guys in irc://irc.oftc.net/#mesonbuild and se what
they have to say and suggest.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-05 14:46   ` Guillaume Bres
@ 2022-08-08 20:58     ` Arnout Vandecappelle
  2022-08-08 21:13       ` Thomas Petazzoni via buildroot
  0 siblings, 1 reply; 12+ messages in thread
From: Arnout Vandecappelle @ 2022-08-08 20:58 UTC (permalink / raw)
  To: Guillaume Bres, buildroot; +Cc: ralf.gommers, Thomas Petazzoni



On 05/08/2022 16:46, Guillaume Bres wrote:
> Hello Thomas,
> 
> Thanks to Ralf Gommers from the Scipy dev team, we understand what was wrong in 
> the previous submissions, we have a fix, see my v3 submission.
> I wanted to mark this one as ready for review, but decided not to.. Build passes 
> for standard platforms, but the runtime test fails.
> 
> $scipy/scipy/meson.build l70: the pybind11 include path is resolved to some 
> $(STAGING_DIR)/include location in the runtime test env,
> which "does not exist". Scipy uses a "python import pybind" command to deduce 
> the pybind11 lib location, which is totally fine, but somehow leads to faulty 
> behavior in the qemu environment

  pybind is consistently annoying, it uses ldd and stuff to discover paths...

  Regards,
  Arnout

> 
> Guillaume W. Bres
> Software engineer
> <guillaume.bressaix@gmail.com <mailto:guillaume.bressaix@gmail.com>>
> 
> 
> Le ven. 5 août 2022 à 16:37, Guillaume W. Bres <guillaume.bressaix@gmail.com 
> <mailto:guillaume.bressaix@gmail.com>> a écrit :
> 
>     SciPy is moving to a meson build infrastructure.
>     The current meson build scripts deduce all internal
>     locations based on the host-python location, which is incorrect
>     when dealing with "npyrandom" and "npymath", that are included
>     in code that gets cross-compiled.
> 
>     https://github.com/scipy/scipy/issues/14812
>     <https://github.com/scipy/scipy/issues/14812>
> 
>     Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com
>     <mailto:guillaume.bressaix@gmail.com>>
>     ---
>     As expected, moving to meson already makes things much easier for us,
>     as meson is capable of deducing a cross-compilation context and can take
>     advantage of pkg-config too. Also, the distutils approach will get removed
>     before 1.12.0.
> 
>     The npy core lib "problem" is acknowledged by the scipy dev team and we can
>     expect improved
>     behavior in the next releases. Ralf Gommers from the team provided
>     valuable input for this patch serie, thank you!
> 
>     Run the runtime test with
>     ./support/testing/run-tests -d dl -o test -k
>     test.package.test_python_scipy.TestPythonPy3SciPy
>     ---
>       ...uild-fix-numpy-core-include-location.patch | 33 +++++++++++++++
>       package/python-scipy/python-scipy.hash        |  2 +-
>       package/python-scipy/python-scipy.mk <http://python-scipy.mk>          |
>     42 +++++++++----------
>       3 files changed, 53 insertions(+), 24 deletions(-)
>       create mode 100644
>     package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
> 
>     diff --git
>     a/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
>     b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
>     new file mode 100644
>     index 0000000000..3b5ee78f3e
>     --- /dev/null
>     +++
>     b/package/python-scipy/0001-scipy-meson.build-fix-numpy-core-include-location.patch
>     @@ -0,0 +1,33 @@
>     +From 3045d4ebc83dda5922175758adc74018aebccdd1 Mon Sep 17 00:00:00 2001
>     +From: "Guillaume W. Bres" <guillaume.bressaix@gmail.com
>     <mailto:guillaume.bressaix@gmail.com>>
>     +Date: Fri, 5 Aug 2022 16:02:21 +0200
>     +Subject: [PATCH] scipy/meson.build: fix numpy core include location
>     +
>     +The numpy core location is deduced from host-python,
>     +while we will include lib npyrandom and npymath when cross compiling.
>     +These two must be previously cross-compiled.
>     +
>     +Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com
>     <mailto:guillaume.bressaix@gmail.com>>
>     +---
>     + scipy/meson.build | 4 ++--
>     + 1 file changed, 2 insertions(+), 2 deletions(-)
>     +
>     +diff --git a/scipy/meson.build b/scipy/meson.build
>     +index 76bb3606e..f65601aee 100644
>     +--- a/scipy/meson.build
>     ++++ b/scipy/meson.build
>     +@@ -52,9 +52,9 @@ inc_f2py = include_directories(incdir_f2py)
>     + fortranobject_c = incdir_f2py / 'fortranobject.c'
>     +
>     + cc = meson.get_compiler('c')
>     +-npymath_path = incdir_numpy / '..' / 'lib'
>     ++npymath_path = @NUMPY_SYSROOT_INCDIR@ / '..' / 'lib'
>     + npymath_lib = cc.find_library('npymath', dirs: npymath_path)
>     +-npyrandom_path = incdir_numpy / '..' / '..' / 'random' / 'lib'
>     ++npyrandom_path = @NUMPY_SYSROOT_INCDIR@ / '..' / '..' / 'random' / 'lib'
>     + # Note: `required: false` can be removed once numpy 1.19 is the minimum
>     version
>     + npyrandom_lib = cc.find_library('npyrandom', dirs: npyrandom_path,
>     required: false)
>     +
>     +--
>     +2.30.2
>     +
>     diff --git a/package/python-scipy/python-scipy.hash
>     b/package/python-scipy/python-scipy.hash
>     index 0274679c43..6637faf5b4 100644
>     --- a/package/python-scipy/python-scipy.hash
>     +++ b/package/python-scipy/python-scipy.hash
>     @@ -1,5 +1,5 @@
>       # Locally generated
>     -sha256  0140dd24d14ce194fdb3df496c2190f0896ca84ac98bf5d69c3b3fb7bde1a1e9 
>     scipy-1.8.1.tar.xz
>     +sha256  c0dfd7d2429452e7e94904c6a3af63cbaa3cf51b348bd9d35b42db7e9ad42791 
>     scipy-1.9.0.tar.gz
>       sha256  96599f8ec69a2cc609265eed13fc75090aacb28742260e55ce623788a33da19c 
>     LICENSE.txt
>       sha256  37e64a498894ac7c3b070023e3689e954a8ecf8a23b90968d09a455f1b4f7b35 
>     scipy/linalg/src/lapack_deprecations/LICENSE
>       sha256  606209a000716c5f66e33e180ce08434b96ed17db4975ab9723c6b5fbcc89609 
>     scipy/ndimage/LICENSE.txt
>     diff --git a/package/python-scipy/python-scipy.mk <http://python-scipy.mk>
>     b/package/python-scipy/python-scipy.mk <http://python-scipy.mk>
>     index b5006a2e15..cb0b078e0b 100644
>     --- a/package/python-scipy/python-scipy.mk <http://python-scipy.mk>
>     +++ b/package/python-scipy/python-scipy.mk <http://python-scipy.mk>
>     @@ -4,8 +4,8 @@
>       #
>       ################################################################################
> 
>     -PYTHON_SCIPY_VERSION = 1.8.1
>     -PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.xz
>     +PYTHON_SCIPY_VERSION = 1.9.0
>     +PYTHON_SCIPY_SOURCE = scipy-$(PYTHON_SCIPY_VERSION).tar.gz
>       PYTHON_SCIPY_SITE =
>     https://github.com/scipy/scipy/releases/download/v$(PYTHON_SCIPY_VERSION)
>     <https://github.com/scipy/scipy/releases/download/v$(PYTHON_SCIPY_VERSION)>
>       PYTHON_SCIPY_LICENSE = \
>              BSD-3-Clause, \
>     @@ -23,6 +23,7 @@ PYTHON_SCIPY_LICENSE_FILES = \
>              scipy/sparse/linalg/_eigen/arpack/ARPACK/COPYING \
>              scipy/spatial/qhull_src/COPYING.txt
>       PYTHON_SCIPY_DEPENDENCIES += \
>     +       host-pkgconf \
>              host-python-numpy \
>              host-python-pythran \
>              zlib \
>     @@ -31,26 +32,21 @@ PYTHON_SCIPY_DEPENDENCIES += \
>              python-pybind
>       PYTHON_SCIPY_INSTALL_STAGING = YES
> 
>     -PYTHON_SCIPY_SETUP_TYPE = setuptools
>     -PYTHON_SCIPY_BUILD_OPTS = config_fc --fcompiler=gnu95
>     -
>     -PYTHON_SCIPY_LDFLAGS = $(TARGET_LDFLAGS) -shared \
>     -       -L$(PYTHON3_PATH)/site-packages/numpy/core/lib
>     -# -lnpyrandom localization
>     -PYTHON_SCIPY_LDFLAGS += \
>     -     
>       -L$(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/numpy/random/lib
>     -
>     -PYTHON_SCIPY_ENV = \
>     -       F90=$(TARGET_FC) \
>     -       LDFLAGS="$(PYTHON_SCIPY_LDFLAGS)"
>     -
>     -# Provide system configuration options to numpy distutils extensions, telling
>     -# to find all include files and libraries in staging directory.
>     -define PYTHON_SCIPY_CONFIGURE_CMDS
>     -       -rm -f $(@D)/site.cfg
>     -       echo "[DEFAULT]" >> $(@D)/site.cfg
>     -       echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
>     -       echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
>     +# the meson.build infrastructure, currently uses host-python
>     +# for code generation, f2py invokation, but also to determine
>     +# all internal dependencies location. It is incorrect regarging
>     +# libnpyrandom and libnpymath because we need the cross-compiled version
>     +# for these two
>     +# sed -i
>     's/@NUMPY_SYSROOT_INCDIR@/$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy/'
>     $(@D)/scipy/meson.build
>     +define PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
>     +       sed -i \
>     +         
>       "s:@NUMPY_SYSROOT_INCDIR@:'$(STAGING_DIR)\/usr\/lib\/python$(PYTHON3_VERSION_MAJOR)\/site-packages\/numpy\/core\/include\/':" \
>     +               $(@D)/scipy/meson.build
>       endef
>     +PYTHON_SCIPY_PRE_CONFIGURE_HOOKS +=
>     PYTHON_SCIPY_FIX_NUMPY_CORE_SYSROOT_LOCATION
>     +
>     +PYTHON_SCIPY_CONF_OPTS = \
>     +       -Dblas=lapack \
>     +       -Dlapack=lapack
> 
>     -$(eval $(python-package))
>     +$(eval $(meson-package))
>     -- 
>     2.30.2
> 
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-08 20:58     ` Arnout Vandecappelle
@ 2022-08-08 21:13       ` Thomas Petazzoni via buildroot
  2022-08-09  8:37         ` Guillaume Bres
  0 siblings, 1 reply; 12+ messages in thread
From: Thomas Petazzoni via buildroot @ 2022-08-08 21:13 UTC (permalink / raw)
  To: Arnout Vandecappelle; +Cc: buildroot, ralf.gommers, Guillaume Bres

On Mon, 8 Aug 2022 22:58:46 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

>   pybind is consistently annoying, it uses ldd and stuff to discover paths...

Well, it's not just pybind. Python itself, in its ctype.find_library()
function, uses ldconfig, tries to use gcc on the target, and more funky
stuff that borks terribly in a Buildroot situation.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-08 21:13       ` Thomas Petazzoni via buildroot
@ 2022-08-09  8:37         ` Guillaume Bres
  2022-08-27  9:11           ` Guillaume Bres
  0 siblings, 1 reply; 12+ messages in thread
From: Guillaume Bres @ 2022-08-09  8:37 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: buildroot, ralf.gommers


[-- Attachment #1.1: Type: text/plain, Size: 849 bytes --]

for information,
pybind will soon become pkg-config knowledgeable, as this issue is about to
get merged https://github.com/pybind/pybind11/pull/4077

Guillaume W. Bres
Software engineer
<guillaume.bressaix@gmail.com>


Le lun. 8 août 2022 à 23:13, Thomas Petazzoni <thomas.petazzoni@bootlin.com>
a écrit :

> On Mon, 8 Aug 2022 22:58:46 +0200
> Arnout Vandecappelle <arnout@mind.be> wrote:
>
> >   pybind is consistently annoying, it uses ldd and stuff to discover
> paths...
>
> Well, it's not just pybind. Python itself, in its ctype.find_library()
> function, uses ldconfig, tries to use gcc on the target, and more funky
> stuff that borks terribly in a Buildroot situation.
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com
>

[-- Attachment #1.2: Type: text/html, Size: 1847 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation
  2022-08-08 19:47   ` Yann E. MORIN
@ 2022-08-09 16:25     ` Yann E. MORIN
  0 siblings, 0 replies; 12+ messages in thread
From: Yann E. MORIN @ 2022-08-09 16:25 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: buildroot, Guillaume W. Bres

Thomas, All,

On 2022-08-08 21:47 +0200, Yann E. MORIN spake thusly:
> On 2022-08-07 00:13 +0200, Thomas Petazzoni spake thusly:
> [--SNIP--]
> > Now, on to the open issue which explains why I have added Yann in Cc.
> > We are currently setting the C++ compiler and Fortran compiler path in
> > Meson's cross-compilation.conf unconditionally... even when the
> > toolchain has no C++ or Fortran support. So it means we're pointing to
> > compilers that don't exist.
> I think this is fine. They don't exist, so they can't be run, so meson
> will not inadvertently be able to build a C++/Fortran/... program and we
> will notice the issue.
> 
> > Ideally, the cross-compilation.conf should be generated with no
> > reference to a C++ or Fortran compiler, or a /bin/false reference, and
> > ditto for the CXXFLAGS/FCFLAGS when there is no C++ or Fortran support.
> 
> In the past, we used to use false as a C++ compiler (but not for meson
> specifically), and it turned out that that caused quite some
> hard-to-understand issues, see 4cd1ab15886a (core: alternate solution to
> disable C++).
> 
> I'll ask the meson guys in irc://irc.oftc.net/#mesonbuild and se what
> they have to say and suggest.

So, @eliebrokeit there said, quoting:

    If you don't have one defined, cross builds won't try to autodetect
    one, IIRC...
    [...] however setting it to /bin/false will result in it detecting
    that that doesn't work
    I guess the safest bet is to deliberately poison it with /bin/false

So, when we do not have a compiler for a specific language, we should
force it to /bin/false.

Guillaume, would you be willing to tackle that?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
  2022-08-09  8:37         ` Guillaume Bres
@ 2022-08-27  9:11           ` Guillaume Bres
       [not found]             ` <CABL7CQjFbqaDd2ULBsPVyKM3cAEjCSHEdnY9vfxA=VMuRbdNjg@mail.gmail.com>
  0 siblings, 1 reply; 12+ messages in thread
From: Guillaume Bres @ 2022-08-27  9:11 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: buildroot, ralf.gommers


[-- Attachment #1.1: Type: text/plain, Size: 1839 bytes --]

Hello all,

I suggest we wait for new updates of scipy before converting the package to
meson.
In the meantime, I expect pybind and numpy to be sanitized too (probably
the impulse of scipy), so we should keep track of these.
For information, scipy-v1.9.1 has just been released but it does not come
with anything new regarding meson scripts.

I currently have a v3 patch that is able to upgrade scipy to v1.9.0 and
meson, but it involves patching the build script (it's fairly minor).
But this patch is not 100% ready and needs to be finalized.
What do you guys think is best? Should I conclude this, or we remain as is
(which works fine) and wait for new releases?
I am worried such tweaks appear fine at first, but might raise lots of
issues once injected into BR

Guillaume W. Bres
Software engineer
<guillaume.bressaix@gmail.com>


Le mar. 9 août 2022 à 10:37, Guillaume Bres <guillaume.bressaix@gmail.com>
a écrit :

> for information,
> pybind will soon become pkg-config knowledgeable, as this issue is about
> to get merged https://github.com/pybind/pybind11/pull/4077
>
> Guillaume W. Bres
> Software engineer
> <guillaume.bressaix@gmail.com>
>
>
> Le lun. 8 août 2022 à 23:13, Thomas Petazzoni <
> thomas.petazzoni@bootlin.com> a écrit :
>
>> On Mon, 8 Aug 2022 22:58:46 +0200
>> Arnout Vandecappelle <arnout@mind.be> wrote:
>>
>> >   pybind is consistently annoying, it uses ldd and stuff to discover
>> paths...
>>
>> Well, it's not just pybind. Python itself, in its ctype.find_library()
>> function, uses ldconfig, tries to use gcc on the target, and more funky
>> stuff that borks terribly in a Buildroot situation.
>>
>> Thomas
>> --
>> Thomas Petazzoni, co-owner and CEO, Bootlin
>> Embedded Linux and Kernel engineering and training
>> https://bootlin.com
>>
>

[-- Attachment #1.2: Type: text/html, Size: 3593 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0
       [not found]             ` <CABL7CQjFbqaDd2ULBsPVyKM3cAEjCSHEdnY9vfxA=VMuRbdNjg@mail.gmail.com>
@ 2022-09-01 19:43               ` Guillaume Bres
  0 siblings, 0 replies; 12+ messages in thread
From: Guillaume Bres @ 2022-09-01 19:43 UTC (permalink / raw)
  To: Ralf Gommers; +Cc: buildroot, Thomas Petazzoni


[-- Attachment #1.1: Type: text/plain, Size: 4181 bytes --]

>1.9.2 should be the first version built with Meson now that we've got the
wheel build machinery switched over [..]
>I will note that SciPy CI now contains one cross-compilation job (macOS
x-86-64 to arm64), so that should be more robust now

Thank you this is valuable information Ralf

>The main issue for cross-compilation is running the Python interpreter on
the build machine to retrieve numpy and pybind11
Indeed I understand the problem

>Note that you can either wait a bit longer, or if you do want to include
SciPy 1.9.1, just build it with distutils - if that worked for 1.8.x it
will still work for 1.9.1
Ok let's do that then

Guillaume W. Bres
Software engineer
<guillaume.bressaix@gmail.com>


Le sam. 27 août 2022 à 11:54, Ralf Gommers <ralf.gommers@gmail.com> a
écrit :

> Hi Guillaume, all,
>
>
> On Sat, Aug 27, 2022 at 12:11 PM Guillaume Bres <
> guillaume.bressaix@gmail.com> wrote:
>
>> Hello all,
>>
>> I suggest we wait for new updates of scipy before converting the package
>> to meson.
>>
>
> That does sound like the right call. The SciPy wheels on PyPI for 1.9.1
> are still built with distutils; 1.9.2 should be the first version built
> with Meson now that we've got the wheel build machinery switched over.
> Making that move to Meson before SciPy itself has finalized it is likely to
> uncover some more issues. I will note that SciPy CI now contains one
> cross-compilation job (macOS x-86-64 to arm64), so that should be more
> robust now.
>
>
>> In the meantime, I expect pybind and numpy to be sanitized too (probably
>> the impulse of scipy), so we should keep track of these.
>>
>
> Indeed. The main issue for cross-compilation is running the Python
> interpreter on the build machine to retrieve numpy and pybind11. I'm trying
> to get rid of that need. I will note that the distutils-based build does
> the same and ends up with incorrect static libraries included in the Python
> extension modules, but it happens to work for odd reasons (basically two
> bugs cancelling each other out there). The Meson builds correctly detects
> that situation as problematic and errors out rather than including a static
> library with the wrong architecture.
>
> For information, scipy-v1.9.1 has just been released but it does not come
>> with anything new regarding meson scripts.
>>
>> I currently have a v3 patch that is able to upgrade scipy to v1.9.0 and
>> meson, but it involves patching the build script (it's fairly minor).
>> But this patch is not 100% ready and needs to be finalized.
>> What do you guys think is best? Should I conclude this, or we remain as
>> is (which works fine) and wait for new releases?
>>
>
> Note that you can either wait a bit longer, or if you do want to include
> SciPy 1.9.1, just build it with distutils - if that worked for 1.8.x it
> will still work for 1.9.1
>
> Cheers,
> Ralf
>
>
> I am worried such tweaks appear fine at first, but might raise lots of
>> issues once injected into BR
>>
>> Guillaume W. Bres
>> Software engineer
>> <guillaume.bressaix@gmail.com>
>>
>>
>> Le mar. 9 août 2022 à 10:37, Guillaume Bres <guillaume.bressaix@gmail.com>
>> a écrit :
>>
>>> for information,
>>> pybind will soon become pkg-config knowledgeable, as this issue is about
>>> to get merged https://github.com/pybind/pybind11/pull/4077
>>>
>>> Guillaume W. Bres
>>> Software engineer
>>> <guillaume.bressaix@gmail.com>
>>>
>>>
>>> Le lun. 8 août 2022 à 23:13, Thomas Petazzoni <
>>> thomas.petazzoni@bootlin.com> a écrit :
>>>
>>>> On Mon, 8 Aug 2022 22:58:46 +0200
>>>> Arnout Vandecappelle <arnout@mind.be> wrote:
>>>>
>>>> >   pybind is consistently annoying, it uses ldd and stuff to discover
>>>> paths...
>>>>
>>>> Well, it's not just pybind. Python itself, in its ctype.find_library()
>>>> function, uses ldconfig, tries to use gcc on the target, and more funky
>>>> stuff that borks terribly in a Buildroot situation.
>>>>
>>>> Thomas
>>>> --
>>>> Thomas Petazzoni, co-owner and CEO, Bootlin
>>>> Embedded Linux and Kernel engineering and training
>>>> https://bootlin.com
>>>>
>>>

[-- Attachment #1.2: Type: text/html, Size: 7472 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation
  2022-08-05 14:36 [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Guillaume W. Bres
  2022-08-05 14:36 ` [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0 Guillaume W. Bres
  2022-08-06 22:13 ` [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Thomas Petazzoni via buildroot
@ 2023-06-26 21:34 ` James Hilliard
  2 siblings, 0 replies; 12+ messages in thread
From: James Hilliard @ 2023-06-26 21:34 UTC (permalink / raw)
  To: Guillaume W. Bres; +Cc: buildroot

On Fri, Aug 5, 2022 at 8:37 AM Guillaume W. Bres
<guillaume.bressaix@gmail.com> wrote:
>
> python-scipy is moving to a meson build infrastructure,
> and requires fortran cross-compilation

I have a patch migrating scipy to meson here:
https://patchwork.ozlabs.org/project/buildroot/patch/20230626181531.2312002-4-james.hilliard1@gmail.com/

>
> Signed-off-by: Guillaume W. Bres <guillaume.bressaix@gmail.com>
> ---
>  package/pkg-meson.mk                   | 2 ++
>  support/misc/cross-compilation.conf.in | 3 +++
>  2 files changed, 5 insertions(+)
>
> diff --git a/package/pkg-meson.mk b/package/pkg-meson.mk
> index 156b005fdd..32d41170f3 100644
> --- a/package/pkg-meson.mk
> +++ b/package/pkg-meson.mk
> @@ -76,10 +76,12 @@ define PKG_MESON_CROSSCONFIG_SED
>          -e "s%@TARGET_CC@%$(TARGET_CC)%g" \
>          -e "s%@TARGET_CXX@%$(TARGET_CXX)%g" \
>          -e "s%@TARGET_AR@%$(TARGET_AR)%g" \
> +        -e "s%@TARGET_FC@%$(TARGET_FC)%g" \
>          -e "s%@TARGET_STRIP@%$(TARGET_STRIP)%g" \
>          -e "s%@TARGET_ARCH@%$(PKG_MESON_TARGET_CPU_FAMILY)%g" \
>          -e "s%@TARGET_CPU@%$(GCC_TARGET_CPU)%g" \
>          -e "s%@TARGET_ENDIAN@%$(call qstrip,$(call LOWERCASE,$(BR2_ENDIAN)))%g" \
> +        -e "s%@TARGET_FCFLAGS@%$(call make-sq-comma-list,$($(strip $(2))))%g" \
>          -e "s%@TARGET_CFLAGS@%$(call make-sq-comma-list,$($(strip $(1))))%g" \
>          -e "s%@TARGET_LDFLAGS@%$(call make-sq-comma-list,$($(strip $(3))))%g" \
>          -e "s%@TARGET_CXXFLAGS@%$(call make-sq-comma-list,$($(strip $(2))))%g" \
> diff --git a/support/misc/cross-compilation.conf.in b/support/misc/cross-compilation.conf.in
> index 235e5a8495..7053df472b 100644
> --- a/support/misc/cross-compilation.conf.in
> +++ b/support/misc/cross-compilation.conf.in
> @@ -9,6 +9,7 @@ cpp = '@TARGET_CXX@'
>  ar = '@TARGET_AR@'
>  strip = '@TARGET_STRIP@'
>  cmake = '@BR2_CMAKE@'
> +fortran = '@TARGET_FC@'
>  pkgconfig = '@PKGCONF_HOST_BINARY@'
>  g-ir-compiler = '@STAGING_DIR@/usr/bin/g-ir-compiler'
>  g-ir-scanner = '@STAGING_DIR@/usr/bin/g-ir-scanner'
> @@ -18,6 +19,8 @@ c_args = [@TARGET_CFLAGS@]
>  c_link_args = [@TARGET_LDFLAGS@]
>  cpp_args = [@TARGET_CXXFLAGS@]
>  cpp_link_args = [@TARGET_LDFLAGS@]
> +fortran_args = [@TARGET_FCFLAGS@]
> +fortran_link_args = [@TARGET_LDFLAGS@]
>  wrap_mode = 'nodownload'
>  cmake_prefix_path = '@STAGING_DIR@/usr/lib/cmake'
>
> --
> 2.30.2
>
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2023-06-26 21:34 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-05 14:36 [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Guillaume W. Bres
2022-08-05 14:36 ` [Buildroot] [RFC v3 2/2] package/python-scipy: bump version to 1.9.0 Guillaume W. Bres
2022-08-05 14:46   ` Guillaume Bres
2022-08-08 20:58     ` Arnout Vandecappelle
2022-08-08 21:13       ` Thomas Petazzoni via buildroot
2022-08-09  8:37         ` Guillaume Bres
2022-08-27  9:11           ` Guillaume Bres
     [not found]             ` <CABL7CQjFbqaDd2ULBsPVyKM3cAEjCSHEdnY9vfxA=VMuRbdNjg@mail.gmail.com>
2022-09-01 19:43               ` Guillaume Bres
2022-08-06 22:13 ` [Buildroot] [RFC v3 1/2] package/pkg-meson.mk: support fortran cross-compilation Thomas Petazzoni via buildroot
2022-08-08 19:47   ` Yann E. MORIN
2022-08-09 16:25     ` Yann E. MORIN
2023-06-26 21:34 ` James Hilliard

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.