buildroot.busybox.net archive mirror
 help / color / mirror / Atom feed
* [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0
@ 2023-06-26  7:05 Gwenhael Goavec-Merou
  2023-06-26  7:05 ` [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile Gwenhael Goavec-Merou
  2023-07-04 20:47 ` [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Arnout Vandecappelle via buildroot
  0 siblings, 2 replies; 9+ messages in thread
From: Gwenhael Goavec-Merou @ 2023-06-26  7:05 UTC (permalink / raw)
  To: buildroot; +Cc: Gwenhael Goavec-Merou

From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>

Since commit 0e82c360942907f5a2f379e64e0d211aaff80774 (and since release
2.10.1), a new variable called PYBIND11_PYTHONLIBS_OVERWRITE was introduced
and set to ON by default.

According to comment before this option:
"Overwrite cached values read from Python library (classic search). Turn off if
cross-compiling and manually setting these values."

In buildroot's context this option must be disabled to keep variables
provided by buildroot and lib/app.

This issue affect branches: master, next and tags 2023.02.x, 2023.05.

Fix:
- http://autobuild.buildroot.net/results/7423df5db5237d94bb49e32698828d4fe470e39b/

Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
---
Changes v1 -> v2:
- move PYBIND11_PYTHONLIBS_OVERWRITE option to python support section
---
 package/gnuradio/gnuradio.mk | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
index d9218b533d..1453b78493 100644
--- a/package/gnuradio/gnuradio.mk
+++ b/package/gnuradio/gnuradio.mk
@@ -116,6 +116,9 @@ ifeq ($(BR2_PACKAGE_GNURADIO_PYTHON),y)
 GNURADIO_DEPENDENCIES += python3 python-pybind \
 	host-python-numpy host-python-packaging
 GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
+# mandatory to avoid pybind11 to overwrite variables provided
+# by gnuradio and buildroot
+GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
 # mandatory to install python modules in site-packages and to use
 # correct path for python libraries
 GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-06-26  7:05 [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Gwenhael Goavec-Merou
@ 2023-06-26  7:05 ` Gwenhael Goavec-Merou
  2023-07-04 20:51   ` Arnout Vandecappelle via buildroot
  2023-07-04 20:47 ` [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Arnout Vandecappelle via buildroot
  1 sibling, 1 reply; 9+ messages in thread
From: Gwenhael Goavec-Merou @ 2023-06-26  7:05 UTC (permalink / raw)
  To: buildroot; +Cc: Gwenhael Goavec-Merou

From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>

By default, module libraries have a suffix based on cpython version + host
architecture: this is fine for a native compile when these libraries are used on
the same computer (or similar computers). But when target architecture is not
the same python is unable to find libraries due to the wrong suffix and produces
unclear errors messages:

# python3
Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from gnuradio import blocks  
Traceback (most recent call last):
  File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
>>>

By adding -DPYTHON_MODULE_EXTENSION=".so" at configure time, pybind11 will use
this option instead of using host-python to forge suffix.

Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
---
Changes v1 -> v2:
- replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
  but less "noisy")
---
 package/gnuradio/gnuradio.mk | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
index 1453b78493..d7b3b86c31 100644
--- a/package/gnuradio/gnuradio.mk
+++ b/package/gnuradio/gnuradio.mk
@@ -119,6 +119,8 @@ GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
 # mandatory to avoid pybind11 to overwrite variables provided
 # by gnuradio and buildroot
 GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
+# force libraries suffix (avoid to have libxxx.PYTHONVER-HOST_ARCH.so)
+GNURADIO_CONF_OPTS += -DPYTHON_MODULE_EXTENSION=".so"
 # mandatory to install python modules in site-packages and to use
 # correct path for python libraries
 GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \
-- 
2.40.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0
  2023-06-26  7:05 [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Gwenhael Goavec-Merou
  2023-06-26  7:05 ` [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile Gwenhael Goavec-Merou
@ 2023-07-04 20:47 ` Arnout Vandecappelle via buildroot
  2023-07-07  8:40   ` Gwenhael Goavec-Merou
  1 sibling, 1 reply; 9+ messages in thread
From: Arnout Vandecappelle via buildroot @ 2023-07-04 20:47 UTC (permalink / raw)
  To: Gwenhael Goavec-Merou, buildroot; +Cc: Gwenhael Goavec-Merou



On 26/06/2023 09:05, Gwenhael Goavec-Merou wrote:
> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> 
> Since commit 0e82c360942907f5a2f379e64e0d211aaff80774 (and since release
> 2.10.1), a new variable called PYBIND11_PYTHONLIBS_OVERWRITE was introduced
> and set to ON by default.
> 
> According to comment before this option:
> "Overwrite cached values read from Python library (classic search). Turn off if
> cross-compiling and manually setting these values."
> 
> In buildroot's context this option must be disabled to keep variables
> provided by buildroot and lib/app.

  I guess something similar should be done for all users of pybind. And indeed, 
I see that zxing-cpp suffers from the same issue [1]. For some reason 
python-scipy doesn't have autobuild failures though.

> 
> This issue affect branches: master, next and tags 2023.02.x, 2023.05.

  This kind of comment should not be in the commit message itself, but rather 
under the --- line.

> 
> Fix:

  And this should be Fixes: (so it gets picked up by patchwork)

> - http://autobuild.buildroot.net/results/7423df5db5237d94bb49e32698828d4fe470e39b/
> 
> Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>

  Applied to master, thanks.

  Regards,
  Arnout

[1] 
http://autobuild.buildroot.net/results/7dc/7dc04cb67280073d24dc550e5ed8aa3673a9c6b1//


> ---
> Changes v1 -> v2:
> - move PYBIND11_PYTHONLIBS_OVERWRITE option to python support section
> ---
>   package/gnuradio/gnuradio.mk | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
> index d9218b533d..1453b78493 100644
> --- a/package/gnuradio/gnuradio.mk
> +++ b/package/gnuradio/gnuradio.mk
> @@ -116,6 +116,9 @@ ifeq ($(BR2_PACKAGE_GNURADIO_PYTHON),y)
>   GNURADIO_DEPENDENCIES += python3 python-pybind \
>   	host-python-numpy host-python-packaging
>   GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
> +# mandatory to avoid pybind11 to overwrite variables provided
> +# by gnuradio and buildroot
> +GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
>   # mandatory to install python modules in site-packages and to use
>   # correct path for python libraries
>   GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-06-26  7:05 ` [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile Gwenhael Goavec-Merou
@ 2023-07-04 20:51   ` Arnout Vandecappelle via buildroot
  2023-07-07  8:58     ` Gwenhael Goavec-Merou
  0 siblings, 1 reply; 9+ messages in thread
From: Arnout Vandecappelle via buildroot @ 2023-07-04 20:51 UTC (permalink / raw)
  To: Gwenhael Goavec-Merou, buildroot; +Cc: Gwenhael Goavec-Merou



On 26/06/2023 09:05, Gwenhael Goavec-Merou wrote:
> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> 
> By default, module libraries have a suffix based on cpython version + host
> architecture: this is fine for a native compile when these libraries are used on

  I think that this is solved in the python infrastructure by setting

_PYTHON_HOST_PLATFORM="$(PKG_PYTHON_HOST_PLATFORM)"

Or it may be some other environment variable that is set. For sure though, the 
same problem would exist in normal python packages, and it is resolved there. So 
ideally we should use the same solution for gnuradio.

  Regards,
  Arnout

> the same computer (or similar computers). But when target architecture is not
> the same python is unable to find libraries due to the wrong suffix and produces
> unclear errors messages:
> 
> # python3
> Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from gnuradio import blocks
> Traceback (most recent call last):
>    File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 18, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py", line 22, in <module>
> ModuleNotFoundError: No module named 'gnuradio.blocks.blocks_python'
>>>>
> 
> By adding -DPYTHON_MODULE_EXTENSION=".so" at configure time, pybind11 will use
> this option instead of using host-python to forge suffix.
> 
> Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> ---
> Changes v1 -> v2:
> - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
>    but less "noisy")
> ---
>   package/gnuradio/gnuradio.mk | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
> index 1453b78493..d7b3b86c31 100644
> --- a/package/gnuradio/gnuradio.mk
> +++ b/package/gnuradio/gnuradio.mk
> @@ -119,6 +119,8 @@ GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
>   # mandatory to avoid pybind11 to overwrite variables provided
>   # by gnuradio and buildroot
>   GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
> +# force libraries suffix (avoid to have libxxx.PYTHONVER-HOST_ARCH.so)
> +GNURADIO_CONF_OPTS += -DPYTHON_MODULE_EXTENSION=".so"
>   # mandatory to install python modules in site-packages and to use
>   # correct path for python libraries
>   GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0
  2023-07-04 20:47 ` [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Arnout Vandecappelle via buildroot
@ 2023-07-07  8:40   ` Gwenhael Goavec-Merou
  0 siblings, 0 replies; 9+ messages in thread
From: Gwenhael Goavec-Merou @ 2023-07-07  8:40 UTC (permalink / raw)
  To: Arnout Vandecappelle; +Cc: Gwenhael Goavec-Merou, buildroot

Arnout, all
On Tue, 4 Jul 2023 22:47:24 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 26/06/2023 09:05, Gwenhael Goavec-Merou wrote:
> > From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> > 
> > Since commit 0e82c360942907f5a2f379e64e0d211aaff80774 (and since release
> > 2.10.1), a new variable called PYBIND11_PYTHONLIBS_OVERWRITE was introduced
> > and set to ON by default.
> > 
> > According to comment before this option:
> > "Overwrite cached values read from Python library (classic search). Turn
> > off if cross-compiling and manually setting these values."
> > 
> > In buildroot's context this option must be disabled to keep variables
> > provided by buildroot and lib/app.  
> 
>   I guess something similar should be done for all users of pybind. And
> indeed, I see that zxing-cpp suffers from the same issue [1]. For some reason 
> python-scipy doesn't have autobuild failures though.
> 
Yes. But the question is about way:
- adding the same fix for all current (and futur) package use pybind
- using sed at pybind's buildtime to change ON by OFF?
> > 
> > This issue affect branches: master, next and tags 2023.02.x, 2023.05.  
> 
>   This kind of comment should not be in the commit message itself, but rather 
> under the --- line.
> 
> > 
> > Fix:  
> 
>   And this should be Fixes: (so it gets picked up by patchwork)
> 
> > -
> > http://autobuild.buildroot.net/results/7423df5db5237d94bb49e32698828d4fe470e39b/
> > 
> > Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> >  
> 
>   Applied to master, thanks.

Thanls;
> 
>   Regards,
>   Arnout
> 
Regards,
Gwenhael
> [1] 
> http://autobuild.buildroot.net/results/7dc/7dc04cb67280073d24dc550e5ed8aa3673a9c6b1//
> 
> 
> > ---
> > Changes v1 -> v2:
> > - move PYBIND11_PYTHONLIBS_OVERWRITE option to python support section
> > ---
> >   package/gnuradio/gnuradio.mk | 3 +++
> >   1 file changed, 3 insertions(+)
> > 
> > diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
> > index d9218b533d..1453b78493 100644
> > --- a/package/gnuradio/gnuradio.mk
> > +++ b/package/gnuradio/gnuradio.mk
> > @@ -116,6 +116,9 @@ ifeq ($(BR2_PACKAGE_GNURADIO_PYTHON),y)
> >   GNURADIO_DEPENDENCIES += python3 python-pybind \
> >   	host-python-numpy host-python-packaging
> >   GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
> > +# mandatory to avoid pybind11 to overwrite variables provided
> > +# by gnuradio and buildroot
> > +GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
> >   # mandatory to install python modules in site-packages and to use
> >   # correct path for python libraries
> >   GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \  
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-07-04 20:51   ` Arnout Vandecappelle via buildroot
@ 2023-07-07  8:58     ` Gwenhael Goavec-Merou
  2023-07-07  9:44       ` Arnout Vandecappelle via buildroot
  0 siblings, 1 reply; 9+ messages in thread
From: Gwenhael Goavec-Merou @ 2023-07-07  8:58 UTC (permalink / raw)
  To: Arnout Vandecappelle; +Cc: Gwenhael Goavec-Merou, buildroot

Arnout, all

On Tue, 4 Jul 2023 22:51:57 +0200
Arnout Vandecappelle <arnout@mind.be> wrote:

> On 26/06/2023 09:05, Gwenhael Goavec-Merou wrote:
> > From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> > 
> > By default, module libraries have a suffix based on cpython version + host
> > architecture: this is fine for a native compile when these libraries are
> > used on  
> 
>   I think that this is solved in the python infrastructure by setting
> 
> _PYTHON_HOST_PLATFORM="$(PKG_PYTHON_HOST_PLATFORM)"
> 
> Or it may be some other environment variable that is set. For sure though,
> the same problem would exist in normal python packages, and it is resolved
> there. So ideally we should use the same solution for gnuradio.
> 
In fact I see I have missed to explain in my message this issue is related to
pybind.
If PYTHON_MODULE_EXTENSION is unset, .cmake uses a python script containing
sysconfig.get_config_var('EXT_SUFFIX')
This one return .cpython-311-x86_64-linux-gnu.so, because host-python is used.
>   Regards,
>   Arnout
> 
Regards,
Gwenhael

> > the same computer (or similar computers). But when target architecture is
> > not the same python is unable to find libraries due to the wrong suffix and
> > produces unclear errors messages:
> > 
> > # python3
> > Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
> > Type "help", "copyright", "credits" or "license" for more information.  
> >>>> from gnuradio import blocks  
> > Traceback (most recent call last):
> >    File
> > "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py",
> > line 18, in <module> ModuleNotFoundError: No module named
> > 'gnuradio.blocks.blocks_python'
> > 
> > During handling of the above exception, another exception occurred:
> > 
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in <module>
> >    File
> > "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py",
> > line 22, in <module> ModuleNotFoundError: No module named
> > 'gnuradio.blocks.blocks_python'  
> >>>>  
> > 
> > By adding -DPYTHON_MODULE_EXTENSION=".so" at configure time, pybind11 will
> > use this option instead of using host-python to forge suffix.
> > 
> > Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
> > ---
> > Changes v1 -> v2:
> > - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
> >    but less "noisy")
> > ---
> >   package/gnuradio/gnuradio.mk | 2 ++
> >   1 file changed, 2 insertions(+)
> > 
> > diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
> > index 1453b78493..d7b3b86c31 100644
> > --- a/package/gnuradio/gnuradio.mk
> > +++ b/package/gnuradio/gnuradio.mk
> > @@ -119,6 +119,8 @@ GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
> >   # mandatory to avoid pybind11 to overwrite variables provided
> >   # by gnuradio and buildroot
> >   GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
> > +# force libraries suffix (avoid to have libxxx.PYTHONVER-HOST_ARCH.so)
> > +GNURADIO_CONF_OPTS += -DPYTHON_MODULE_EXTENSION=".so"
> >   # mandatory to install python modules in site-packages and to use
> >   # correct path for python libraries
> >   GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \  
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-07-07  8:58     ` Gwenhael Goavec-Merou
@ 2023-07-07  9:44       ` Arnout Vandecappelle via buildroot
  2023-07-10 18:25         ` Thomas Petazzoni via buildroot
  0 siblings, 1 reply; 9+ messages in thread
From: Arnout Vandecappelle via buildroot @ 2023-07-07  9:44 UTC (permalink / raw)
  To: Gwenhael Goavec-Merou; +Cc: Gwenhael Goavec-Merou, buildroot



On 07/07/2023 10:58, Gwenhael Goavec-Merou wrote:
> Arnout, all
> 
> On Tue, 4 Jul 2023 22:51:57 +0200
> Arnout Vandecappelle <arnout@mind.be> wrote:
> 
>> On 26/06/2023 09:05, Gwenhael Goavec-Merou wrote:
>>> From: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
>>>
>>> By default, module libraries have a suffix based on cpython version + host
>>> architecture: this is fine for a native compile when these libraries are
>>> used on
>>
>>    I think that this is solved in the python infrastructure by setting
>>
>> _PYTHON_HOST_PLATFORM="$(PKG_PYTHON_HOST_PLATFORM)"
>>
>> Or it may be some other environment variable that is set. For sure though,
>> the same problem would exist in normal python packages, and it is resolved
>> there. So ideally we should use the same solution for gnuradio.
>>
> In fact I see I have missed to explain in my message this issue is related to
> pybind.
> If PYTHON_MODULE_EXTENSION is unset, .cmake uses a python script containing
> sysconfig.get_config_var('EXT_SUFFIX')
> This one return .cpython-311-x86_64-linux-gnu.so, because host-python is used.

  Ah, in that case the environment variable you need is 
_PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)"
This will make sure the sysconfig that gets used is the one for the target 
instead of for the host.

  It's quite likely that you need all of PKG_PYTHON_ENV to be present in the 
environment.

  Regards,
  Arnout

>>    Regards,
>>    Arnout
>>
> Regards,
> Gwenhael
> 
>>> the same computer (or similar computers). But when target architecture is
>>> not the same python is unable to find libraries due to the wrong suffix and
>>> produces unclear errors messages:
>>>
>>> # python3
>>> Python 3.11.3 (main, Jun 19 2023, 14:15:44) [GCC 11.4.0] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> from gnuradio import blocks
>>> Traceback (most recent call last):
>>>     File
>>> "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py",
>>> line 18, in <module> ModuleNotFoundError: No module named
>>> 'gnuradio.blocks.blocks_python'
>>>
>>> During handling of the above exception, another exception occurred:
>>>
>>> Traceback (most recent call last):
>>>     File "<stdin>", line 1, in <module>
>>>     File
>>> "/home/xxx/buildroot/output/build/gnuradio-3.10.4.0/gr-blocks/python/blocks/__init__.py",
>>> line 22, in <module> ModuleNotFoundError: No module named
>>> 'gnuradio.blocks.blocks_python'
>>>>>>   
>>>
>>> By adding -DPYTHON_MODULE_EXTENSION=".so" at configure time, pybind11 will
>>> use this option instead of using host-python to forge suffix.
>>>
>>> Signed-off-by: Gwenhael Goavec-Merou <gwenhael.goavec-merou@trabucayre.com>
>>> ---
>>> Changes v1 -> v2:
>>> - replaces patch by PYTHON_MODULE_EXTENSION option (result is the same
>>>     but less "noisy")
>>> ---
>>>    package/gnuradio/gnuradio.mk | 2 ++
>>>    1 file changed, 2 insertions(+)
>>>
>>> diff --git a/package/gnuradio/gnuradio.mk b/package/gnuradio/gnuradio.mk
>>> index 1453b78493..d7b3b86c31 100644
>>> --- a/package/gnuradio/gnuradio.mk
>>> +++ b/package/gnuradio/gnuradio.mk
>>> @@ -119,6 +119,8 @@ GNURADIO_CONF_OPTS += -DENABLE_PYTHON=ON
>>>    # mandatory to avoid pybind11 to overwrite variables provided
>>>    # by gnuradio and buildroot
>>>    GNURADIO_CONF_OPTS += -DPYBIND11_PYTHONLIBS_OVERWRITE=OFF
>>> +# force libraries suffix (avoid to have libxxx.PYTHONVER-HOST_ARCH.so)
>>> +GNURADIO_CONF_OPTS += -DPYTHON_MODULE_EXTENSION=".so"
>>>    # mandatory to install python modules in site-packages and to use
>>>    # correct path for python libraries
>>>    GNURADIO_CONF_OPTS += -DGR_PYTHON_RELATIVE=ON \
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-07-07  9:44       ` Arnout Vandecappelle via buildroot
@ 2023-07-10 18:25         ` Thomas Petazzoni via buildroot
  2023-07-17 15:17           ` Gwenhael Goavec-Merou
  0 siblings, 1 reply; 9+ messages in thread
From: Thomas Petazzoni via buildroot @ 2023-07-10 18:25 UTC (permalink / raw)
  To: Arnout Vandecappelle via buildroot; +Cc: Gwenhael Goavec-Merou

Hello Gwenhael,

On Fri, 7 Jul 2023 11:44:21 +0200
Arnout Vandecappelle via buildroot <buildroot@buildroot.org> wrote:

> > In fact I see I have missed to explain in my message this issue is related to
> > pybind.
> > If PYTHON_MODULE_EXTENSION is unset, .cmake uses a python script containing
> > sysconfig.get_config_var('EXT_SUFFIX')
> > This one return .cpython-311-x86_64-linux-gnu.so, because host-python is used.  
> 
>   Ah, in that case the environment variable you need is 
> _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)"
> This will make sure the sysconfig that gets used is the one for the target 
> instead of for the host.
> 
>   It's quite likely that you need all of PKG_PYTHON_ENV to be present in the 
> environment.

Have you had the chance to test this suggestion from Arnout?

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile
  2023-07-10 18:25         ` Thomas Petazzoni via buildroot
@ 2023-07-17 15:17           ` Gwenhael Goavec-Merou
  0 siblings, 0 replies; 9+ messages in thread
From: Gwenhael Goavec-Merou @ 2023-07-17 15:17 UTC (permalink / raw)
  To: Thomas Petazzoni
  Cc: Gwenhael Goavec-Merou, Arnout Vandecappelle via buildroot

Thomas, Arnout, all
Sorry for delay, was away for a correct/stable connection.

On Mon, 10 Jul 2023 20:25:03 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:

> Hello Gwenhael,
> 
> On Fri, 7 Jul 2023 11:44:21 +0200
> Arnout Vandecappelle via buildroot <buildroot@buildroot.org> wrote:
> 
> > > In fact I see I have missed to explain in my message this issue is
> > > related to pybind.
> > > If PYTHON_MODULE_EXTENSION is unset, .cmake uses a python script
> > > containing sysconfig.get_config_var('EXT_SUFFIX')
> > > This one return .cpython-311-x86_64-linux-gnu.so, because host-python is
> > > used.    
> > 
> >   Ah, in that case the environment variable you need is 
> > _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)"
> > This will make sure the sysconfig that gets used is the one for the target 
> > instead of for the host.
> > 
> >   It's quite likely that you need all of PKG_PYTHON_ENV to be present in
> > the environment.  
> 
> Have you had the chance to test this suggestion from Arnout?
> 
Adding:
GNURADIO_CONF_ENV += _PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" \
    PYTHONPATH=$(PYTHON3_PATH)
Do the job.
Thanks. I have to redo my patch
> Thomas

Gwen
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2023-07-17 15:18 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-06-26  7:05 [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Gwenhael Goavec-Merou
2023-06-26  7:05 ` [Buildroot] [PATCH v2 2/2] package/gnuradio: fix gnuradio python libraries for cross-compile Gwenhael Goavec-Merou
2023-07-04 20:51   ` Arnout Vandecappelle via buildroot
2023-07-07  8:58     ` Gwenhael Goavec-Merou
2023-07-07  9:44       ` Arnout Vandecappelle via buildroot
2023-07-10 18:25         ` Thomas Petazzoni via buildroot
2023-07-17 15:17           ` Gwenhael Goavec-Merou
2023-07-04 20:47 ` [Buildroot] [PATCH v2 1/2] package/gnuradio: fix build with python-pybind > 2.10.0 Arnout Vandecappelle via buildroot
2023-07-07  8:40   ` Gwenhael Goavec-Merou

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).