* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
@ 2019-10-23 21:19 Grzegorz Blach
2019-10-24 8:05 ` Thomas Petazzoni
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Grzegorz Blach @ 2019-10-23 21:19 UTC (permalink / raw)
To: buildroot
test/import-repeatedly.c uses an embedded python interpreter
and PYTHON_EXTRA_LIBS is used only for building this test case,
so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
to build this test case and don't link the module with libpython3.so.
Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
---
Changes v1 -> v2:
- Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
instead of appending --embed to PYTHON_LIBS
---
package/dbus-python/dbus-python.mk | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/package/dbus-python/dbus-python.mk b/package/dbus-python/dbus-python.mk
index eaf4e5f483..d48208f1d1 100644
--- a/package/dbus-python/dbus-python.mk
+++ b/package/dbus-python/dbus-python.mk
@@ -35,14 +35,16 @@ DBUS_PYTHON_DEPENDENCIES += python3 host-python3
DBUS_PYTHON_CONF_ENV += \
PYTHON=$(HOST_DIR)/bin/python3 \
PYTHON_INCLUDES="`$(STAGING_DIR)/usr/bin/python3-config --includes`" \
- PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`"
+ PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`" \
+ PYTHON_EXTRA_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --libs --embed`"
HOST_DBUS_PYTHON_DEPENDENCIES += host-python3
HOST_DBUS_PYTHON_CONF_ENV += \
PYTHON=$(HOST_DIR)/bin/python3 \
PYTHON_INCLUDES="`$(HOST_DIR)/usr/bin/python3-config --includes`" \
- PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`"
+ PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`" \
+ PYTHON_EXTRA_LIBS="`$(HOST_DIR)/usr/bin/python3-config --libs --embed`"
endif
$(eval $(autotools-package))
--
2.23.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach
@ 2019-10-24 8:05 ` Thomas Petazzoni
2019-10-24 8:06 ` Thomas Petazzoni
2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach
2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni
2 siblings, 1 reply; 6+ messages in thread
From: Thomas Petazzoni @ 2019-10-24 8:05 UTC (permalink / raw)
To: buildroot
On Wed, 23 Oct 2019 21:19:21 +0000
Grzegorz Blach <grzegorz@blach.pl> wrote:
> test/import-repeatedly.c uses an embedded python interpreter
> and PYTHON_EXTRA_LIBS is used only for building this test case,
> so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
> to build this test case and don't link the module with libpython3.so.
>
> Fixes:
> - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
> - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
> and many similar failures.
>
> Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
> ---
> Changes v1 -> v2:
> - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
> instead of appending --embed to PYTHON_LIBS
This doesn't change the fix, which I think still isn't correct (but I
haven't checked in details). A Python extension should not need the
--embed option.
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
2019-10-24 8:05 ` Thomas Petazzoni
@ 2019-10-24 8:06 ` Thomas Petazzoni
2019-10-24 8:43 ` Arnout Vandecappelle
0 siblings, 1 reply; 6+ messages in thread
From: Thomas Petazzoni @ 2019-10-24 8:06 UTC (permalink / raw)
To: buildroot
On Thu, 24 Oct 2019 10:05:34 +0200
Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
> On Wed, 23 Oct 2019 21:19:21 +0000
> Grzegorz Blach <grzegorz@blach.pl> wrote:
>
> > test/import-repeatedly.c uses an embedded python interpreter
> > and PYTHON_EXTRA_LIBS is used only for building this test case,
> > so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
> > to build this test case and don't link the module with libpython3.so.
> >
> > Fixes:
> > - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
> > - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
> > and many similar failures.
> >
> > Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
> > ---
> > Changes v1 -> v2:
> > - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
> > instead of appending --embed to PYTHON_LIBS
>
> This doesn't change the fix, which I think still isn't correct (but I
> haven't checked in details). A Python extension should not need the
> --embed option.
And I replied too fast, without reading your commit log. Reading it, it
makes more sense. However, I think it should be fixed in the
dbus-python build system itself, so that the fix can be upstreamed.
Could you have a look at that ?
Of course, it's not trivial, because you need to pass --embed for
Python >= 3.8, but not for older versions of Python.
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
2019-10-24 8:06 ` Thomas Petazzoni
@ 2019-10-24 8:43 ` Arnout Vandecappelle
0 siblings, 0 replies; 6+ messages in thread
From: Arnout Vandecappelle @ 2019-10-24 8:43 UTC (permalink / raw)
To: buildroot
On 24/10/2019 10:06, Thomas Petazzoni wrote:
> On Thu, 24 Oct 2019 10:05:34 +0200
> Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote:
>
>> On Wed, 23 Oct 2019 21:19:21 +0000
>> Grzegorz Blach <grzegorz@blach.pl> wrote:
>>
>>> test/import-repeatedly.c uses an embedded python interpreter
>>> and PYTHON_EXTRA_LIBS is used only for building this test case,
>>> so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
>>> to build this test case and don't link the module with libpython3.so.
>>>
>>> Fixes:
>>> - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
>>> - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
>>> and many similar failures.
>>>
>>> Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
>>> ---
>>> Changes v1 -> v2:
>>> - Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
>>> instead of appending --embed to PYTHON_LIBS
>>
>> This doesn't change the fix, which I think still isn't correct (but I
>> haven't checked in details). A Python extension should not need the
>> --embed option.
>
> And I replied too fast, without reading your commit log. Reading it, it
> makes more sense. However, I think it should be fixed in the
> dbus-python build system itself, so that the fix can be upstreamed.
> Could you have a look at that ?
>
> Of course, it's not trivial, because you need to pass --embed for
> Python >= 3.8, but not for older versions of Python.
Actually, it looks like this should be retrieved from sysconfig. So maybe it's
just a matter of setting the proper environment variables like we do in
pkg-python? In other words, I think the entire logic in dbus-python.mk could be
replaced with:
DBUS_PYTHON_CONF_ENV = $(PKG_PYTHON_SETUPTOOLS_ENV)
HOST_DBUS_PYTHON_CONF_ENV = $(HOST_PKG_PYTHON_SETUPTOOLS_ENV)
We may still need to set PYTHON=$(HOST_DIR)/bin/python[23] as well.
Regards,
Arnout
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH v3] package/dbus-python: Fix build with python 3.8
2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach
2019-10-24 8:05 ` Thomas Petazzoni
@ 2019-10-24 17:27 ` Grzegorz Blach
2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni
2 siblings, 0 replies; 6+ messages in thread
From: Grzegorz Blach @ 2019-10-24 17:27 UTC (permalink / raw)
To: buildroot
Python 3.8 doesn't allow python modules to be linked with
libpythonX.Y.so, but applications that embed a python interpreter
still need to be linked with libpythonX.Y.so. As a solution
-lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS,
but only for python 3.8 or newer.
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
---
Changes v2 -> v3:
- Patch m4/ax_python_devel.m4 instead of setting PYTHON_EXTRA_LIBS
Changes v1 -> v2:
- Set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
instead of appending --embed to PYTHON_LIBS
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
---
.../0001-Fix-build-with-python-3.8.patch | 95 +++++++++++++++++++
package/dbus-python/dbus-python.mk | 4 +
2 files changed, 99 insertions(+)
create mode 100644 package/dbus-python/0001-Fix-build-with-python-3.8.patch
diff --git a/package/dbus-python/0001-Fix-build-with-python-3.8.patch b/package/dbus-python/0001-Fix-build-with-python-3.8.patch
new file mode 100644
index 0000000000..db4a0d0b10
--- /dev/null
+++ b/package/dbus-python/0001-Fix-build-with-python-3.8.patch
@@ -0,0 +1,95 @@
+From 1d7695681f1db3f6e3b487abbffc9665f1422bf4 Mon Sep 17 00:00:00 2001
+From: Grzegorz Blach <grzegorz@blach.pl>
+Date: Thu, 24 Oct 2019 15:42:55 +0000
+Subject: [PATCH] Fix build with python 3.8
+
+Python 3.8 doesn't allow python modules to be linked with
+libpythonX.Y.so, but applications that embed a python interpreter
+still need to be linked with libpythonX.Y.so. As a solution
+-lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS,
+but only for python 3.8 or newer.
+
+Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
+---
+ m4/ax_python_devel.m4 | 36 +++++++++++++++++++++++-------------
+ 1 file changed, 23 insertions(+), 13 deletions(-)
+
+diff --git a/m4/ax_python_devel.m4 b/m4/ax_python_devel.m4
+index 44dbd83..0de0c6e 100644
+--- a/m4/ax_python_devel.m4
++++ b/m4/ax_python_devel.m4
+@@ -40,6 +40,7 @@
+ # Copyright (c) 2009 Matteo Settenvini <matteo@member.fsf.org>
+ # Copyright (c) 2009 Horst Knorr <hk_classes@knoda.org>
+ # Copyright (c) 2013 Daniel Mullner <muellner@math.stanford.edu>
++# Copyright (c) 2019 Grzegorz Blach <grzegorz@blach.pl>
+ #
+ # This program is free software: you can redistribute it and/or modify it
+ # under the terms of the GNU General Public License as published by the
+@@ -67,7 +68,7 @@
+ # modified version of the Autoconf Macro, you may extend this special
+ # exception to the GPL to apply to your modified version as well.
+
+-#serial 21
++#serial 21 + python 3.8 fixes
+
+ AU_ALIAS([AC_PYTHON_DEVEL], [AX_PYTHON_DEVEL])
+ AC_DEFUN([AX_PYTHON_DEVEL],[
+@@ -198,6 +199,11 @@ EOD`
+ AC_DEFINE_UNQUOTED([HAVE_PYTHON], ["$ac_python_version"],
+ [If available, contains the Python version number currently in use.])
+
++ # Check if we should omit -lpython in PYTHON_LIBS
++ ac_omit_python_library=`$PYTHON -c "import sys; \
++ ver = sys.version.split ()[[0]]; \
++ print (ver >= '3.8.0')"`
++
+ # First, the library directory:
+ ac_python_libdir=`cat<<EOD | $PYTHON -
+
+@@ -223,18 +229,22 @@ EOD`
+ # credits goes to momjian, I think. I'd like to put the right name
+ # in the credits, if someone can point me in the right direction... ?
+ #
+- if test -n "$ac_python_libdir" -a -n "$ac_python_library"
+- then
+- # use the official shared library
+- ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"`
+- PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library"
++ if test "$ac_omit_python_library" = "True"; then
++ PYTHON_LIBS="-L$ac_python_libdir"
+ else
+- # old way: use libpython from python_configdir
+- ac_python_libdir=`$PYTHON -c \
+- "from distutils.sysconfig import get_python_lib as f; \
+- import os; \
+- print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"`
+- PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version"
++ if test -n "$ac_python_libdir" -a -n "$ac_python_library"
++ then
++ # use the official shared library
++ ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"`
++ PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library"
++ else
++ # old way: use libpython from python_configdir
++ ac_python_libdir=`$PYTHON -c \
++ "from distutils.sysconfig import get_python_lib as f; \
++ import os; \
++ print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"`
++ PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version"
++ fi
+ fi
+
+ if test -z "PYTHON_LIBS"; then
+@@ -265,7 +275,7 @@ EOD`
+ if test -z "$PYTHON_EXTRA_LIBS"; then
+ PYTHON_EXTRA_LIBS=`$PYTHON -c "import distutils.sysconfig; \
+ conf = distutils.sysconfig.get_config_var; \
+- print (conf('LIBS') + ' ' + conf('SYSLIBS'))"`
++ print ('-lpython' + (conf('LDVERSION') or conf('VERSION')) + ' ' + conf('LIBS') + ' ' + conf('SYSLIBS'))"`
+ fi
+ AC_MSG_RESULT([$PYTHON_EXTRA_LIBS])
+ AC_SUBST(PYTHON_EXTRA_LIBS)
+--
+2.23.0
+
diff --git a/package/dbus-python/dbus-python.mk b/package/dbus-python/dbus-python.mk
index eaf4e5f483..201c7130ec 100644
--- a/package/dbus-python/dbus-python.mk
+++ b/package/dbus-python/dbus-python.mk
@@ -15,6 +15,10 @@ DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs
HOST_DBUS_PYTHON_DEPENDENCIES = host-dbus-glib
HOST_DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs
+# We patch m4/ax_python_devel.m4
+DBUS_PYTHON_AUTORECONF = YES
+HOST_DBUS_PYTHON_AUTORECONF = YES
+
ifeq ($(BR2_PACKAGE_PYTHON),y)
DBUS_PYTHON_DEPENDENCIES += python host-python
--
2.23.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8
2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach
2019-10-24 8:05 ` Thomas Petazzoni
2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach
@ 2019-10-25 16:22 ` Thomas Petazzoni
2 siblings, 0 replies; 6+ messages in thread
From: Thomas Petazzoni @ 2019-10-25 16:22 UTC (permalink / raw)
To: buildroot
On Wed, 23 Oct 2019 21:19:21 +0000
Grzegorz Blach <grzegorz@blach.pl> wrote:
> test/import-repeatedly.c uses an embedded python interpreter
> and PYTHON_EXTRA_LIBS is used only for building this test case,
> so set PYTHON_EXTRA_LIBS with `python3-config --libs --embed`
> to build this test case and don't link the module with libpython3.so.
>
> Fixes:
> - http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
> - http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
> and many similar failures.
>
> Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
Guess what, in the end, after investigating, I agree that this patch
was the right solution! PYTHON_LIBS is used when building Python
extensions and PYTHON_EXTRA_LIBS is used when building code that embeds
a Python interpreter.
So this patch made perfect sense. Of course, it would be *much* better
for ax_python_devel.m4 to use python-config, rather than do its
convoluted logic, but your patch is simple is easy, so I've applied.
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-10-25 16:22 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-23 21:19 [Buildroot] [PATCH] package/dbus-python: Fix build with python 3.8 Grzegorz Blach
2019-10-24 8:05 ` Thomas Petazzoni
2019-10-24 8:06 ` Thomas Petazzoni
2019-10-24 8:43 ` Arnout Vandecappelle
2019-10-24 17:27 ` [Buildroot] [PATCH v3] " Grzegorz Blach
2019-10-25 16:22 ` [Buildroot] [PATCH] " Thomas Petazzoni
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.