[Buildroot] [PATCH v3 1/2] dependencies: build a host python2 if no suitable one cane be found

[Buildroot] [PATCH v3 1/2] dependencies: build a host python2 if no suitable one cane be found

[Samuel Martin]

Some distros choose to change the /usr/bin/python binary, make it pointed to
python3 instead of python2.

This may have some bad consequences for packages that uses some
non-python3-compliant python scripts in their build system (eg. in install or
post-install scripts).

This patch checks for a suitable python2 version (2.6 or 2.7) on the host
system, and declares the following variables:
- PYTHON2: pointing to the host python2 binary;
- NEED_PYTHON2: sets to "host-python" if no python2 binary has been found.

This way, a package using some python2 scripts must:
- adds $(NEED_PYTHON2) to its dependency list;
- sets $(PYTHON2) as the python binary to be used.

A side effect of this patch is getting rid of any host python. Buildroot can
runs on a host without python, or with a too old python2 version, or with only
python3.

Changes since v2:
- misc. fixes and cleanup

Changes since v1:
- use support/dependency infrastructure
- rename some symbols

Signed-off-by: Samuel Martin <s.martin49@gmail.com>

 create mode 100644 support/dependencies/check-host-python2.mk
 create mode 100755 support/dependencies/check-host-python2.sh

diff --git a/support/dependencies/check-host-python2.mk b/support/dependencies/check-host-python2.mk
new file mode 100644
index 0000000..3b66ffc
--- /dev/null
+++ b/support/dependencies/check-host-python2.mk
@@ -0,0 +1,6 @@
+PYTHON2 = $(call suitable-host-package,python2)
+
+ifeq (,$(PYTHON2))
+  NEED_PYTHON2 = host-python
+  PYTHON2 = $(HOST_DIR)/usr/bin/python
+endif
diff --git a/support/dependencies/check-host-python2.sh b/support/dependencies/check-host-python2.sh
new file mode 100755
index 0000000..6ad4b3a
--- /dev/null
+++ b/support/dependencies/check-host-python2.sh
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+candidates="python python2"
+
+for candidate in ${candidates} ; do
+  which ${candidate} &>/dev/null || continue
+  # restrict version of python2 to 2.6 or 2.7
+  if ${candidate} --version 2>&1 | grep -q 'Python 2\.[6-7].*' ; then
+    # found a valid candidate, so quit now
+    echo $(which ${candidate})
+    exit
+  fi
+done
--
1.7.10.3

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Samuel Martin]

The libglib2's build system uses some python2 scripts in the install rules.

This patch ensures to have a valid host python2 binary and enforces this python2
binary at configure-time.

Signed-off-by: Samuel Martin <s.martin49@gmail.com>

diff --git a/package/libglib2/libglib2.mk b/package/libglib2/libglib2.mk
index d2445a1..500f309 100644
--- a/package/libglib2/libglib2.mk
+++ b/package/libglib2/libglib2.mk
@@ -42,7 +42,10 @@ LIBGLIB2_CONF_ENV = \
 		ac_use_included_regex=no gl_cv_c_restrict=no \
 		ac_cv_path_GLIB_GENMARSHAL=$(HOST_DIR)/usr/bin/glib-genmarshal ac_cv_prog_F77=no \
 		ac_cv_func_posix_getgrgid_r=no \
-		gt_cv_c_wchar_t=$(if $(BR2_USE_WCHAR),yes,no)
+		gt_cv_c_wchar_t=$(if $(BR2_USE_WCHAR),yes,no) \
+		ac_cv_path_PYTHON=$(PYTHON2)
+
+HOST_LIBGLIB2_CONF_ENV += ac_cv_path_PYTHON=$(PYTHON2)
 
 # old uClibc versions don't provide qsort_r
 ifeq ($(BR2_UCLIBC_VERSION_0_9_31)$(BR2_UCLIBC_VERSION_0_9_32)$(BR2_TOOLCHAIN_CTNG_uClibc)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y)
@@ -57,7 +60,9 @@ HOST_LIBGLIB2_CONF_OPT = \
 
 LIBGLIB2_DEPENDENCIES = host-pkg-config host-libglib2 libffi zlib $(if $(BR2_NEEDS_GETTEXT),gettext libintl)
 
-HOST_LIBGLIB2_DEPENDENCIES = host-pkg-config host-libffi host-zlib
+HOST_LIBGLIB2_DEPENDENCIES = host-pkg-config host-libffi host-zlib $(NEED_PYTHON2)
+
+LIBGLIB2_DEPENDENCIES += $(NEED_PYTHON2)
 
 ifneq ($(BR2_ENABLE_LOCALE),y)
 LIBGLIB2_DEPENDENCIES += libiconv
-- 
1.7.10.3

[Buildroot] [PATCH v3 1/2] dependencies: build a host python2 if no suitable one cane be found

[Samuel Martin]

ping?

-- 
Sam

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Samuel Martin]

ping?

-- 
Sam

[Buildroot] [PATCH v3 1/2] dependencies: build a host python2 if no suitable one cane be found

[Arnout Vandecappelle]

On 06/03/12 00:08, Samuel Martin wrote:
> Some distros choose to change the /usr/bin/python binary, make it pointed to
> python3 instead of python2.
>
> This may have some bad consequences for packages that uses some
> non-python3-compliant python scripts in their build system (eg. in install or
> post-install scripts).
>
> This patch checks for a suitable python2 version (2.6 or 2.7) on the host
> system, and declares the following variables:
> - PYTHON2: pointing to the host python2 binary;
> - NEED_PYTHON2: sets to "host-python" if no python2 binary has been found.
>
> This way, a package using some python2 scripts must:
> - adds $(NEED_PYTHON2) to its dependency list;
> - sets $(PYTHON2) as the python binary to be used.
>
> A side effect of this patch is getting rid of any host python. Buildroot can
> runs on a host without python, or with a too old python2 version, or with only
> python3.
>
> Changes since v2:
> - misc. fixes and cleanup
>
> Changes since v1:
> - use support/dependency infrastructure
> - rename some symbols

  The patch changelog should go under the Signed-off-by line, separated with
---

>
> Signed-off-by: Samuel Martin<s.martin49@gmail.com>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>


>
>   create mode 100644 support/dependencies/check-host-python2.mk
>   create mode 100755 support/dependencies/check-host-python2.sh
>
> diff --git a/support/dependencies/check-host-python2.mk b/support/dependencies/check-host-python2.mk
> new file mode 100644
> index 0000000..3b66ffc
> --- /dev/null
> +++ b/support/dependencies/check-host-python2.mk
> @@ -0,0 +1,6 @@
> +PYTHON2 = $(call suitable-host-package,python2)
> +
> +ifeq (,$(PYTHON2))

  Minor nit: we always write ifeq ($(PYTHON2),)


  Regards,
  Arnout

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Arnout Vandecappelle]

On 06/03/12 00:08, Samuel Martin wrote:
> The libglib2's build system uses some python2 scripts in the install rules.
>
> This patch ensures to have a valid host python2 binary and enforces this python2
> binary at configure-time.
>
> Signed-off-by: Samuel Martin<s.martin49@gmail.com>

Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

  And a bump.

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Thomas Petazzoni]

Le Sun,  3 Jun 2012 00:08:17 +0200,
Samuel Martin <s.martin49@gmail.com> a ?crit :

> The libglib2's build system uses some python2 scripts in the install rules.
> 
> This patch ensures to have a valid host python2 binary and enforces this python2
> binary at configure-time.
> 
> Signed-off-by: Samuel Martin <s.martin49@gmail.com>

Isn't there a way to make the libglib2 python script python2 *and*
python3 compatible?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Samuel Martin]

2012/7/17 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Le Sun,  3 Jun 2012 00:08:17 +0200,
> Samuel Martin <s.martin49@gmail.com> a ?crit :
>
>> The libglib2's build system uses some python2 scripts in the install rules.
>>
>> This patch ensures to have a valid host python2 binary and enforces this python2
>> binary at configure-time.
>>
>> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
>
> Isn't there a way to make the libglib2 python script python2 *and*
> python3 compatible?
>
The truth is I have not checked that.
Will do and report here.

Cheers,

-- 
Sam

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Samuel Martin]

Hi all,

Reviving an old thread... ;-)

2012/7/17 Samuel Martin <s.martin49@gmail.com>:
> 2012/7/17 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
>> Le Sun,  3 Jun 2012 00:08:17 +0200,
>> Samuel Martin <s.martin49@gmail.com> a ?crit :
>>
>>> The libglib2's build system uses some python2 scripts in the install rules.
>>>
>>> This patch ensures to have a valid host python2 binary and enforces this python2
>>> binary at configure-time.
>>>
>>> Signed-off-by: Samuel Martin <s.martin49@gmail.com>
>>
>> Isn't there a way to make the libglib2 python script python2 *and*
>> python3 compatible?
>>
> The truth is I have not checked that.
> Will do and report here.


Current state in BR, with or without host-python already installed:
- if host-python is already installed in the host directory, then
libglib2 found it; in this case PYTHON=$(HOST_DIR)/usr/bin/python, so
no problem (this is a side effect of the the way we set the PATH env.
var.).
- otherwise, libglib2 use the python binary from the system:
PYTHON=/usr/bin/python, which is wrong if /usr/bin/python points to
/usr/bin/python3.

With no host python2 found, the build error is:
[...]
Making all in tests
make[5]: Entering directory
`/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/tests'
UNINSTALLED_GLIB_SRCDIR=../.. \
	UNINSTALLED_GLIB_BUILDDIR=../.. \
	/usr/bin/python ../../gio/gdbus-2.0/codegen/gdbus-codegen \
	--interface-prefix org.project. \
	--generate-c-code gdbus-test-codegen-generated \
	--c-generate-object-manager \
	--c-namespace Foo_iGen \
	--generate-docbook gdbus-test-codegen-generated-doc \
	--annotate "org.project.Bar" Key1 Value1 \
	--annotate "org.project.Bar" org.gtk.GDBus.Internal Value2 \
	--annotate "org.project.Bar.HelloWorld()" Key3 Value3 \
	--annotate "org.project.Bar::TestSignal" Key4 Value4 \
	--annotate "org.project.Bar:ay" Key5 Value5 \
	--annotate "org.project.Bar.TestPrimitiveTypes()[val_int32]" Key6 Value6 \
	--annotate "org.project.Bar.TestPrimitiveTypes()[ret_uint32]" Key7 Value7 \
	--annotate "org.project.Bar::TestSignal[array_of_strings]" Key8 Value8 \
	./test-codegen.xml \
	
UNINSTALLED_GLIB_SRCDIR=../.. \
	UNINSTALLED_GLIB_BUILDDIR=../.. \
	/usr/bin/python ../../gio/gdbus-2.0/codegen/gdbus-codegen \
	--interface-prefix org.project. \
	--generate-c-code gdbus-test-codegen-generated \
	--c-generate-object-manager \
	--c-namespace Foo_iGen \
	--generate-docbook gdbus-test-codegen-generated-doc \
	--annotate "org.project.Bar" Key1 Value1 \
	--annotate "org.project.Bar" org.gtk.GDBus.Internal Value2 \
	--annotate "org.project.Bar.HelloWorld()" Key3 Value3 \
	--annotate "org.project.Bar::TestSignal" Key4 Value4 \
	--annotate "org.project.Bar:ay" Key5 Value5 \
	--annotate "org.project.Bar.TestPrimitiveTypes()[val_int32]" Key6 Value6 \
	--annotate "org.project.Bar.TestPrimitiveTypes()[ret_uint32]" Key7 Value7 \
	--annotate "org.project.Bar::TestSignal[array_of_strings]" Key8 Value8 \
	./test-codegen.xml \
	
Traceback (most recent call last):
  File "../../gio/gdbus-2.0/codegen/gdbus-codegen", line 41, in <module>
    sys.exit(codegen_main.codegen_main())
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/codegen_main.py",
line 171, in codegen_main
    parsed_ifaces = parser.parse_dbus_xml(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 289, in parse_dbus_xml
    parser = DBusXMLParser(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 57, in __init__
    self._parser.Parse(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 155, in handle_start_element
    if attrs.has_key('name') and self.doc_comment_last_symbol == attrs['name']:
AttributeError: 'dict' object has no attribute 'has_key'
make[5]: *** [gdbus-test-codegen-generated.c] Error 1
make[5]: *** Waiting for unfinished jobs....
Traceback (most recent call last):
  File "../../gio/gdbus-2.0/codegen/gdbus-codegen", line 41, in <module>
    sys.exit(codegen_main.codegen_main())
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/codegen_main.py",
line 171, in codegen_main
    parsed_ifaces = parser.parse_dbus_xml(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 289, in parse_dbus_xml
    parser = DBusXMLParser(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 57, in __init__
    self._parser.Parse(xml_data)
  File "/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/gdbus-2.0/codegen/parser.py",
line 155, in handle_start_element
    if attrs.has_key('name') and self.doc_comment_last_symbol == attrs['name']:
AttributeError: 'dict' object has no attribute 'has_key'
make[5]: *** [gdbus-test-codegen-generated.h] Error 1
make[5]: Leaving directory
`/home/samuel/data/workspace/src/buildroot/master/output/build/host-libglib2-2.30.2/gio/tests'


Current state in somewhere else:
Most of the time, other distros keep the dependency to python2 to
build glib2/gdbus-codegen, e.g.:
- archlinux (though the package has already been bumped to 2.32.4):
https://projects.archlinux.org/svntogit/packages.git/tree/trunk/PKGBUILD?h=packages/glib2&id=614e5ff9b1c81454a96eebad2baf0ab7a803329b
- gentoo:
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-libs/glib/glib-2.30.2.ebuild?view=markup
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-util/gdbus-codegen/gdbus-codegen-2.30.2.ebuild?view=markup


Current state in glib repository:
A patch fixing this has quite recently been merged upstream (which is
already included in the glib release 2.32.4 or later, and works fine
with the current version integrated in BR: 2.30.2):
http://git.gnome.org/browse/glib/patch/?id=03611f7c0670ea14eedbc121972aed7ce60bb9ee


So, I'll send a patch integrating the upstream patch.
If this new patch is accepted, then this patch series (host-python2
and libglib2 depending on python2) could be discarded.


Cheers,

-- 
Sam

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Thomas Petazzoni]

Hello,

Le Sat, 18 Aug 2012 14:06:22 +0200,
Samuel Martin <s.martin49@gmail.com> a ?crit :

> Current state in glib repository:
> A patch fixing this has quite recently been merged upstream (which is
> already included in the glib release 2.32.4 or later, and works fine
> with the current version integrated in BR: 2.30.2):
> http://git.gnome.org/browse/glib/patch/?id=03611f7c0670ea14eedbc121972aed7ce60bb9ee
> 
> So, I'll send a patch integrating the upstream patch.
> If this new patch is accepted, then this patch series (host-python2
> and libglib2 depending on python2) could be discarded.

Excellent! Thanks a lot for the follow-up work. I will discard your
previous patches from the patchwork, and we'll wait for your new patch
integrating the upstream glib solution for this.

Thanks again for working on this!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH v3 2/2] libglib2: fix install even if the host python binary refers to python3

[Samuel Martin]

Hi Thomas,

2012/8/20 Thomas Petazzoni <thomas.petazzoni@free-electrons.com>:
> Hello,
>
[...]
>
> Excellent! Thanks a lot for the follow-up work. I will discard your
> previous patches from the patchwork, and we'll wait for your new patch
> integrating the upstream glib solution for this.
Here it is:
http://patchwork.ozlabs.org/patch/178456/

> Thanks again for working on this!
de nada ;)


Cheers,

-- 
Sam