[Buildroot] [PATCH] meson: bump version to 0.48.1

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Eric Le Bihan]

Bump version to 0.48.1 and re-add the RPATH patch.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
---
 ...y-fix-RPATH-if-install_rpath-is-not-empty.patch | 31 ++++++++++++++++++++++
 package/meson/meson.hash                           |  4 +--
 package/meson/meson.mk                             |  2 +-
 3 files changed, 34 insertions(+), 3 deletions(-)
 create mode 100644 package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch

diff --git a/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
new file mode 100644
index 0000000000..752042f7ec
--- /dev/null
+++ b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
@@ -0,0 +1,31 @@
+From cb5af88b1102ea8379323f95b9588c0bef71f175 Mon Sep 17 00:00:00 2001
+From: Eric Le Bihan <eric.le.bihan.dev@free.fr>
+Date: Sat, 14 Jul 2018 11:18:45 +0200
+Subject: [PATCH] Only fix RPATH if install_rpath is not empty
+
+---
+ mesonbuild/minstall.py | 8 ++++++++
+ 1 file changed, 8 insertions(+)
+
+diff --git a/mesonbuild/minstall.py b/mesonbuild/minstall.py
+index 4615b6dd..dd476ea8 100644
+--- a/mesonbuild/minstall.py
++++ b/mesonbuild/minstall.py
+@@ -440,6 +440,14 @@ class Installer:
+                 try:
+                     depfixer.fix_rpath(outname, install_rpath, final_path,
+                                        install_name_mappings, verbose=False)
++                    # Buildroot check-host-rpath script expects RPATH
++                    # But if install_rpath is empty, it will stripped.
++                    # So, preserve it in this case
++                    if install_rpath:
++                        depfixer.fix_rpath(outname, install_rpath, final_path,
++                                           install_name_mappings, verbose=False)
++                    else:
++                        print("Skipping RPATH fixing")
+                 except SystemExit as e:
+                     if isinstance(e.code, int) and e.code == 0:
+                         pass
+-- 
+2.14.4
+
diff --git a/package/meson/meson.hash b/package/meson/meson.hash
index 4a4923bd12..656f51f01d 100644
--- a/package/meson/meson.hash
+++ b/package/meson/meson.hash
@@ -1,4 +1,4 @@
 # Locally calculated after checking pgp signature
-# https://github.com/mesonbuild/meson/releases/download/0.47.1/meson-0.47.1.tar.gz.asc
-sha256 d673de79f7bab064190a5ea06140eaa8415efb386d0121ba549f6d66c555ada6 meson-0.47.1.tar.gz
+# https://github.com/mesonbuild/meson/releases/download/0.48.1/meson-0.48.1.tar.gz.asc
+sha256 425d12edbb81498314926afa93a3eb2fca4ca6929cbe0f8abeb6ab45d1d9fa8f meson-0.48.1.tar.gz
 sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 COPYING
diff --git a/package/meson/meson.mk b/package/meson/meson.mk
index 08e5c505c9..f24aef06fc 100644
--- a/package/meson/meson.mk
+++ b/package/meson/meson.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-MESON_VERSION = 0.47.1
+MESON_VERSION = 0.48.1
 MESON_SITE = https://github.com/mesonbuild/meson/releases/download/$(MESON_VERSION)
 MESON_LICENSE = Apache-2.0
 MESON_LICENSE_FILES = COPYING
-- 
2.14.4

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Thomas Petazzoni]

Hello ?ric,

On Thu, 18 Oct 2018 22:44:31 +0200, Eric Le Bihan wrote:
> Bump version to 0.48.1 and re-add the RPATH patch.
> 
> Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>

There was another thread on the mailing list saying that the RPATH
patch was in fact needed for host meson packages, and it seems
therefore independent from the version bump.

Would it be possible to have two separate patches, one re-adding the
RPATH fix first, one doing the bump? Indeed we may need to backport the
former to one of our stable branches.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Peter Seiderer]

Hello Eric,

On Thu, 18 Oct 2018 22:44:31 +0200, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote:

> Bump version to 0.48.1 and re-add the RPATH patch.
> 
> Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> ---
>  ...y-fix-RPATH-if-install_rpath-is-not-empty.patch | 31 ++++++++++++++++++++++
>  package/meson/meson.hash                           |  4 +--
>  package/meson/meson.mk                             |  2 +-
>  3 files changed, 34 insertions(+), 3 deletions(-)
>  create mode 100644 package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> 
> diff --git a/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> new file mode 100644
> index 0000000000..752042f7ec
> --- /dev/null
> +++ b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> @@ -0,0 +1,31 @@
> +From cb5af88b1102ea8379323f95b9588c0bef71f175 Mon Sep 17 00:00:00 2001
> +From: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> +Date: Sat, 14 Jul 2018 11:18:45 +0200
> +Subject: [PATCH] Only fix RPATH if install_rpath is not empty
> +

Signed-off-by line missing here...

> +---
> + mesonbuild/minstall.py | 8 ++++++++
> + 1 file changed, 8 insertions(+)
> +
> +diff --git a/mesonbuild/minstall.py b/mesonbuild/minstall.py
> +index 4615b6dd..dd476ea8 100644
> +--- a/mesonbuild/minstall.py
> ++++ b/mesonbuild/minstall.py
> +@@ -440,6 +440,14 @@ class Installer:
> +                 try:
> +                     depfixer.fix_rpath(outname, install_rpath, final_path,
> +                                        install_name_mappings, verbose=False)
> ++                    # Buildroot check-host-rpath script expects RPATH
> ++                    # But if install_rpath is empty, it will stripped.
> ++                    # So, preserve it in this case
> ++                    if install_rpath:
> ++                        depfixer.fix_rpath(outname, install_rpath, final_path,
> ++                                           install_name_mappings, verbose=False)
> ++                    else:
> ++                        print("Skipping RPATH fixing")
> +                 except SystemExit as e:
> +                     if isinstance(e.code, int) and e.code == 0:
> +                         pass
> +-- 
> +2.14.4
> +
> diff --git a/package/meson/meson.hash b/package/meson/meson.hash
> index 4a4923bd12..656f51f01d 100644
> --- a/package/meson/meson.hash
> +++ b/package/meson/meson.hash
> @@ -1,4 +1,4 @@
>  # Locally calculated after checking pgp signature
> -# https://github.com/mesonbuild/meson/releases/download/0.47.1/meson-0.47.1.tar.gz.asc
> -sha256 d673de79f7bab064190a5ea06140eaa8415efb386d0121ba549f6d66c555ada6 meson-0.47.1.tar.gz
> +# https://github.com/mesonbuild/meson/releases/download/0.48.1/meson-0.48.1.tar.gz.asc
> +sha256 425d12edbb81498314926afa93a3eb2fca4ca6929cbe0f8abeb6ab45d1d9fa8f meson-0.48.1.tar.gz
>  sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 COPYING
> diff --git a/package/meson/meson.mk b/package/meson/meson.mk
> index 08e5c505c9..f24aef06fc 100644
> --- a/package/meson/meson.mk
> +++ b/package/meson/meson.mk
> @@ -4,7 +4,7 @@
>  #
>  ################################################################################
>  
> -MESON_VERSION = 0.47.1
> +MESON_VERSION = 0.48.1
>  MESON_SITE = https://github.com/mesonbuild/meson/releases/download/$(MESON_VERSION)
>  MESON_LICENSE = Apache-2.0
>  MESON_LICENSE_FILES = COPYING

With the patch applied I get the following error:

>>> host-meson 0.48.1 Building
(cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
Traceback (most recent call last):
  File "setup.py", line 26, in <module>
    from setuptools import setup
ModuleNotFoundError: No module named 'setuptools'

	$ ls -d build/host-python*
build/host-python-2.7.15
build/host-python3-3.7.0
build/host-python-setuptools-40.0.0

Regards,
Peter

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Peter Seiderer]

On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:

> Hello Eric,
> 
> On Thu, 18 Oct 2018 22:44:31 +0200, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote:
> 
> > Bump version to 0.48.1 and re-add the RPATH patch.
> > 
> > Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > ---
> >  ...y-fix-RPATH-if-install_rpath-is-not-empty.patch | 31 ++++++++++++++++++++++
> >  package/meson/meson.hash                           |  4 +--
> >  package/meson/meson.mk                             |  2 +-
> >  3 files changed, 34 insertions(+), 3 deletions(-)
> >  create mode 100644 package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > 
> > diff --git a/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > new file mode 100644
> > index 0000000000..752042f7ec
> > --- /dev/null
> > +++ b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > @@ -0,0 +1,31 @@
> > +From cb5af88b1102ea8379323f95b9588c0bef71f175 Mon Sep 17 00:00:00 2001
> > +From: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > +Date: Sat, 14 Jul 2018 11:18:45 +0200
> > +Subject: [PATCH] Only fix RPATH if install_rpath is not empty
> > +  
> 
> Signed-off-by line missing here...
> 
> > +---
> > + mesonbuild/minstall.py | 8 ++++++++
> > + 1 file changed, 8 insertions(+)
> > +
> > +diff --git a/mesonbuild/minstall.py b/mesonbuild/minstall.py
> > +index 4615b6dd..dd476ea8 100644
> > +--- a/mesonbuild/minstall.py
> > ++++ b/mesonbuild/minstall.py
> > +@@ -440,6 +440,14 @@ class Installer:
> > +                 try:
> > +                     depfixer.fix_rpath(outname, install_rpath, final_path,
> > +                                        install_name_mappings, verbose=False)
> > ++                    # Buildroot check-host-rpath script expects RPATH
> > ++                    # But if install_rpath is empty, it will stripped.
> > ++                    # So, preserve it in this case
> > ++                    if install_rpath:
> > ++                        depfixer.fix_rpath(outname, install_rpath, final_path,
> > ++                                           install_name_mappings, verbose=False)
> > ++                    else:
> > ++                        print("Skipping RPATH fixing")
> > +                 except SystemExit as e:
> > +                     if isinstance(e.code, int) and e.code == 0:
> > +                         pass
> > +-- 
> > +2.14.4
> > +
> > diff --git a/package/meson/meson.hash b/package/meson/meson.hash
> > index 4a4923bd12..656f51f01d 100644
> > --- a/package/meson/meson.hash
> > +++ b/package/meson/meson.hash
> > @@ -1,4 +1,4 @@
> >  # Locally calculated after checking pgp signature
> > -# https://github.com/mesonbuild/meson/releases/download/0.47.1/meson-0.47.1.tar.gz.asc
> > -sha256 d673de79f7bab064190a5ea06140eaa8415efb386d0121ba549f6d66c555ada6 meson-0.47.1.tar.gz
> > +# https://github.com/mesonbuild/meson/releases/download/0.48.1/meson-0.48.1.tar.gz.asc
> > +sha256 425d12edbb81498314926afa93a3eb2fca4ca6929cbe0f8abeb6ab45d1d9fa8f meson-0.48.1.tar.gz
> >  sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 COPYING
> > diff --git a/package/meson/meson.mk b/package/meson/meson.mk
> > index 08e5c505c9..f24aef06fc 100644
> > --- a/package/meson/meson.mk
> > +++ b/package/meson/meson.mk
> > @@ -4,7 +4,7 @@
> >  #
> >  ################################################################################
> >  
> > -MESON_VERSION = 0.47.1
> > +MESON_VERSION = 0.48.1
> >  MESON_SITE = https://github.com/mesonbuild/meson/releases/download/$(MESON_VERSION)
> >  MESON_LICENSE = Apache-2.0
> >  MESON_LICENSE_FILES = COPYING  
> 
> With the patch applied I get the following error:
> 
> >>> host-meson 0.48.1 Building  
> (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
> Traceback (most recent call last):
>   File "setup.py", line 26, in <module>
>     from setuptools import setup
> ModuleNotFoundError: No module named 'setuptools'
> 
> 	$ ls -d build/host-python*
> build/host-python-2.7.15
> build/host-python3-3.7.0
> build/host-python-setuptools-40.0.0

graph-depends.pdf shows the following host-meson dependencies:

- host-meson --> host-python3
- host-meson --> host-python-setuptools --> host-python
- host-meson --> host-ninja --> host-python

Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?

Regards,
Peter

> 
> Regards,
> Peter
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Eric Le Bihan]

Hi!

On 2018-10-19 20:37, Peter Seiderer wrote:
> On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:
>
> > Hello Eric,
> >
> > On Thu, 18 Oct 2018 22:44:31 +0200, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote:
> >
> > > Bump version to 0.48.1 and re-add the RPATH patch.
> > >
> > > Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > > ---
> > >  ...y-fix-RPATH-if-install_rpath-is-not-empty.patch | 31 ++++++++++++++++++++++
> > >  package/meson/meson.hash                           |  4 +--
> > >  package/meson/meson.mk                             |  2 +-
> > >  3 files changed, 34 insertions(+), 3 deletions(-)
> > >  create mode 100644 package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > >
> > > diff --git a/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > > new file mode 100644
> > > index 0000000000..752042f7ec
> > > --- /dev/null
> > > +++ b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > > @@ -0,0 +1,31 @@
> > > +From cb5af88b1102ea8379323f95b9588c0bef71f175 Mon Sep 17 00:00:00 2001
> > > +From: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > > +Date: Sat, 14 Jul 2018 11:18:45 +0200
> > > +Subject: [PATCH] Only fix RPATH if install_rpath is not empty
> > > +
> >
> > Signed-off-by line missing here...
> >
Good catch! Will fix.

> > With the patch applied I get the following error:
> >
> > >>> host-meson 0.48.1 Building
> > (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
> > Traceback (most recent call last):
> >   File "setup.py", line 26, in <module>
> >     from setuptools import setup
> > ModuleNotFoundError: No module named 'setuptools'
> >
> > 	$ ls -d build/host-python*
> > build/host-python-2.7.15
> > build/host-python3-3.7.0
> > build/host-python-setuptools-40.0.0
>
> graph-depends.pdf shows the following host-meson dependencies:
>
> - host-meson --> host-python3
> - host-meson --> host-python-setuptools --> host-python
> - host-meson --> host-ninja --> host-python
>
> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?

I got the same issue and thought that was related to using an old build
environment and not a fresh one.

The package host-python-setuptools was using $(HOST_DIR)/bin/python,
symlinked to $(HOST_DIR)/bin/python2.7. Symlinking to python3 and
rebuilding host-python-setuptools solves host-meson build issue, but
that is not a proper solution.

I'm a bit puzzled... If for some reason host-meson had been built before
a package requiring host-python2 and host-python-setuptools, would the build
of this package fail due to host-python-setuptools having been built
for host-python3?

Thanks for the review.

Regards,

--
ELB

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Peter Seiderer]

Hello Eric,

On Fri, 19 Oct 2018 23:10:39 +0200, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote:

> Hi!
> 
> On 2018-10-19 20:37, Peter Seiderer wrote:
> > On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:
> >  
> > > Hello Eric,
> > >
> > > On Thu, 18 Oct 2018 22:44:31 +0200, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote:
> > >  
> > > > Bump version to 0.48.1 and re-add the RPATH patch.
> > > >
> > > > Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > > > ---
> > > >  ...y-fix-RPATH-if-install_rpath-is-not-empty.patch | 31 ++++++++++++++++++++++
> > > >  package/meson/meson.hash                           |  4 +--
> > > >  package/meson/meson.mk                             |  2 +-
> > > >  3 files changed, 34 insertions(+), 3 deletions(-)
> > > >  create mode 100644 package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > > >
> > > > diff --git a/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > > > new file mode 100644
> > > > index 0000000000..752042f7ec
> > > > --- /dev/null
> > > > +++ b/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
> > > > @@ -0,0 +1,31 @@
> > > > +From cb5af88b1102ea8379323f95b9588c0bef71f175 Mon Sep 17 00:00:00 2001
> > > > +From: Eric Le Bihan <eric.le.bihan.dev@free.fr>
> > > > +Date: Sat, 14 Jul 2018 11:18:45 +0200
> > > > +Subject: [PATCH] Only fix RPATH if install_rpath is not empty
> > > > +  
> > >
> > > Signed-off-by line missing here...
> > >  
> Good catch! Will fix.
> 
> > > With the patch applied I get the following error:
> > >  
> > > >>> host-meson 0.48.1 Building  
> > > (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
> > > Traceback (most recent call last):
> > >   File "setup.py", line 26, in <module>
> > >     from setuptools import setup
> > > ModuleNotFoundError: No module named 'setuptools'
> > >
> > > 	$ ls -d build/host-python*
> > > build/host-python-2.7.15
> > > build/host-python3-3.7.0
> > > build/host-python-setuptools-40.0.0  
> >
> > graph-depends.pdf shows the following host-meson dependencies:
> >
> > - host-meson --> host-python3
> > - host-meson --> host-python-setuptools --> host-python
> > - host-meson --> host-ninja --> host-python
> >
> > Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?  
> 
> I got the same issue and thought that was related to using an old build
> environment and not a fresh one.
> 
> The package host-python-setuptools was using $(HOST_DIR)/bin/python,
> symlinked to $(HOST_DIR)/bin/python2.7. Symlinking to python3 and
> rebuilding host-python-setuptools solves host-meson build issue, but
> that is not a proper solution.

Maybe a host-python3-setuptools package needed?

> 
> I'm a bit puzzled... If for some reason host-meson had been built before
> a package requiring host-python2 and host-python-setuptools, would the build
> of this package fail due to host-python-setuptools having been built
> for host-python3?

And host-pyhton-setuptools changed to always build for python2?

Regards,
Peter

> 
> Thanks for the review.
> 
> Regards,
> 
> --
> ELB
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Arnout Vandecappelle]

 Hi Eric,

On 19/10/2018 22:10, Eric Le Bihan wrote:
> Hi!
> 
> On 2018-10-19 20:37, Peter Seiderer wrote:
>> On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:
[snip]
>>> With the patch applied I get the following error:
>>>
>>>>>> host-meson 0.48.1 Building
>>> (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
>>> Traceback (most recent call last):
>>>   File "setup.py", line 26, in <module>
>>>     from setuptools import setup
>>> ModuleNotFoundError: No module named 'setuptools'>>>
>>> 	$ ls -d build/host-python*
>>> build/host-python-2.7.15
>>> build/host-python3-3.7.0
>>> build/host-python-setuptools-40.0.0
>>
>> graph-depends.pdf shows the following host-meson dependencies:
>>
>> - host-meson --> host-python3
>> - host-meson --> host-python-setuptools --> host-python
>> - host-meson --> host-ninja --> host-python
>>
>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?

 That is correct. When target python or python3 is selected, then the
corresponding host python is also selected, and *all* host python packages are
built for that specific python version. But if only target python is selected,
then you will get host-python as well, but you can also still depend on
host-python3. In that case, however, all the host python packages will be built
only for python2, not for python3. Hence the breakage.

 I'm surprised though that we don't see this in the autobuilders... Your patch
doesn't change anything there, does it?

> 
> I got the same issue and thought that was related to using an old build
> environment and not a fresh one.
> 
> The package host-python-setuptools was using $(HOST_DIR)/bin/python,
> symlinked to $(HOST_DIR)/bin/python2.7. Symlinking to python3 and
> rebuilding host-python-setuptools solves host-meson build issue, but
> that is not a proper solution.
> 
> I'm a bit puzzled... If for some reason host-meson had been built before
> a package requiring host-python2 and host-python-setuptools, would the build
> of this package fail due to host-python-setuptools having been built
> for host-python3?

 No, it purely depends on what you have selected for the target.

 Hm, actually, if you have *nothing* selected for the target, it gets even more
confusing, because any host package that depends on python 2-or-3 will default
to python2. I guess it then depends on the build order which version gets
used... Bad!

 Yann is working on a series to support side-by-side python2 and python3 on the
host and target. That would solve this issue.

 Regards,
 Arnout

> 
> Thanks for the review.
> 
> Regards,
> 
> --
> ELB
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Arnout Vandecappelle]

On 20/10/2018 13:57, Arnout Vandecappelle wrote:
>  Hi Eric,
>
> On 19/10/2018 22:10, Eric Le Bihan wrote:
>> Hi!
>>
>> On 2018-10-19 20:37, Peter Seiderer wrote:
>>> On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:
> [snip]
>>>> With the patch applied I get the following error:
>>>>
>>>>>>> host-meson 0.48.1 Building
>>>> (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
>>>> Traceback (most recent call last):
>>>>   File "setup.py", line 26, in <module>
>>>>     from setuptools import setup
>>>> ModuleNotFoundError: No module named 'setuptools'>>>
>>>> 	$ ls -d build/host-python*
>>>> build/host-python-2.7.15
>>>> build/host-python3-3.7.0
>>>> build/host-python-setuptools-40.0.0
>>> graph-depends.pdf shows the following host-meson dependencies:
>>>
>>> - host-meson --> host-python3
>>> - host-meson --> host-python-setuptools --> host-python
>>> - host-meson --> host-ninja --> host-python
>>>
>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
>  That is correct. When target python or python3 is selected, then the
> corresponding host python is also selected, and *all* host python packages are
> built for that specific python version. But if only target python is selected,
> then you will get host-python as well, but you can also still depend on
> host-python3. In that case, however, all the host python packages will be built
> only for python2, not for python3. Hence the breakage.
>
>  I'm surprised though that we don't see this in the autobuilders... Your patch
> doesn't change anything there, does it?

?It actually does... meson 0.47.1 still had a fallback on distutils if
setuptools wasn't available, but 0.48.1 removed that fallback.


?Regards,
?Arnout


>
>> I got the same issue and thought that was related to using an old build
>> environment and not a fresh one.
>>
>> The package host-python-setuptools was using $(HOST_DIR)/bin/python,
>> symlinked to $(HOST_DIR)/bin/python2.7. Symlinking to python3 and
>> rebuilding host-python-setuptools solves host-meson build issue, but
>> that is not a proper solution.
>>
>> I'm a bit puzzled... If for some reason host-meson had been built before
>> a package requiring host-python2 and host-python-setuptools, would the build
>> of this package fail due to host-python-setuptools having been built
>> for host-python3?
>  No, it purely depends on what you have selected for the target.
>
>  Hm, actually, if you have *nothing* selected for the target, it gets even more
> confusing, because any host package that depends on python 2-or-3 will default
> to python2. I guess it then depends on the build order which version gets
> used... Bad!
>
>  Yann is working on a series to support side-by-side python2 and python3 on the
> host and target. That would solve this issue.
>
>  Regards,
>  Arnout
>
>> Thanks for the review.
>>
>> Regards,
>>
>> --
>> ELB
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
>>

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Arnout Vandecappelle]

On 20/10/2018 15:00, Arnout Vandecappelle wrote:
>
> On 20/10/2018 13:57, Arnout Vandecappelle wrote:
[snip]
>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
>>  That is correct. When target python or python3 is selected, then the
>> corresponding host python is also selected, and *all* host python packages are
>> built for that specific python version. But if only target python is selected,
>> then you will get host-python as well, but you can also still depend on
>> host-python3. In that case, however, all the host python packages will be built
>> only for python2, not for python3. Hence the breakage.
>>
>>  I'm surprised though that we don't see this in the autobuilders... Your patch
>> doesn't change anything there, does it?
> ?It actually does... meson 0.47.1 still had a fallback on distutils if
> setuptools wasn't available, but 0.48.1 removed that fallback.

?Long discussion an the BR developer meeting, here is the conclusion. Adding
Yegor and Asaf in Cc as python maintainers.

?The issue we have at the moment is that all host python packages are built for
Python 2, not Python 3, *unless* Python 3 is selected for the target.

?However, *most* packages support Python 3 as well.

?So, the idea is to default to Python 3 for the host. In other words, all host
packages will use Python 3 (independent of what is going on the target), except
for the packages that work on python2 only. This step by itself is quite a bit
of work because there are many packages that can use python2 OR python3.

?This leaves us with two problems.

?First of all, there may be python-2-only host packages that depend on some
other host-python package that would only get built for python3. We investigated
that, and it looks OK: there are only 4 python-2-only host packages: gdb,
python-cheetah, python-enum34, and python-yieldfrom. The latter two are
compatibility libraries so obviously only available on Python2. python-yieldfrom
is not used at all. python-enum34 is only used by setools, in case it needs to
be built for host-python2, so that should be OK. host-python-cheetah is only
used by gr-osmosdr; bumping python-cheetah probably enables it for Python 3 so
would solve that issue. And gdb doesn't need any other python package, just
python itself. So all this looks fixable.

?The bigger problem, is when a setup.py script tries to import a module from
some python package. The clearest one of those is python-setuptools: any python
package that uses setuptools will import setuptools using the same host-python
version as on the target. So when you have python2 on the target, the setup.py
script will be run in python2, so we need host-python-setuptools for python2.
The solution there, we think, is to add a separate host-python2-setuptools
package that is used instead of host-python-setuptools in case the target is
python2. Unfortunately there are quite a few others that may need the same
treatment. Some of them, however, probably don't really need to be imported in
the setup.py script so they can be removed as a dependency. Others, however,
will need the same treatment. In some cases, we might be able to just build both
python2 and python3 versions (e.g. for python-sip this should be possible).

?Finally, there are most likely some corner cases that we didn't think of...


?So it looks like this will be a pretty big migration. Therefore, for the time
being, it may be easier to just add a host-python3-setuptools package and use
that in meson.


?By the way, lirc-tools has the same problem. It adds python3
host-python-setuptools to its dependencies, but the setuptools don't get
imported so it just doesn't work (I think). Baruch, care to take a look at that?


?Regards,
?Arnout

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Eric Le Bihan]

Hi!

On 2018-10-20 15:00, Arnout Vandecappelle wrote:
>
>
> On 20/10/2018 13:57, Arnout Vandecappelle wrote:
> >  Hi Eric,
> >
> > On 19/10/2018 22:10, Eric Le Bihan wrote:
> >> Hi!
> >>
> >> On 2018-10-19 20:37, Peter Seiderer wrote:
> >>> On Fri, 19 Oct 2018 20:30:50 +0200, Peter Seiderer <ps.report@gmx.net> wrote:
> > [snip]
> >>>> With the patch applied I get the following error:
> >>>>
> >>>>>>> host-meson 0.48.1 Building
> >>>> (cd .../build/host-meson-0.48.1//; PATH=".../host/bin:.../host/sbin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/sbin" PYTHONNOUSERSITE=1  .../host/bin/python3 setup.py build  )
> >>>> Traceback (most recent call last):
> >>>>   File "setup.py", line 26, in <module>
> >>>>     from setuptools import setup
> >>>> ModuleNotFoundError: No module named 'setuptools'>>>
> >>>> 	$ ls -d build/host-python*
> >>>> build/host-python-2.7.15
> >>>> build/host-python3-3.7.0
> >>>> build/host-python-setuptools-40.0.0
> >>> graph-depends.pdf shows the following host-meson dependencies:
> >>>
> >>> - host-meson --> host-python3
> >>> - host-meson --> host-python-setuptools --> host-python
> >>> - host-meson --> host-ninja --> host-python
> >>>
> >>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
> >  That is correct. When target python or python3 is selected, then the
> > corresponding host python is also selected, and *all* host python packages are
> > built for that specific python version. But if only target python is selected,
> > then you will get host-python as well, but you can also still depend on
> > host-python3. In that case, however, all the host python packages will be built
> > only for python2, not for python3. Hence the breakage.
> >
> >  I'm surprised though that we don't see this in the autobuilders... Your patch
> > doesn't change anything there, does it?
>
> ?It actually does... meson 0.47.1 still had a fallback on distutils if
> setuptools wasn't available, but 0.48.1 removed that fallback.

Indeed, upstream commit 86298f2109d215ad6b26d3462af7b685d52d0dd7 does
it. I'll try to revert it and see how Meson behaves.

Regards,

--
ELB

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Baruch Siach]

Hi Arnout,

Arnout Vandecappelle writes:
> On 20/10/2018 15:00, Arnout Vandecappelle wrote:
>> On 20/10/2018 13:57, Arnout Vandecappelle wrote:
> [snip]
>>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
>>>  That is correct. When target python or python3 is selected, then the
>>> corresponding host python is also selected, and *all* host python packages are
>>> built for that specific python version. But if only target python is selected,
>>> then you will get host-python as well, but you can also still depend on
>>> host-python3. In that case, however, all the host python packages will be built
>>> only for python2, not for python3. Hence the breakage.
>>>
>>>  I'm surprised though that we don't see this in the autobuilders... Your patch
>>> doesn't change anything there, does it?
>> It actually does... meson 0.47.1 still had a fallback on distutils if
>> setuptools wasn't available, but 0.48.1 removed that fallback.
>
> Long discussion an the BR developer meeting, here is the conclusion. Adding
> Yegor and Asaf in Cc as python maintainers.
>
> The issue we have at the moment is that all host python packages are built for
> Python 2, not Python 3, *unless* Python 3 is selected for the target.
>
> However, *most* packages support Python 3 as well.
>
> So, the idea is to default to Python 3 for the host. In other words, all host
> packages will use Python 3 (independent of what is going on the target), except
> for the packages that work on python2 only. This step by itself is quite a bit
> of work because there are many packages that can use python2 OR python3.
>
> This leaves us with two problems.
>
> First of all, there may be python-2-only host packages that depend on some
> other host-python package that would only get built for python3. We investigated
> that, and it looks OK: there are only 4 python-2-only host packages: gdb,
> python-cheetah, python-enum34, and python-yieldfrom. The latter two are
> compatibility libraries so obviously only available on Python2. python-yieldfrom
> is not used at all. python-enum34 is only used by setools, in case it needs to
> be built for host-python2, so that should be OK. host-python-cheetah is only
> used by gr-osmosdr; bumping python-cheetah probably enables it for Python 3 so
> would solve that issue. And gdb doesn't need any other python package, just
> python itself. So all this looks fixable.
>
> The bigger problem, is when a setup.py script tries to import a module from
> some python package. The clearest one of those is python-setuptools: any python
> package that uses setuptools will import setuptools using the same host-python
> version as on the target. So when you have python2 on the target, the setup.py
> script will be run in python2, so we need host-python-setuptools for python2.
> The solution there, we think, is to add a separate host-python2-setuptools
> package that is used instead of host-python-setuptools in case the target is
> python2. Unfortunately there are quite a few others that may need the same
> treatment. Some of them, however, probably don't really need to be imported in
> the setup.py script so they can be removed as a dependency. Others, however,
> will need the same treatment. In some cases, we might be able to just build both
> python2 and python3 versions (e.g. for python-sip this should be possible).
>
> Finally, there are most likely some corner cases that we didn't think of...
>
> So it looks like this will be a pretty big migration. Therefore, for the time
> being, it may be easier to just add a host-python3-setuptools package and use
> that in meson.
>
> By the way, lirc-tools has the same problem. It adds python3
> host-python-setuptools to its dependencies, but the setuptools don't get
> imported so it just doesn't work (I think). Baruch, care to take a look at that?

I'm not that much of a Python expert so I might be missing something
here.

The python-pkg/setup.py file in the lirc-tools source tree have this:

from setuptools import setup, Extension

A build test with python3 enabled installs lirc python and binary code
under /usr/lib/python3.7/site-packages/lirc in the target.

So what is it that "doesn't work"? Please enlighten me.

Thanks,
baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Peter Seiderer]

Hello Baruch,

On Tue, 23 Oct 2018 14:17:12 +0300, Baruch Siach <baruch@tkos.co.il> wrote:

> Hi Arnout,
> 
> Arnout Vandecappelle writes:
> > On 20/10/2018 15:00, Arnout Vandecappelle wrote:  
> >> On 20/10/2018 13:57, Arnout Vandecappelle wrote:  
> > [snip]  
> >>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?  
> >>>  That is correct. When target python or python3 is selected, then the
> >>> corresponding host python is also selected, and *all* host python packages are
> >>> built for that specific python version. But if only target python is selected,
> >>> then you will get host-python as well, but you can also still depend on
> >>> host-python3. In that case, however, all the host python packages will be built
> >>> only for python2, not for python3. Hence the breakage.
> >>>
> >>>  I'm surprised though that we don't see this in the autobuilders... Your patch
> >>> doesn't change anything there, does it?  
> >> It actually does... meson 0.47.1 still had a fallback on distutils if
> >> setuptools wasn't available, but 0.48.1 removed that fallback.  
> >
> > Long discussion an the BR developer meeting, here is the conclusion. Adding
> > Yegor and Asaf in Cc as python maintainers.
> >
> > The issue we have at the moment is that all host python packages are built for
> > Python 2, not Python 3, *unless* Python 3 is selected for the target.
> >
> > However, *most* packages support Python 3 as well.
> >
> > So, the idea is to default to Python 3 for the host. In other words, all host
> > packages will use Python 3 (independent of what is going on the target), except
> > for the packages that work on python2 only. This step by itself is quite a bit
> > of work because there are many packages that can use python2 OR python3.
> >
> > This leaves us with two problems.
> >
> > First of all, there may be python-2-only host packages that depend on some
> > other host-python package that would only get built for python3. We investigated
> > that, and it looks OK: there are only 4 python-2-only host packages: gdb,
> > python-cheetah, python-enum34, and python-yieldfrom. The latter two are
> > compatibility libraries so obviously only available on Python2. python-yieldfrom
> > is not used at all. python-enum34 is only used by setools, in case it needs to
> > be built for host-python2, so that should be OK. host-python-cheetah is only
> > used by gr-osmosdr; bumping python-cheetah probably enables it for Python 3 so
> > would solve that issue. And gdb doesn't need any other python package, just
> > python itself. So all this looks fixable.
> >
> > The bigger problem, is when a setup.py script tries to import a module from
> > some python package. The clearest one of those is python-setuptools: any python
> > package that uses setuptools will import setuptools using the same host-python
> > version as on the target. So when you have python2 on the target, the setup.py
> > script will be run in python2, so we need host-python-setuptools for python2.
> > The solution there, we think, is to add a separate host-python2-setuptools
> > package that is used instead of host-python-setuptools in case the target is
> > python2. Unfortunately there are quite a few others that may need the same
> > treatment. Some of them, however, probably don't really need to be imported in
> > the setup.py script so they can be removed as a dependency. Others, however,
> > will need the same treatment. In some cases, we might be able to just build both
> > python2 and python3 versions (e.g. for python-sip this should be possible).
> >
> > Finally, there are most likely some corner cases that we didn't think of...
> >
> > So it looks like this will be a pretty big migration. Therefore, for the time
> > being, it may be easier to just add a host-python3-setuptools package and use
> > that in meson.
> >
> > By the way, lirc-tools has the same problem. It adds python3
> > host-python-setuptools to its dependencies, but the setuptools don't get
> > imported so it just doesn't work (I think). Baruch, care to take a look at that?  
> 
> I'm not that much of a Python expert so I might be missing something
> here.
> 
> The python-pkg/setup.py file in the lirc-tools source tree have this:
> 
> from setuptools import setup, Extension
> 
> A build test with python3 enabled installs lirc python and binary code
> under /usr/lib/python3.7/site-packages/lirc in the target.
> 
> So what is it that "doesn't work"? Please enlighten me.

In the case of meson python3 is enabled for it AND python2 for ninja (via meson
dependency), in this case the host-setuptools are build/installed only for python2
and the meson build fails...

Regards,
Peter

> 
> Thanks,
> baruch
> 

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Baruch Siach]

Hi Peter,

Peter Seiderer writes:
> On Tue, 23 Oct 2018 14:17:12 +0300, Baruch Siach <baruch@tkos.co.il> wrote:
>> Arnout Vandecappelle writes:
>> > On 20/10/2018 15:00, Arnout Vandecappelle wrote:
>> >> On 20/10/2018 13:57, Arnout Vandecappelle wrote:
>> > [snip]
>> >>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
>> >>>  That is correct. When target python or python3 is selected, then the
>> >>> corresponding host python is also selected, and *all* host python packages are
>> >>> built for that specific python version. But if only target python is selected,
>> >>> then you will get host-python as well, but you can also still depend on
>> >>> host-python3. In that case, however, all the host python packages will be built
>> >>> only for python2, not for python3. Hence the breakage.
>> >>>
>> >>>  I'm surprised though that we don't see this in the autobuilders... Your patch
>> >>> doesn't change anything there, does it?
>> >> It actually does... meson 0.47.1 still had a fallback on distutils if
>> >> setuptools wasn't available, but 0.48.1 removed that fallback.
>> >
>> > Long discussion an the BR developer meeting, here is the conclusion. Adding
>> > Yegor and Asaf in Cc as python maintainers.
>> >
>> > The issue we have at the moment is that all host python packages are built for
>> > Python 2, not Python 3, *unless* Python 3 is selected for the target.
>> >
>> > However, *most* packages support Python 3 as well.
>> >
>> > So, the idea is to default to Python 3 for the host. In other words, all host
>> > packages will use Python 3 (independent of what is going on the target), except
>> > for the packages that work on python2 only. This step by itself is quite a bit
>> > of work because there are many packages that can use python2 OR python3.
>> >
>> > This leaves us with two problems.
>> >
>> > First of all, there may be python-2-only host packages that depend on some
>> > other host-python package that would only get built for python3. We investigated
>> > that, and it looks OK: there are only 4 python-2-only host packages: gdb,
>> > python-cheetah, python-enum34, and python-yieldfrom. The latter two are
>> > compatibility libraries so obviously only available on Python2. python-yieldfrom
>> > is not used at all. python-enum34 is only used by setools, in case it needs to
>> > be built for host-python2, so that should be OK. host-python-cheetah is only
>> > used by gr-osmosdr; bumping python-cheetah probably enables it for Python 3 so
>> > would solve that issue. And gdb doesn't need any other python package, just
>> > python itself. So all this looks fixable.
>> >
>> > The bigger problem, is when a setup.py script tries to import a module from
>> > some python package. The clearest one of those is python-setuptools: any python
>> > package that uses setuptools will import setuptools using the same host-python
>> > version as on the target. So when you have python2 on the target, the setup.py
>> > script will be run in python2, so we need host-python-setuptools for python2.
>> > The solution there, we think, is to add a separate host-python2-setuptools
>> > package that is used instead of host-python-setuptools in case the target is
>> > python2. Unfortunately there are quite a few others that may need the same
>> > treatment. Some of them, however, probably don't really need to be imported in
>> > the setup.py script so they can be removed as a dependency. Others, however,
>> > will need the same treatment. In some cases, we might be able to just build both
>> > python2 and python3 versions (e.g. for python-sip this should be possible).
>> >
>> > Finally, there are most likely some corner cases that we didn't think of...
>> >
>> > So it looks like this will be a pretty big migration. Therefore, for the time
>> > being, it may be easier to just add a host-python3-setuptools package and use
>> > that in meson.
>> >
>> > By the way, lirc-tools has the same problem. It adds python3
>> > host-python-setuptools to its dependencies, but the setuptools don't get
>> > imported so it just doesn't work (I think). Baruch, care to take a look at that?
>>
>> I'm not that much of a Python expert so I might be missing something
>> here.
>>
>> The python-pkg/setup.py file in the lirc-tools source tree have this:
>>
>> from setuptools import setup, Extension
>>
>> A build test with python3 enabled installs lirc python and binary code
>> under /usr/lib/python3.7/site-packages/lirc in the target.
>>
>> So what is it that "doesn't work"? Please enlighten me.
>
> In the case of meson python3 is enabled for it AND python2 for ninja (via meson
> dependency), in this case the host-setuptools are build/installed only for python2
> and the meson build fails...

So mason needs host-python3 (via HOST_MESON_NEEDS_HOST_PYTHON), but
since BR2_PACKAGE_PYTHON3 is disabled, ninja drags in host-python as
well. Is that correct?

I tried to reproduce something similar with BR2_PACKAGE_PYTHON3 enabled
using this config:

BR2_aarch64=y
BR2_cortex_a72=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_PACKAGE_LIRC_TOOLS=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_MONGODB=y

mongodb depends on host-scons that makes host-python enabled in addition
to host-python3 for target python3. Yet the build of 'mongodb
lirc-tools' targets succeeds, with the expected files in the lirc python
package on target.

baruch

--
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

[Buildroot] [PATCH] meson: bump version to 0.48.1

[Arnout Vandecappelle]

On 10/23/18 6:34 PM, Baruch Siach wrote:
> Hi Peter,
> 
> Peter Seiderer writes:
>> On Tue, 23 Oct 2018 14:17:12 +0300, Baruch Siach <baruch@tkos.co.il> wrote:
>>> Arnout Vandecappelle writes:
>>>> On 20/10/2018 15:00, Arnout Vandecappelle wrote:
>>>>> On 20/10/2018 13:57, Arnout Vandecappelle wrote:
>>>> [snip]
>>>>>>> Maybe host-python-setuptools are build for the wrong python version (python2 instead of python3)?
>>>>>>  That is correct. When target python or python3 is selected, then the
>>>>>> corresponding host python is also selected, and *all* host python packages are
>>>>>> built for that specific python version. But if only target python is selected,
>>>>>> then you will get host-python as well, but you can also still depend on
>>>>>> host-python3. In that case, however, all the host python packages will be built
>>>>>> only for python2, not for python3. Hence the breakage.
>>>>>>
>>>>>>  I'm surprised though that we don't see this in the autobuilders... Your patch
>>>>>> doesn't change anything there, does it?
>>>>> It actually does... meson 0.47.1 still had a fallback on distutils if
>>>>> setuptools wasn't available, but 0.48.1 removed that fallback.
>>>>
>>>> Long discussion an the BR developer meeting, here is the conclusion. Adding
>>>> Yegor and Asaf in Cc as python maintainers.
>>>>
>>>> The issue we have at the moment is that all host python packages are built for
>>>> Python 2, not Python 3, *unless* Python 3 is selected for the target.
>>>>
>>>> However, *most* packages support Python 3 as well.
>>>>
>>>> So, the idea is to default to Python 3 for the host. In other words, all host
>>>> packages will use Python 3 (independent of what is going on the target), except
>>>> for the packages that work on python2 only. This step by itself is quite a bit
>>>> of work because there are many packages that can use python2 OR python3.
>>>>
>>>> This leaves us with two problems.
>>>>
>>>> First of all, there may be python-2-only host packages that depend on some
>>>> other host-python package that would only get built for python3. We investigated
>>>> that, and it looks OK: there are only 4 python-2-only host packages: gdb,
>>>> python-cheetah, python-enum34, and python-yieldfrom. The latter two are
>>>> compatibility libraries so obviously only available on Python2. python-yieldfrom
>>>> is not used at all. python-enum34 is only used by setools, in case it needs to
>>>> be built for host-python2, so that should be OK. host-python-cheetah is only
>>>> used by gr-osmosdr; bumping python-cheetah probably enables it for Python 3 so
>>>> would solve that issue. And gdb doesn't need any other python package, just
>>>> python itself. So all this looks fixable.
>>>>
>>>> The bigger problem, is when a setup.py script tries to import a module from
>>>> some python package. The clearest one of those is python-setuptools: any python
>>>> package that uses setuptools will import setuptools using the same host-python
>>>> version as on the target. So when you have python2 on the target, the setup.py
>>>> script will be run in python2, so we need host-python-setuptools for python2.
>>>> The solution there, we think, is to add a separate host-python2-setuptools
>>>> package that is used instead of host-python-setuptools in case the target is
>>>> python2. Unfortunately there are quite a few others that may need the same
>>>> treatment. Some of them, however, probably don't really need to be imported in
>>>> the setup.py script so they can be removed as a dependency. Others, however,
>>>> will need the same treatment. In some cases, we might be able to just build both
>>>> python2 and python3 versions (e.g. for python-sip this should be possible).
>>>>
>>>> Finally, there are most likely some corner cases that we didn't think of...
>>>>
>>>> So it looks like this will be a pretty big migration. Therefore, for the time
>>>> being, it may be easier to just add a host-python3-setuptools package and use
>>>> that in meson.
>>>>
>>>> By the way, lirc-tools has the same problem. It adds python3
>>>> host-python-setuptools to its dependencies, but the setuptools don't get
>>>> imported so it just doesn't work (I think). Baruch, care to take a look at that?
>>>
>>> I'm not that much of a Python expert so I might be missing something
>>> here.
>>>
>>> The python-pkg/setup.py file in the lirc-tools source tree have this:
>>>
>>> from setuptools import setup, Extension
>>>
>>> A build test with python3 enabled installs lirc python and binary code
>>> under /usr/lib/python3.7/site-packages/lirc in the target.
>>>
>>> So what is it that "doesn't work"? Please enlighten me.
>>
>> In the case of meson python3 is enabled for it AND python2 for ninja (via meson
>> dependency), in this case the host-setuptools are build/installed only for python2
>> and the meson build fails...
> 
> So mason needs host-python3 (via HOST_MESON_NEEDS_HOST_PYTHON), but
> since BR2_PACKAGE_PYTHON3 is disabled, ninja drags in host-python as
> well. Is that correct?

 Correct. For the host python we have:

- if target python3 is enabled, we build host-python3 and all host python
packages are generated for python3;

- in any other case (i.e. whether or not target python is enabled), we build
host-python and all host python packages are generated for python2.

 So, any host package that tries to run its setup.py script which python3 will
fail if target python3 is not selected.


> I tried to reproduce something similar with BR2_PACKAGE_PYTHON3 enabled
> using this config:
> 
> BR2_aarch64=y
> BR2_cortex_a72=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_PACKAGE_LIRC_TOOLS=y
> BR2_PACKAGE_PYTHON3=y
> BR2_PACKAGE_MONGODB=y
> 
> mongodb depends on host-scons that makes host-python enabled in addition
> to host-python3 for target python3. Yet the build of 'mongodb
> lirc-tools' targets succeeds, with the expected files in the lirc python
> package on target.

 Here you have enabled target python3, so all is well. The meson case will also
work if you do have target python3 selected BTW.

 So indeed, lirc-tools will not have a problem, because the dependency on
setuptools is only there if python3 is enabled.

 Regards,
 Arnout