[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Adam Duskett]

Currently, packages that rely on host-python, and are compatible with both
python2 and 3, only build python3 if python3 is selected for the target.

This causes a problem in the following scenario:

- A user does not have a target python selected.
- Package A depends on host-python3
- Package B is compatible with host-python and host-python3

In this scenario, host-python is not needed, as package A and B are
both compatible with python3. However; both host-python and host-python3 will
be built because no target-python has been selected.

To fix this, the following patch series introduces a few minimal changes
that will allow the user to manually select a host version of python.
(Note: I have changed only the packages that I personally use and have tested.)

Adam Duskett (8):
  python: add host-entry
  python3: add host-entry
  python-setuptools: check host-python version
  pkg-waf.mk: check host-python version
  ninja: check host-python version
  libselinux: check host-python version
  setools: check host-python version
  libselinux: check host-python version

 package/Config.in.host                         |  2 ++
 package/libselinux/libselinux.mk               |  2 +-
 package/libsemanage/libsemanage.mk             |  2 +-
 package/ninja/ninja.mk                         |  2 +-
 package/pkg-waf.mk                             | 20 +++++++++++++-------
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 package/python/Config.in.host                  |  6 ++++++
 package/python3/Config.in.host                 |  6 ++++++
 package/setools/setools.mk                     |  6 ++++++
 9 files changed, 45 insertions(+), 11 deletions(-)
 create mode 100644 package/python/Config.in.host
 create mode 100644 package/python3/Config.in.host

-- 
2.14.3

[Buildroot] [PATCH 1/8] python: add selectable host entry

[Adam Duskett]

Add a user selectable entry to build python2 as a host package.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/Config.in.host        | 1 +
 package/python/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index dd6415bba5..7d908c409c 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -41,6 +41,7 @@ menu "Host utilities"
 	source "package/pkgconf/Config.in.host"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
+	source "package/python/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
diff --git a/package/python/Config.in.host b/package/python/Config.in.host
new file mode 100644
index 0000000000..0a8d6d179f
--- /dev/null
+++ b/package/python/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON
+	bool "host python"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
-- 
2.14.3

[Buildroot] [PATCH 2/8] python3: add selectable host entry

[Adam Duskett]

Add a user selectable entry to build python3 as a host package.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/Config.in.host         | 1 +
 package/python3/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python3/Config.in.host

diff --git a/package/Config.in.host b/package/Config.in.host
index 7d908c409c..19d45e9d9e 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -42,6 +42,7 @@ menu "Host utilities"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
 	source "package/python/Config.in.host"
+	source "package/python3/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
diff --git a/package/python3/Config.in.host b/package/python3/Config.in.host
new file mode 100644
index 0000000000..876661e71b
--- /dev/null
+++ b/package/python3/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON3
+	bool "host python3"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
-- 
2.14.3

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Adam Duskett]

python-setuptools is compatible with both python2 and python3, as such,
there is no need to force python2 as a dependency.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
index 28d4f29b5b..a21494a288 100644
--- a/package/python-setuptools/python-setuptools.mk
+++ b/package/python-setuptools/python-setuptools.mk
@@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
 PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif
+
 # recent setuptools versions require bootstrap.py to be invoked
 # before the standard setup process.
 define PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-	cd  $(@D) && $(HOST_DIR)/bin/python ./bootstrap.py
+	cd  $(@D) && $(HOST_DIR)/bin/$(HOST_PYTHON_SETUPTOOLS_PYLIBVER) ./bootstrap.py
 endef
 
 PYTHON_SETUPTOOLS_PRE_CONFIGURE_HOOKS = PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-- 
2.14.3

[Buildroot] [PATCH 4/8] pkg-waf.mk: check host-python version

[Adam Duskett]

waf is compatible with both python2 and python3. Now that there are host
entries for python, the make logic can be changed to check if host-python3
has been selected, and if so, depend on it instead of explicitly depending
only on python2.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/pkg-waf.mk | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/package/pkg-waf.mk b/package/pkg-waf.mk
index e5b606f063..8df287074b 100644
--- a/package/pkg-waf.mk
+++ b/package/pkg-waf.mk
@@ -36,9 +36,6 @@
 
 define inner-waf-package
 
-# We need host-python to run waf
-$(2)_DEPENDENCIES += host-python
-
 $(2)_NEEDS_EXTERNAL_WAF ?= NO
 
 # If the package does not have its own waf, use our own.
@@ -49,6 +46,15 @@ else
 $(2)_WAF = ./waf
 endif
 
+# We need a host-python to run waf
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+$(2)_DEPENDENCIES += host-python3
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python3
+else
+$(2)_DEPENDENCIES += host-python
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python2
+endif
+
 $(2)_BUILD_OPTS				?=
 $(2)_INSTALL_STAGING_OPTS		?=
 $(2)_INSTALL_TARGET_OPTS		?=
@@ -63,7 +69,7 @@ define $(2)_CONFIGURE_CMDS
 	cd $$(@D) && \
 	$$(TARGET_CONFIGURE_OPTS) \
 	$$($(2)_CONF_ENV) \
-	$$(HOST_DIR)/bin/python2 $$($(2)_WAF) configure \
+	$$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) configure \
 		--prefix=/usr \
 		--libdir=/usr/lib \
 		$$($(2)_CONF_OPTS) \
@@ -78,7 +84,7 @@ endif
 ifndef $(2)_BUILD_CMDS
 define $(2)_BUILD_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		build -j $$(PARALLEL_JOBS) $$($(2)_BUILD_OPTS) \
 		$$($(2)_WAF_OPTS)
 endef
@@ -91,7 +97,7 @@ endif
 ifndef $(2)_INSTALL_STAGING_CMDS
 define $(2)_INSTALL_STAGING_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(STAGING_DIR) \
 		$$($(2)_INSTALL_STAGING_OPTS) \
 		$$($(2)_WAF_OPTS)
@@ -105,7 +111,7 @@ endif
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(TARGET_DIR) \
 		$$($(2)_INSTALL_TARGET_OPTS) \
 		$$($(2)_WAF_OPTS)
-- 
2.14.3

[Buildroot] [PATCH 5/8] ninja: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/ninja/ninja.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/ninja/ninja.mk b/package/ninja/ninja.mk
index 259a87d216..b58d1a0f43 100644
--- a/package/ninja/ninja.mk
+++ b/package/ninja/ninja.mk
@@ -9,7 +9,7 @@ NINJA_SITE = $(call github,ninja-build,ninja,$(NINJA_VERSION))
 NINJA_LICENSE = Apache-2.0
 NINJA_LICENSE_FILES = COPYING
 
-HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
+HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_HOST_PYTHON3),host-python3,host-python)
 
 define HOST_NINJA_BUILD_CMDS
 	(cd $(@D); ./configure.py --bootstrap)
-- 
2.14.3

[Buildroot] [PATCH 6/8] libselinux: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/libselinux/libselinux.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/libselinux/libselinux.mk b/package/libselinux/libselinux.mk
index 8ac8000de5..f8e8ea1285 100644
--- a/package/libselinux/libselinux.mk
+++ b/package/libselinux/libselinux.mk
@@ -75,7 +75,7 @@ endef
 HOST_LIBSELINUX_DEPENDENCIES = \
 	host-libsepol host-pcre host-swig
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSELINUX_DEPENDENCIES += host-python3
 HOST_LIBSELINUX_PYINC = -I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/
 HOST_LIBSELINUX_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
-- 
2.14.3

[Buildroot] [PATCH 7/8] setools: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/setools/setools.mk | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/package/setools/setools.mk b/package/setools/setools.mk
index 6748c95c23..f4644b091c 100644
--- a/package/setools/setools.mk
+++ b/package/setools/setools.mk
@@ -20,6 +20,12 @@ SETOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
 SETOOLS_DEPENDENCIES += python-enum34
 endif
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python3
+else
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python
+endif
+
 define SETOOLS_FIX_SETUP
 	# By default, setup.py will look for libsepol.a in the host machines
 	# /usr/lib directory. This needs to be changed to the staging directory.
-- 
2.14.3

[Buildroot] [PATCH 8/8] libselinux: check host-python version

[Adam Duskett]

Check if host-python3 is selected instead of target-python3.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/libsemanage/libsemanage.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/libsemanage/libsemanage.mk b/package/libsemanage/libsemanage.mk
index 829409f527..b9a98c7b6c 100644
--- a/package/libsemanage/libsemanage.mk
+++ b/package/libsemanage/libsemanage.mk
@@ -37,7 +37,7 @@ HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PREFIX=$(HOST_DIR) \
 	SWIG_LIB="$(HOST_DIR)/share/swig/$(SWIG_VERSION)/"
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSEMANAGE_DEPENDENCIES += host-python3
 HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PYINC="-I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/" \
-- 
2.14.3

[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Thomas Petazzoni]

Hello,

On Tue,  2 Jan 2018 11:33:29 -0500, Adam Duskett wrote:
> Currently, packages that rely on host-python, and are compatible with both
> python2 and 3, only build python3 if python3 is selected for the target.
> 
> This causes a problem in the following scenario:
> 
> - A user does not have a target python selected.
> - Package A depends on host-python3
> - Package B is compatible with host-python and host-python3
> 
> In this scenario, host-python is not needed, as package A and B are
> both compatible with python3. However; both host-python and host-python3 will
> be built because no target-python has been selected.
> 
> To fix this, the following patch series introduces a few minimal changes
> that will allow the user to manually select a host version of python.
> (Note: I have changed only the packages that I personally use and have tested.)

I will have to think a lot to fully grasp the potential consequences of
your changes.

However, there is one thing that I believe is missing in your patch
series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and
BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Thomas Petazzoni]

Hello,

On Tue,  2 Jan 2018 11:33:32 -0500, Adam Duskett wrote:
> python-setuptools is compatible with both python2 and python3, as such,
> there is no need to force python2 as a dependency.
> 
> Signed-off-by: Adam Duskett <aduskett@gmail.com>
> ---
>  package/python-setuptools/python-setuptools.mk | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
> index 28d4f29b5b..a21494a288 100644
> --- a/package/python-setuptools/python-setuptools.mk
> +++ b/package/python-setuptools/python-setuptools.mk
> @@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
>  PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
>  PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
>  
> +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
> +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
> +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
> +else
> +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
> +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
> +endif

So, now, if the user selects:

BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ARROW=y
BR2_PACKAGE_HOST_PYTHON3=y

then the build will break. Indeed, since python2 is selected as the
Python for the target, python-arrow should be built and installed for
python2, so it will expect host-python-setuptools to be built and
installed for host-python.

But because the user has selected BR2_PACKAGE_HOST_PYTHON3,
host-python-setuptools will be installed for host-python3.

So, I believe the build breaks with your change.

Basically the current reasoning of the current Python packaging is:

 - The user selects the target version of Python, and from that we
   derive:

   * That we need to build the same Python version for the host

   * That all host Python modules should be built installed for that
     Python version

 - Optionally, packages may depend on the other host-python{,3} if they
   really need that specific Python version. However, we do not support
   installing third party Python modules for this secondary host Python
   version.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Yann E. MORIN]

Thomas, Adam, All,

On 2018-01-02 21:40 +0100, Thomas Petazzoni spake thusly:
> On Tue,  2 Jan 2018 11:33:29 -0500, Adam Duskett wrote:
> > Currently, packages that rely on host-python, and are compatible with both
> > python2 and 3, only build python3 if python3 is selected for the target.
> > 
> > This causes a problem in the following scenario:
> > 
> > - A user does not have a target python selected.
> > - Package A depends on host-python3
> > - Package B is compatible with host-python and host-python3
> > 
> > In this scenario, host-python is not needed, as package A and B are
> > both compatible with python3. However; both host-python and host-python3 will
> > be built because no target-python has been selected.
> > 
> > To fix this, the following patch series introduces a few minimal changes
> > that will allow the user to manually select a host version of python.
> > (Note: I have changed only the packages that I personally use and have tested.)
> 
> I will have to think a lot to fully grasp the potential consequences of
> your changes.
> 
> However, there is one thing that I believe is missing in your patch
> series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and
> BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.

Furthermore, I believe the prompt should be hidden, because it does not
make sense for the user to actually select the host version for python.

Besides, the current series allows for building both host-python and
host-python3 in the same configuration, and we do not support this for
now (or we would have made it possible for the target variant too).

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 0/8] python: clean up host version logic.

[Thomas Petazzoni]

Hello,

On Tue, 2 Jan 2018 22:40:32 +0100, Yann E. MORIN wrote:

> > However, there is one thing that I believe is missing in your patch
> > series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and
> > BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.  
> 
> Furthermore, I believe the prompt should be hidden, because it does not
> make sense for the user to actually select the host version for python.

If the prompt is not user-selectable, then I don't see the point of the
patch series. Indeed, you can already do:

<pkg>_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)

to use host-python3 if already built as part of the python3 build
process, or fallback to host-python otherwise. We already use this
construct in a number of places.

> Besides, the current series allows for building both host-python and
> host-python3 in the same configuration, and we do not support this for
> now (or we would have made it possible for the target variant too).

Of course we support building both host-python and host-python3. What
we don't support is:

 - python and python3 in the same configuration

 - building third-party host python modules for a host python version
   different from the selected target python version

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Trent Piepho]

On Tue, 2018-01-02 at 11:33 -0500, Adam Duskett wrote:
> python-setuptools is compatible with both python2 and python3, as such,
> there is no need to force python2 as a dependency.
> 
>  
> +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
> +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
> +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
> +else
> +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
> +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
> +endif

From why I see in pkg-python, the default behavior for host packages is
to depend on the default interpreter, which is python3 if enabled
otherwise python2.  Isn't the above block basically doing the same
thing?  Other than using HOST_PYTHON3 rather than PYTHON3.

But shouldn't this be changed in pkg-python?

Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in
use could be eliminated, unless the package needs a specific version,
which appear to be how it's supposed to work.

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Thomas Petazzoni]

Hello,

On Tue, 2 Jan 2018 22:13:15 +0000, Trent Piepho wrote:
> On Tue, 2018-01-02 at 11:33 -0500, Adam Duskett wrote:
> > python-setuptools is compatible with both python2 and python3, as such,
> > there is no need to force python2 as a dependency.
> > 
> >  
> > +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
> > +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
> > +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
> > +else
> > +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
> > +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
> > +endif  
> 
> From why I see in pkg-python, the default behavior for host packages is
> to depend on the default interpreter, which is python3 if enabled
> otherwise python2.  Isn't the above block basically doing the same
> thing?  Other than using HOST_PYTHON3 rather than PYTHON3.
> 
> But shouldn't this be changed in pkg-python?
> 
> Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in
> use could be eliminated, unless the package needs a specific version,
> which appear to be how it's supposed to work.

See my reply to this patch, which explains why it is (IMO) broken.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Trent Piepho]

On Fri, 2018-01-05 at 21:29 +0100, Thomas Petazzoni wrote:
> n Tue, 2 Jan 2018 22:13:15 +0000, Trent Piepho wrote:
> > On Tue, 2018-01-02 at 11:33 -0500, Adam Duskett wrote:
> > > 
> > > +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
> > > +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
> > > +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
> > > +else
> > > +HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
> > > +HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
> > > +endif  
> > 
> > From why I see in pkg-python, the default behavior for host packages is
> > to depend on the default interpreter, which is python3 if enabled
> > otherwise python2.  Isn't the above block basically doing the same
> > thing?  Other than using HOST_PYTHON3 rather than PYTHON3.
> > 
> > But shouldn't this be changed in pkg-python?
> > 
> > Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in
> > use could be eliminated, unless the package needs a specific version,
> > which appear to be how it's supposed to work.
> 
> See my reply to this patch, which explains why it is (IMO) broken.
> 

If I understand correctly, the fundamental problem is that Adam's patch
allows for simultaneously enabling both host python2 and host python3
and building host python packages using both python versions or a
version different than the target python version.

But I think Adam pointed out a real flaw that can be fixed.  It's
caused problems for me as well.

The issue is that the host python version is linked to the target
python version.  Which is fine, if there is a target python.  But the
logic fails if there is no target python.

Right now, one gets this:
Target Python 2 -> Host Python 2
Target Python 3 -> Host Python 3
No Target Python -> Host Python 2

I'd like to build host python 3 utilities and don't have python on the
target.  Can't see a way to configure buildroot to do that.  I can add
a build dep on host-python3 and get the interpreter built, but I can't
make a host-python-package using python3.

I think this could be fixed by making host python config options, which
are forced by the target python configs, so that host python can only
be selected manually if no target python is enabled.

Packages that want to depend on host python should use the host python
config to select which python to depends on, rather than the target
python config as they do now.  If target python is enabled, then host
python == target python so there is no change.  But without target
python it will be possible to build select either host-python3 or host-
python as the python version to be used by host-python-setuptools.

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Thomas Petazzoni]

Hello,

On Tue, 9 Jan 2018 01:47:42 +0000, Trent Piepho wrote:

> If I understand correctly, the fundamental problem is that Adam's patch
> allows for simultaneously enabling both host python2 and host python3
> and building host python packages using both python versions or a
> version different than the target python version.

Correct.

> But I think Adam pointed out a real flaw that can be fixed.  It's
> caused problems for me as well.
> 
> The issue is that the host python version is linked to the target
> python version.  Which is fine, if there is a target python.  But the
> logic fails if there is no target python.
> 
> Right now, one gets this:
> Target Python 2 -> Host Python 2
> Target Python 3 -> Host Python 3
> No Target Python -> Host Python 2

Absolutely.

> I'd like to build host python 3 utilities and don't have python on the
> target.  Can't see a way to configure buildroot to do that.  I can add
> a build dep on host-python3 and get the interpreter built, but I can't
> make a host-python-package using python3.
> 
> I think this could be fixed by making host python config options, which
> are forced by the target python configs, so that host python can only
> be selected manually if no target python is enabled.
> 
> Packages that want to depend on host python should use the host python
> config to select which python to depends on, rather than the target
> python config as they do now.  If target python is enabled, then host
> python == target python so there is no change.  But without target
> python it will be possible to build select either host-python3 or host-
> python as the python version to be used by host-python-setuptools.

Yes, this would be possible. Feel free to make a proposal in this
direction.

In general, we have already discussed several times adding Config.in
options for all host packages. They would be hidden options for most of
the packages, but they would still be visible for some packages (like
we have today in Config.in.host).

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH 3/8] python-setuptools: check host-python version

[Adam Duskett]

python-setuptools is compatible with both python2 and python3, as such,
there is no need to force python2 as a dependency.

Signed-off-by: Adam Duskett <aduskett@gmail.com>
---
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/package/python-setuptools/python-setuptools.mk b/package/python-setuptools/python-setuptools.mk
index 28d4f29b5b..a21494a288 100644
--- a/package/python-setuptools/python-setuptools.mk
+++ b/package/python-setuptools/python-setuptools.mk
@@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
 PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif
+
 # recent setuptools versions require bootstrap.py to be invoked
 # before the standard setup process.
 define PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-	cd  $(@D) && $(HOST_DIR)/bin/python ./bootstrap.py
+	cd  $(@D) && $(HOST_DIR)/bin/$(HOST_PYTHON_SETUPTOOLS_PYLIBVER) ./bootstrap.py
 endef
 
 PYTHON_SETUPTOOLS_PRE_CONFIGURE_HOOKS = PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-- 
2.14.3