All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory
@ 2020-02-04 14:24 Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 1/5] package/Makefile.in: add HOST_BIN_CROSS Arnout Vandecappelle
                   ` (5 more replies)
  0 siblings, 6 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

This series is the start of a "cross-bin" directory that I've mentioned
a couple of times already.

The idea is that host-tools that are meant for cross-building but don't
encode the cross-target in their name can be put in this directory, and
we put this directory in front of PATH for cross-compilation. For
example, the pkg-config wrapper is really a cross-compilation wrapper.
To be able to support building of host tools with it, we need to add
environment variables to force host compilation in HOST_CONFIGURE_OPTS.
By moving the pkg-config wrapper to the cross-bin directory, we can
avoid that: for cross-compilation (i.e. target packages), the cross-bin
dir is put in front of PATH, so the pkg-config wrapper for target is
found. For native compilation (i.e. host packages), the cross-bin dir
is *not* put in PATH, so the normal pkg-config configuration will be
used.

This series does not do that yet for pkg-config itself. It is only done
for python and python3. The idea is to start with something a little
bit less far-reaching that pkg-config. Also, the python3 change is
needed for meson packages that build python modules from C code (e.g.
recent python-gobject). These need to have _PYTHON_SYSCONFIGDATA_NAME
set in the environment, but we can't just export that unconditionally
even for target packages because sometimes (e.g. in libglib2) meson
builds native python modules as part of the build and those should
*not* have _PYTHON_SYSCONFIGDATA_NAME set to the target name... The
python3 wrapper nicely solves this.

The python2 wrapper is added as well for symmetry, and because it
allows us to simplify the inner-python-package definitions.

This series is posted as an RFC, but it can actually be applied as is.
It is a requirement for many packages that use the newly-added
gobject-introspection (e.g. gstreamer1, python-gobject).

Arnout Vandecappelle (5):
  package/Makefile.in: add HOST_BIN_CROSS
  package/python3: install wrapper in HOST_BIN_CROSS
  package/python: install wrapper in HOST_BIN_CROSS
  package/pkg-python: rely on python wrapper in HOST_BIN_CROSS
  package/pkg-meson: use TARGET_MAKE_ENV in configure

 package/Makefile.in               |  5 ++++-
 package/pkg-meson.mk              |  2 +-
 package/pkg-python.mk             | 22 +++-------------------
 package/python/python-wrapper.in  | 10 ++++++++++
 package/python/python.mk          | 17 ++++++++++++++---
 package/python3/python-wrapper.in | 24 ++++++++++++++++++++++++
 package/python3/python3.mk        | 17 ++++++++++++++---
 7 files changed, 70 insertions(+), 27 deletions(-)
 create mode 100644 package/python/python-wrapper.in
 create mode 100644 package/python3/python-wrapper.in

-- 
2.24.1

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 1/5] package/Makefile.in: add HOST_BIN_CROSS
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
@ 2020-02-04 14:24 ` Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 2/5] package/python3: install wrapper in HOST_BIN_CROSS Arnout Vandecappelle
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

Many packages make use of configuration scripts to find various paths
(includes, libraries, etc.). In the best case, pkg-config is used, but
there are dozen of other scripts as well. These scripts have to be
configured or called differently depending on whether we're doing cross
compilation (i.e. building a target package) or native compilation (i.e.
host compilation). Currently, we use various ways to make sure the
correct configure script with the correct settings is used - mostly by
passing environment variables in the configuration step.

To create more convergence and simplify our understanding of what is
going on, let's make a separate directory for cross-compilation
configuration scripts. This is particularly useful for meson, because
meson makes an explicit difference between native and cross compilation,
with a full configuration file for both.

Define HOST_BIN_CROSS as a subdirectory of
$(HOST_DIR)/$(GNU_TARGET_NAME). This makes it easy for the wrapper
script to use a relative path to ../sysroot to find STAGING_DIR, which
makes them relocatable. Note that we can't use
$(HOST_DIR)/$(GNU_TARGET_NAME)/bin, because for an internal toolchain,
gcc and binutils install the cross-executables there without target
prefix. Putting that directory in PATH makes it impossible to do native
compilation at all, because gcc will call 'as' and 'ld' from PATH, i.e.
the cross-tools instead of the native tools.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/Makefile.in | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/package/Makefile.in b/package/Makefile.in
index 285e2837ef..c82e30069e 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -230,6 +230,9 @@ HOST_CFLAGS   += $(HOST_CPPFLAGS)
 HOST_CXXFLAGS += $(HOST_CFLAGS)
 HOST_LDFLAGS  += -L$(HOST_DIR)/lib -Wl,-rpath,$(HOST_DIR)/lib
 
+# Location of scripts/wrappers for cross-compilation
+HOST_BIN_CROSS = $(HOST_DIR)/$(GNU_TARGET_NAME)/cross-bin
+
 # host-intltool should be executed with the system perl, so we save
 # the path to the system perl, before a host-perl built by Buildroot
 # might get installed into $(HOST_DIR)/bin and therefore appears
@@ -241,7 +244,7 @@ export PERL=$(shell which perl)
 # finds this perl module by exporting the proper value for PERL5LIB.
 export PERL5LIB=$(HOST_DIR)/lib/perl
 
-TARGET_MAKE_ENV = PATH=$(BR_PATH)
+TARGET_MAKE_ENV = PATH=$(HOST_BIN_CROSS):$(BR_PATH)
 
 TARGET_CONFIGURE_OPTS = \
 	$(TARGET_MAKE_ENV) \
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 2/5] package/python3: install wrapper in HOST_BIN_CROSS
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 1/5] package/Makefile.in: add HOST_BIN_CROSS Arnout Vandecappelle
@ 2020-02-04 14:24 ` Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 3/5] package/python: " Arnout Vandecappelle
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

When building python packages, we set some environment variables to make
sure python picks up the correct configuration even though we're running
host-python intead of target-python.

Create a wrapper script that does the same, without requiring an
explicit export from Buildroot. This script is installed in
HOST_BIN_CROSS so that it can be found when cross-compiling.

The script is made auto-discovering as much as possible. Since we know
the script will be installed in $(HOST_DIR)/$(GNU_TARGET_NAME)/bin, we
know that STAGING_DIR is at ../sysroot. sysconfigdata can be calculated
based on that. Since this is specific for python3, we don't need the
python3 condition. Only PYTHONPATH is still passed in explicitly, since
that refers to TARGET_DIR - because most python packages are only
installed in target and not in staging. So until we install everything
in staging as well, we keep an explicit TARGET_DIR.

Since the installation hook uses PYTHON3_PATH, move the definition of
that variable higher in the file so it comes before its first use.

It is not really necessary to have a similar script for native python,
because the only environment variable we need in that case is
PYTHONNOUSERSITE=1.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/python3/python-wrapper.in | 24 ++++++++++++++++++++++++
 package/python3/python3.mk        | 17 ++++++++++++++---
 2 files changed, 38 insertions(+), 3 deletions(-)
 create mode 100644 package/python3/python-wrapper.in

diff --git a/package/python3/python-wrapper.in b/package/python3/python-wrapper.in
new file mode 100644
index 0000000000..c2f34f32df
--- /dev/null
+++ b/package/python3/python-wrapper.in
@@ -0,0 +1,24 @@
+#! /bin/sh
+
+STAGING_DIR=$(cd "${0%/*}/../sysroot"; pwd)
+
+SYSCONFIGDATA=""
+for sysconfig in ${STAGING_DIR}/usr/lib/python*/_sysconfigdata__linux_*.py; do
+    if [ -n "$SYSCONFIGDATA" ]; then
+        echo "*** More than one sysconfigdata exists. Buildroot doesn't support mixing targets." 1>&2
+        exit 1
+    fi
+    if [ ! -r "$sysconfig" ]; then
+        echo "*** No sysconfigdata found in $sysconfig." 1>&2
+        exit 1
+    fi
+    SYSCONFIGDATA="$(echo $sysconfig | sed 's%.*/\([^/]*\).py%\1%')"
+done
+
+PYTHONPATH="@PYTHON_PATH@" \
+PYTHONNOUSERSITE=1 \
+_PYTHON_SYSCONFIGDATA_NAME="$SYSCONFIGDATA" \
+_python_sysroot="$STAGING_DIR" \
+_python_prefix=/usr \
+_python_exec_prefix=/usr \
+exec "${0%/*}/../../bin/python3" "$@"
diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 9432f1c59e..6280b4a550 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -239,6 +239,20 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
+# Provided to other packages
+PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/
+
+define PYTHON3_INSTALL_CROSS_WRAPPER
+	mkdir -p $(HOST_BIN_CROSS)
+	sed 's%@PYTHON_PATH@%$(PYTHON3_PATH)%' \
+		package/python3/python-wrapper.in \
+		> $(HOST_BIN_CROSS)/python
+	chmod 0755 $(HOST_BIN_CROSS)/python
+	ln -sf python $(HOST_BIN_CROSS)/python3
+endef
+
+PYTHON3_POST_INSTALL_STAGING_HOOKS += PYTHON3_INSTALL_CROSS_WRAPPER
+
 # Some packages may have build scripts requiring python3, whatever is the
 # python version chosen for the target.
 # Only install the python symlink in the host tree if python3 is enabled
@@ -252,9 +266,6 @@ endef
 HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
 endif
 
-# Provided to other packages
-PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/
-
 # Support for socket.AF_BLUETOOTH
 ifeq ($(BR2_PACKAGE_BLUEZ5_UTILS_HEADERS),y)
 PYTHON3_DEPENDENCIES += bluez5_utils-headers
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 3/5] package/python: install wrapper in HOST_BIN_CROSS
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 1/5] package/Makefile.in: add HOST_BIN_CROSS Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 2/5] package/python3: install wrapper in HOST_BIN_CROSS Arnout Vandecappelle
@ 2020-02-04 14:24 ` Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 4/5] package/pkg-python: rely on python " Arnout Vandecappelle
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

When building python packages, we set some environment variables to make
sure python picks up the correct configuration even though we're running
host-python intead of target-python.

Create a wrapper script that does the same, without requiring an
explicit export from Buildroot. This script is installed in
HOST_BIN_CROSS so that it can be found when cross-compiling.

The script is made auto-discovering as much as possible. Since we know
the script will be installed in $(HOST_DIR)/$(GNU_TARGET_NAME)/bin, we
know that STAGING_DIR is at ../sysroot. PYTHONPATH is still passed in
explicitly, since that refers to TARGET_DIR - because most python
packages are only installed in target and not in staging. So until we
install everything in staging as well, we keep an explicit TARGET_DIR.

sysconfigdata is not needed for python2 because it is not "multilib":
there is only a target sysonfigdata module in STAGING_DIR.

Since the installation hook uses PYTHON_PATH, move the definition of
that variable higher in the file so it comes before its first use.

It is not really necessary to have a similar script for native python,
because the only environment variable we need in that case is
PYTHONNOUSERSITE=1.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/python/python-wrapper.in | 10 ++++++++++
 package/python/python.mk         | 17 ++++++++++++++---
 2 files changed, 24 insertions(+), 3 deletions(-)
 create mode 100644 package/python/python-wrapper.in

diff --git a/package/python/python-wrapper.in b/package/python/python-wrapper.in
new file mode 100644
index 0000000000..c5e6e5481b
--- /dev/null
+++ b/package/python/python-wrapper.in
@@ -0,0 +1,10 @@
+#! /bin/sh
+
+STAGING_DIR=$(cd "${0%/*}/../sysroot"; pwd)
+
+PYTHONPATH="@PYTHON_PATH@" \
+PYTHONNOUSERSITE=1 \
+_python_sysroot="$STAGING_DIR" \
+_python_prefix=/usr \
+_python_exec_prefix=/usr \
+exec "${0%/*}/../../bin/python" "$@"
diff --git a/package/python/python.mk b/package/python/python.mk
index 41a981e3d3..6f2e01386e 100644
--- a/package/python/python.mk
+++ b/package/python/python.mk
@@ -227,6 +227,20 @@ PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLIN
 
 PYTHON_AUTORECONF = YES
 
+# Provided to other packages
+PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/
+
+define PYTHON_INSTALL_CROSS_WRAPPER
+	mkdir -p $(HOST_BIN_CROSS)
+	sed 's%@PYTHON_PATH@%$(PYTHON_PATH)%' \
+		package/python/python-wrapper.in \
+		> $(HOST_BIN_CROSS)/python
+	chmod 0755 $(HOST_BIN_CROSS)/python
+	ln -sf python $(HOST_BIN_CROSS)/python2
+endef
+
+PYTHON_POST_INSTALL_STAGING_HOOKS += PYTHON_INSTALL_CROSS_WRAPPER
+
 # Some packages may have build scripts requiring python2.
 # Only install the python symlink in the host tree if python3 is not enabled
 # for the target, otherwise the default python program may be missing.
@@ -239,9 +253,6 @@ endef
 HOST_PYTHON_POST_INSTALL_HOOKS += HOST_PYTHON_INSTALL_PYTHON_SYMLINK
 endif
 
-# Provided to other packages
-PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
 
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 4/5] package/pkg-python: rely on python wrapper in HOST_BIN_CROSS
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
                   ` (2 preceding siblings ...)
  2020-02-04 14:24 ` [Buildroot] [PATCH 3/5] package/python: " Arnout Vandecappelle
@ 2020-02-04 14:24 ` Arnout Vandecappelle
  2020-02-04 14:24 ` [Buildroot] [PATCH 5/5] package/pkg-meson: use TARGET_MAKE_ENV in configure Arnout Vandecappelle
  2020-02-04 15:15 ` [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
  5 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

Since we now have a python wrapper in HOST_BIN_CROSS, we no longer need
to set all these environment variables from the python-package
infrastructure.

Add HOST_BIN_CROSS to the PATH for target packages, and remove all
environment variables which are already set by the wrapper. Use this
python as the python that is used to run setup.py.

PKG_PYTHON_SYSCONFIGDATA_NAME is no longer used so it can be removed.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/pkg-python.mk | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)

diff --git a/package/pkg-python.mk b/package/pkg-python.mk
index 4ded4fde83..93f24856bf 100644
--- a/package/pkg-python.mk
+++ b/package/pkg-python.mk
@@ -20,21 +20,11 @@
 #
 ################################################################################
 
-define PKG_PYTHON_SYSCONFIGDATA_NAME
-$(basename $(notdir $(wildcard $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/_sysconfigdata__linux_*.py)))
-endef
-
 # Target distutils-based packages
 PKG_PYTHON_DISTUTILS_ENV = \
-	PATH=$(BR_PATH) \
+	PATH=$(HOST_BIN_CROSS):$(BR_PATH) \
 	$(TARGET_CONFIGURE_OPTS) \
 	LDSHARED="$(TARGET_CROSS)gcc -shared" \
-	PYTHONPATH="$(if $(BR2_PACKAGE_PYTHON3),$(PYTHON3_PATH),$(PYTHON_PATH))" \
-	PYTHONNOUSERSITE=1 \
-	_PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" \
-	_python_sysroot=$(STAGING_DIR) \
-	_python_prefix=/usr \
-	_python_exec_prefix=/usr
 
 PKG_PYTHON_DISTUTILS_BUILD_OPTS = \
 	--executable=/usr/bin/python
@@ -58,14 +48,8 @@ HOST_PKG_PYTHON_DISTUTILS_INSTALL_OPTS = \
 
 # Target setuptools-based packages
 PKG_PYTHON_SETUPTOOLS_ENV = \
-	_PYTHON_SYSCONFIGDATA_NAME="$(PKG_PYTHON_SYSCONFIGDATA_NAME)" \
-	PATH=$(BR_PATH) \
+	PATH=$(HOST_BIN_CROSS):$(BR_PATH) \
 	$(TARGET_CONFIGURE_OPTS) \
-	PYTHONPATH="$(if $(BR2_PACKAGE_PYTHON3),$(PYTHON3_PATH),$(PYTHON_PATH))" \
-	PYTHONNOUSERSITE=1 \
-	_python_sysroot=$(STAGING_DIR) \
-	_python_prefix=/usr \
-	_python_exec_prefix=/usr
 
 PKG_PYTHON_SETUPTOOLS_INSTALL_TARGET_OPTS = \
 	--prefix=/usr \
@@ -225,7 +209,7 @@ endif # SETUP_TYPE
 #   - otherwise, we use the one requested by *_NEEDS_HOST_PYTHON.
 #
 ifeq ($(4),target)
-$(2)_PYTHON_INTERPRETER = $$(HOST_DIR)/bin/python
+$(2)_PYTHON_INTERPRETER = $$(HOST_BIN_CROSS)/python
 else
 ifeq ($$($(2)_NEEDS_HOST_PYTHON),)
 $(2)_PYTHON_INTERPRETER = $$(HOST_DIR)/bin/python
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 5/5] package/pkg-meson: use TARGET_MAKE_ENV in configure
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
                   ` (3 preceding siblings ...)
  2020-02-04 14:24 ` [Buildroot] [PATCH 4/5] package/pkg-python: rely on python " Arnout Vandecappelle
@ 2020-02-04 14:24 ` Arnout Vandecappelle
  2020-02-04 15:15 ` [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
  5 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 14:24 UTC (permalink / raw)
  To: buildroot

The configure step doesn't use TARGET_MAKE_ENV, but sets PATH
explicitly. Thus, it missed the update of PATH with HOST_BIN_CROSS.

Instead of adding it explicitly again, just use TARGET_MAKE_ENV. Yes,
it's not really a 'make environment' that we're expecting to use in the
configure step, 'conf environment' would make more sense. However,
TARGET_MAKE_ENV is indeed used in the configure step since it's part of
TARGET_CONFIGURE_OPTS.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
 package/pkg-meson.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/pkg-meson.mk b/package/pkg-meson.mk
index e7eea2aa58..7bb08157f5 100644
--- a/package/pkg-meson.mk
+++ b/package/pkg-meson.mk
@@ -84,7 +84,7 @@ define $(2)_CONFIGURE_CMDS
 	    ) \
 	    package/meson/cross-compilation.conf.in \
 	    > $$($$(PKG)_SRCDIR)/build/cross-compilation.conf
-	PATH=$$(BR_PATH) $$($$(PKG)_CONF_ENV) $$(MESON) \
+	$$(TARGET_MAKE_ENV) $$($$(PKG)_CONF_ENV) $$(MESON) \
 		--prefix=/usr \
 		--libdir=lib \
 		--default-library=$(if $(BR2_STATIC_LIBS),static,shared) \
-- 
2.24.1

^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory
  2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
                   ` (4 preceding siblings ...)
  2020-02-04 14:24 ` [Buildroot] [PATCH 5/5] package/pkg-meson: use TARGET_MAKE_ENV in configure Arnout Vandecappelle
@ 2020-02-04 15:15 ` Arnout Vandecappelle
  2021-05-26 18:45   ` Arnout Vandecappelle
  5 siblings, 1 reply; 8+ messages in thread
From: Arnout Vandecappelle @ 2020-02-04 15:15 UTC (permalink / raw)
  To: buildroot



On 04/02/2020 15:24, Arnout Vandecappelle (Essensium/Mind) wrote:
> This series is the start of a "cross-bin" directory that I've mentioned
> a couple of times already.
> 
> The idea is that host-tools that are meant for cross-building but don't
> encode the cross-target in their name can be put in this directory, and
> we put this directory in front of PATH for cross-compilation. For
> example, the pkg-config wrapper is really a cross-compilation wrapper.
> To be able to support building of host tools with it, we need to add
> environment variables to force host compilation in HOST_CONFIGURE_OPTS.
> By moving the pkg-config wrapper to the cross-bin directory, we can
> avoid that: for cross-compilation (i.e. target packages), the cross-bin
> dir is put in front of PATH, so the pkg-config wrapper for target is
> found. For native compilation (i.e. host packages), the cross-bin dir
> is *not* put in PATH, so the normal pkg-config configuration will be
> used.
> 
> This series does not do that yet for pkg-config itself. It is only done
> for python and python3. The idea is to start with something a little
> bit less far-reaching that pkg-config. Also, the python3 change is
> needed for meson packages that build python modules from C code (e.g.
> recent python-gobject). These need to have _PYTHON_SYSCONFIGDATA_NAME
> set in the environment, but we can't just export that unconditionally
> even for target packages because sometimes (e.g. in libglib2) meson
> builds native python modules as part of the build and those should
> *not* have _PYTHON_SYSCONFIGDATA_NAME set to the target name... The
> python3 wrapper nicely solves this.
> 
> The python2 wrapper is added as well for symmetry, and because it
> allows us to simplify the inner-python-package definitions.
> 
> This series is posted as an RFC, but it can actually be applied as is.
> It is a requirement for many packages that use the newly-added
> gobject-introspection (e.g. gstreamer1, python-gobject).

 Yann noticed that he has posted something similar/related three years ago:

http://lists.busybox.net/pipermail/buildroot/2015-December/147891.html


 Regards,
 Arnout

> 
> Arnout Vandecappelle (5):
>   package/Makefile.in: add HOST_BIN_CROSS
>   package/python3: install wrapper in HOST_BIN_CROSS
>   package/python: install wrapper in HOST_BIN_CROSS
>   package/pkg-python: rely on python wrapper in HOST_BIN_CROSS
>   package/pkg-meson: use TARGET_MAKE_ENV in configure
> 
>  package/Makefile.in               |  5 ++++-
>  package/pkg-meson.mk              |  2 +-
>  package/pkg-python.mk             | 22 +++-------------------
>  package/python/python-wrapper.in  | 10 ++++++++++
>  package/python/python.mk          | 17 ++++++++++++++---
>  package/python3/python-wrapper.in | 24 ++++++++++++++++++++++++
>  package/python3/python3.mk        | 17 ++++++++++++++---
>  7 files changed, 70 insertions(+), 27 deletions(-)
>  create mode 100644 package/python/python-wrapper.in
>  create mode 100644 package/python3/python-wrapper.in
> 

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory
  2020-02-04 15:15 ` [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
@ 2021-05-26 18:45   ` Arnout Vandecappelle
  0 siblings, 0 replies; 8+ messages in thread
From: Arnout Vandecappelle @ 2021-05-26 18:45 UTC (permalink / raw)
  To: buildroot



On 04/02/2020 16:15, Arnout Vandecappelle wrote:
> 
> 
> On 04/02/2020 15:24, Arnout Vandecappelle (Essensium/Mind) wrote:
>> This series is the start of a "cross-bin" directory that I've mentioned
>> a couple of times already.
>>
>> The idea is that host-tools that are meant for cross-building but don't
>> encode the cross-target in their name can be put in this directory, and
>> we put this directory in front of PATH for cross-compilation. For
>> example, the pkg-config wrapper is really a cross-compilation wrapper.
>> To be able to support building of host tools with it, we need to add
>> environment variables to force host compilation in HOST_CONFIGURE_OPTS.
>> By moving the pkg-config wrapper to the cross-bin directory, we can
>> avoid that: for cross-compilation (i.e. target packages), the cross-bin
>> dir is put in front of PATH, so the pkg-config wrapper for target is
>> found. For native compilation (i.e. host packages), the cross-bin dir
>> is *not* put in PATH, so the normal pkg-config configuration will be
>> used.
>>
>> This series does not do that yet for pkg-config itself. It is only done
>> for python and python3. The idea is to start with something a little
>> bit less far-reaching that pkg-config. Also, the python3 change is
>> needed for meson packages that build python modules from C code (e.g.
>> recent python-gobject). These need to have _PYTHON_SYSCONFIGDATA_NAME
>> set in the environment, but we can't just export that unconditionally
>> even for target packages because sometimes (e.g. in libglib2) meson
>> builds native python modules as part of the build and those should
>> *not* have _PYTHON_SYSCONFIGDATA_NAME set to the target name... The
>> python3 wrapper nicely solves this.
>>
>> The python2 wrapper is added as well for symmetry, and because it
>> allows us to simplify the inner-python-package definitions.
>>
>> This series is posted as an RFC, but it can actually be applied as is.
>> It is a requirement for many packages that use the newly-added
>> gobject-introspection (e.g. gstreamer1, python-gobject).

 In the end, it seems this series was not really that important for
gobject-introspection. And it turns out not to work as well as I hoped to begin
with.

 Therefore, I've marked it as Rejected in patchwork.

> 
>  Yann noticed that he has posted something similar/related three years ago:
> 
> http://lists.busybox.net/pipermail/buildroot/2015-December/147891.html

 Maybe another attempt in 2025 then :-)

 Regards,
 Arnout

> 
> 
>  Regards,
>  Arnout
> 
>>
>> Arnout Vandecappelle (5):
>>   package/Makefile.in: add HOST_BIN_CROSS
>>   package/python3: install wrapper in HOST_BIN_CROSS
>>   package/python: install wrapper in HOST_BIN_CROSS
>>   package/pkg-python: rely on python wrapper in HOST_BIN_CROSS
>>   package/pkg-meson: use TARGET_MAKE_ENV in configure
>>
>>  package/Makefile.in               |  5 ++++-
>>  package/pkg-meson.mk              |  2 +-
>>  package/pkg-python.mk             | 22 +++-------------------
>>  package/python/python-wrapper.in  | 10 ++++++++++
>>  package/python/python.mk          | 17 ++++++++++++++---
>>  package/python3/python-wrapper.in | 24 ++++++++++++++++++++++++
>>  package/python3/python3.mk        | 17 ++++++++++++++---
>>  7 files changed, 70 insertions(+), 27 deletions(-)
>>  create mode 100644 package/python/python-wrapper.in
>>  create mode 100644 package/python3/python-wrapper.in
>>

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-05-26 18:45 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-04 14:24 [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
2020-02-04 14:24 ` [Buildroot] [PATCH 1/5] package/Makefile.in: add HOST_BIN_CROSS Arnout Vandecappelle
2020-02-04 14:24 ` [Buildroot] [PATCH 2/5] package/python3: install wrapper in HOST_BIN_CROSS Arnout Vandecappelle
2020-02-04 14:24 ` [Buildroot] [PATCH 3/5] package/python: " Arnout Vandecappelle
2020-02-04 14:24 ` [Buildroot] [PATCH 4/5] package/pkg-python: rely on python " Arnout Vandecappelle
2020-02-04 14:24 ` [Buildroot] [PATCH 5/5] package/pkg-meson: use TARGET_MAKE_ENV in configure Arnout Vandecappelle
2020-02-04 15:15 ` [Buildroot] [PATCH 0/5] [RFC] Introduce cross-bin directory Arnout Vandecappelle
2021-05-26 18:45   ` Arnout Vandecappelle

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.