[Buildroot] [PATCH] python-lxml: allow build as host package

[Buildroot] [PATCH] python-lxml: allow build as host package

[Carlos Santos]

While currently there is no in-tree Buildroot package which depends on
host-python-lxml, we (DATACOM) have some proprietary modules that use it
in their test scripts.

We tested python-lxml as host package and confirmed that it builds and
works correctly. Someone else might require it, so we are proposing its
inclusion.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>

---
Additional background (DATACOM-specific): so far we circumvent the lack
of host-python-lxml invoking python with an absolute path in the tests
(/usr/bin/python) but this forces us to install python-lxml on all build
hosts and developer workstations.

Change-Id: If51b034e9579e938a4e7470a58689d7eb34a6a3f
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
 package/python-lxml/python-lxml.mk | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/package/python-lxml/python-lxml.mk b/package/python-lxml/python-lxml.mk
index 62c2980..22287b9 100644
--- a/package/python-lxml/python-lxml.mk
+++ b/package/python-lxml/python-lxml.mk
@@ -27,5 +27,9 @@ PYTHON_LXML_DEPENDENCIES = libxml2 libxslt zlib
 PYTHON_LXML_BUILD_OPTS = \
 	--with-xslt-config=$(STAGING_DIR)/usr/bin/xslt-config \
 	--with-xml2-config=$(STAGING_DIR)/usr/bin/xml2-config
+HOST_PYTHON_LXML_BUILD_OPTS = \
+	--with-xslt-config=$(HOST_DIR)/usr/bin/xslt-config \
+	--with-xml2-config=$(HOST_DIR)/usr/bin/xml2-config
 
 $(eval $(python-package))
+$(eval $(host-python-package))
-- 
2.7.4

[Buildroot] [PATCH] python-lxml: allow build as host package

[Thomas Petazzoni]

Hello,

On Mon, 23 Jan 2017 11:51:16 -0200, Carlos Santos wrote:
> While currently there is no in-tree Buildroot package which depends on
> host-python-lxml, we (DATACOM) have some proprietary modules that use it
> in their test scripts.
> 
> We tested python-lxml as host package and confirmed that it builds and
> works correctly. Someone else might require it, so we are proposing its
> inclusion.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>

I remain a little bit uncomfortable with host packages that have no
in-tree user, as it means the code is dead and never tested by our
autobuilder infrastructure.

That being said, lots of aspects of Buildroot are already not tested by
our autobuilder infra, so maybe my argument is not so good after all.

Peter, what is your feeling on this ?

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH v2] python-lxml: allow build as host package

[Carlos Santos]

While currently there is no in-tree Buildroot package which depends on
host-python-lxml, we (DATACOM) have some proprietary modules that use it
in their test scripts.

We tested python-lxml as host package and confirmed that it builds and
works correctly. Someone else might require it, so we are proposing its
inclusion.

Signed-off-by: Carlos Santos <casantos@datacom.ind.br>

---
Changes v1->v2:
- Add explicit host package dependencies, as required after commit
  4bdb067e380e72c3a6cb9c6ccf3791a6cf8661e0 (infra: remove auto
  derivation of host dependencies).

Additional background (DATACOM-specific): so far we circumvent the lack
of host-python-lxml invoking python with an absolute path in the tests
(/usr/bin/python) but this forces us to install python-lxml on all build
hosts and developer workstations.

Change-Id: If51b034e9579e938a4e7470a58689d7eb34a6a3f
Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
---
 package/python-lxml/python-lxml.mk | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/package/python-lxml/python-lxml.mk b/package/python-lxml/python-lxml.mk
index 62c2980..977ec5f 100644
--- a/package/python-lxml/python-lxml.mk
+++ b/package/python-lxml/python-lxml.mk
@@ -21,11 +21,16 @@ PYTHON_LXML_LICENSE_FILES = \
 PYTHON_LXML_SETUP_TYPE = setuptools
 
 PYTHON_LXML_DEPENDENCIES = libxml2 libxslt zlib
+HOST_PYTHON_LXML_DEPENDENCIES = host-libxml2 host-libxslt host-zlib
 
 # python-lxml needs these scripts in order to properly detect libxml2 and
 # libxslt compiler and linker flags
 PYTHON_LXML_BUILD_OPTS = \
 	--with-xslt-config=$(STAGING_DIR)/usr/bin/xslt-config \
 	--with-xml2-config=$(STAGING_DIR)/usr/bin/xml2-config
+HOST_PYTHON_LXML_BUILD_OPTS = \
+	--with-xslt-config=$(HOST_DIR)/usr/bin/xslt-config \
+	--with-xml2-config=$(HOST_DIR)/usr/bin/xml2-config
 
 $(eval $(python-package))
+$(eval $(host-python-package))
-- 
2.7.4

[Buildroot] [PATCH v2] python-lxml: allow build as host package

[Carlos Santos]

> From: "Carlos Santos" <casantos@datacom.ind.br>
> To: buildroot at buildroot.org
> Cc: "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>
> Sent: Wednesday, February 1, 2017 3:51:08 PM
> Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package

> While currently there is no in-tree Buildroot package which depends on
> host-python-lxml, we (DATACOM) have some proprietary modules that use it
> in their test scripts.
> 
> We tested python-lxml as host package and confirmed that it builds and
> works correctly. Someone else might require it, so we are proposing its
> inclusion.

Hello,

Any news about this proposal? I think it could fall into the new policy of
"loosen up a little on accepting host tools"[1] if I add the corresponding
Config.in.host, right? I know I'm pushing here, but host-python-lxml would
help us (DATACOM) to have more reproducible builds.

1. http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user

Carlos Santos (Casantos) - DATACOM, P&D
?Something must be done. This is something. Therefore we must do it.?
(from the BBC series ?Yes, Prime Minister?, ?Power to the people?)

[Buildroot] [PATCH v2] python-lxml: allow build as host package

[Arnout Vandecappelle]

On 14-02-17 12:20, Carlos Santos wrote:
>> From: "Carlos Santos" <casantos@datacom.ind.br>
>> To: buildroot at buildroot.org
>> Cc: "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>
>> Sent: Wednesday, February 1, 2017 3:51:08 PM
>> Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package
> 
>> While currently there is no in-tree Buildroot package which depends on
>> host-python-lxml, we (DATACOM) have some proprietary modules that use it
>> in their test scripts.
>>
>> We tested python-lxml as host package and confirmed that it builds and
>> works correctly. Someone else might require it, so we are proposing its
>> inclusion.
> 
> Hello,
> 
> Any news about this proposal? I think it could fall into the new policy of
> "loosen up a little on accepting host tools"[1] if I add the corresponding
> Config.in.host, right? I know I'm pushing here, but host-python-lxml would
> help us (DATACOM) to have more reproducible builds.
> 
> 1. http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user

 We didn't reach a satisfactory conclusion for this case. We (or at least I :-)
certainly understand your need. But it is really weird to see a completely
random python package appear in the host tools menu. I personally don't mind
that much, but if all 265 host packages would appear in the menu it would get a
little bit overloaded...

 *Some* way to include it in the autobuilders is certainly needed. As written in
the report: "Maybe for these it's not even so important that they are tested in
the autobuilder - almost nobody is going to use it anyway. However, in that case
it's not very useful to have it anyway. Especially because it almost certainly
will break at some point. "

 So, no clear answer I'm afraid...

 Regards,
 Arnout
-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] [PATCH v2] python-lxml: allow build as host package

[Carlos Santos]

> From: "Arnout Vandecappelle" <arnout@mind.be>
> To: "Carlos Santos" <casantos@datacom.ind.br>, buildroot at buildroot.org
> Cc: "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>
> Sent: Thursday, February 16, 2017 8:08:17 PM
> Subject: Re: [Buildroot] [PATCH v2] python-lxml: allow build as host package

> On 14-02-17 12:20, Carlos Santos wrote:
>>> From: "Carlos Santos" <casantos@datacom.ind.br>
>>> To: buildroot at buildroot.org
>>> Cc: "Thomas Petazzoni" <thomas.petazzoni@free-electrons.com>
>>> Sent: Wednesday, February 1, 2017 3:51:08 PM
>>> Subject: [Buildroot] [PATCH v2] python-lxml: allow build as host package
>> 
>>> While currently there is no in-tree Buildroot package which depends on
>>> host-python-lxml, we (DATACOM) have some proprietary modules that use it
>>> in their test scripts.
>>>
>>> We tested python-lxml as host package and confirmed that it builds and
>>> works correctly. Someone else might require it, so we are proposing its
>>> inclusion.
>> 
>> Hello,
>> 
>> Any news about this proposal? I think it could fall into the new policy of
>> "loosen up a little on accepting host tools"[1] if I add the corresponding
>> Config.in.host, right? I know I'm pushing here, but host-python-lxml would
>> help us (DATACOM) to have more reproducible builds.
>> 
>> 1.
>> http://elinux.org/Buildroot:DeveloperDaysFOSDEM2017#Host_packages_with_no_in-tree_user
> 
> We didn't reach a satisfactory conclusion for this case. We (or at least I :-)
> certainly understand your need. But it is really weird to see a completely
> random python package appear in the host tools menu. I personally don't mind
> that much, but if all 265 host packages would appear in the menu it would get a
> little bit overloaded...
> 
> *Some* way to include it in the autobuilders is certainly needed. As written in
> the report: "Maybe for these it's not even so important that they are tested in
> the autobuilder - almost nobody is going to use it anyway. However, in that case
> it's not very useful to have it anyway. Especially because it almost certainly
> will break at some point. "
> 
> So, no clear answer I'm afraid...

Hello,

Could we have a verdict on this, please?

   http://patchwork.ozlabs.org/patch/722644/

-- 
Carlos Santos (Casantos) - DATACOM, P&D
?The greatest triumph that modern PR can offer is the transcendent 
success of having your words and actions judged by your reputation, 
rather than the other way about.? ? Christopher Hitchens

[Buildroot] [PATCH v2] python-lxml: allow build as host package

[Thomas Petazzoni]

Hello,

On Wed,  1 Feb 2017 15:51:08 -0200, Carlos Santos wrote:
> While currently there is no in-tree Buildroot package which depends on
> host-python-lxml, we (DATACOM) have some proprietary modules that use it
> in their test scripts.
> 
> We tested python-lxml as host package and confirmed that it builds and
> works correctly. Someone else might require it, so we are proposing its
> inclusion.
> 
> Signed-off-by: Carlos Santos <casantos@datacom.ind.br>
> 
> ---
> Changes v1->v2:
> - Add explicit host package dependencies, as required after commit
>   4bdb067e380e72c3a6cb9c6ccf3791a6cf8661e0 (infra: remove auto
>   derivation of host dependencies).

In the end, I've applied to master, after adding a Config.in.host file
that makes the host package show up in menuconfig.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com