[Buildroot] Ensafe header/library path issue with <pkg>_OVERRIDE_SRCDIR

[Buildroot] Ensafe header/library path issue with <pkg>_OVERRIDE_SRCDIR

[Jörg Krause]

Hi,

I have an issue when using an autotools package cloned with git. The
clone is added to <pkg>_OVERRIDE_SRCDIR, e.g.

LIBUPNPP_OVERRIDE_SRCDIR=/home/me/override/libupnpp

As this git clone has no configure file autoreconf needs to be run
first. This is currently not supported by the autotools infrastructure.
So I have to choices:

1) Add <pkg>_AUTORECONF=YES
2) Add <pkg>_PRE_CONFIGURE_HOOKS which runs the packages autogen.sh

The first option works fine and is for sure the prefered way.

However, If I choose for curiosity the second option, I run into unsafe
header/library issue for the package libupnpp when doing a "libtool
install" step:

```
make DESTDIR=/home/buildroot/output/host/usr/x86_64-buildroot-linux-
musl/sysroot install -C /home/buildroot/output/build/libupnpp/
?/bin/sh ./libtool???--mode=install /usr/bin/install -c???libupnpp.la
'/home/buildroot/output/host/usr/x86_64-buildroot-linux-
musl/sysroot/usr/lib'

libtool: warning: relinking 'libupnpp.la'
libtool: install: [..]

x86_64-linux-musl-g++: ERROR: unsafe header/library path used in cross-
compilation: '/usr/lib'
libtool:???error: error: relink 'libupnpp.la' with the above command
before installing it
make[2]: *** [Makefile:562: install-libLTLIBRARIES] Error 1
```

I suppose the first option is working as Buildroot patches libtool,
whereas for the second option the host libtool is executed, right? I've
read some post on different mailing lists remarking that libtool has
some issues with cross-compiling.

Is it possible for Buildroot to detect if autoreconf has to be run for
override sources?

Or is it problem of the package and it can fixed by adding some crucial
autoconf/libtool flags? As I did not cited the complete build log, the
steps to reproduce the issue are described below...

Otherwise, I would suggest to add a note to the manual that in case for
autotools packages clone from a repository, a <pkg>_AUTORECONF=YES has
to be added the <pkg>.mk file manually.

---

Steps to reproduce this issue:

1) In the override directory:
? ?git clone?https://github.com/medoc92/libupnpp.git

2) Add the libupnpp override path to the local.mk in the Buildroot dir.

3) Minimal defconfig:
BR2_x86_64=y
BR2_x86_corei7=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_MUSL_CROSS=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_LIBUPNPP=y

4) Add to libupnpp.mk:
define LIBUPNPP_RUN_AUTOGEN
	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
endef
LIBUPNPP_PRE_CONFIGURE_HOOKS += LIBUPNPP_RUN_AUTOGEN

---

Best regards
J?rg Krause

[Buildroot] Ensafe header/library path issue with <pkg>_OVERRIDE_SRCDIR

[Arnout Vandecappelle]

On 13-09-16 22:29, J?rg Krause wrote:
> Hi,
> 
> I have an issue when using an autotools package cloned with git. The
> clone is added to <pkg>_OVERRIDE_SRCDIR, e.g.
> 
> LIBUPNPP_OVERRIDE_SRCDIR=/home/me/override/libupnpp
> 
> As this git clone has no configure file autoreconf needs to be run
> first. This is currently not supported by the autotools infrastructure.
> So I have to choices:
> 
> 1) Add <pkg>_AUTORECONF=YES
> 2) Add <pkg>_PRE_CONFIGURE_HOOKS which runs the packages autogen.sh

 Or the third choice: run autogen manually (once) in your override source dir.
This way the libtool patch will still be applied and your worries are over. And
it completely avoids the need to patch buildroot just because you have an
override sourcedir.

> 
> The first option works fine and is for sure the prefered way.
> 
> However, If I choose for curiosity the second option, I run into unsafe
> header/library issue for the package libupnpp when doing a "libtool
> install" step:
> 
> ```
> make DESTDIR=/home/buildroot/output/host/usr/x86_64-buildroot-linux-
> musl/sysroot install -C /home/buildroot/output/build/libupnpp/
>  /bin/sh ./libtool   --mode=install /usr/bin/install -c   libupnpp.la
> '/home/buildroot/output/host/usr/x86_64-buildroot-linux-
> musl/sysroot/usr/lib'
> 
> libtool: warning: relinking 'libupnpp.la'
> libtool: install: [..]
> 
> x86_64-linux-musl-g++: ERROR: unsafe header/library path used in cross-
> compilation: '/usr/lib'
> libtool:   error: error: relink 'libupnpp.la' with the above command
> before installing it
> make[2]: *** [Makefile:562: install-libLTLIBRARIES] Error 1
> ```
> 
> I suppose the first option is working as Buildroot patches libtool,
> whereas for the second option the host libtool is executed, right? I've
> read some post on different mailing lists remarking that libtool has
> some issues with cross-compiling.

 Yes, that seems like a good assumption.

> 
> Is it possible for Buildroot to detect if autoreconf has to be run for
> override sources?

 Yes: use _AUTORECONF = YES. That will make sure that libtool is patched after
autoreconf has run, and it will also add the required dependencies to the package.

> 
> Or is it problem of the package and it can fixed by adding some crucial
> autoconf/libtool flags? As I did not cited the complete build log, the
> steps to reproduce the issue are described below...

 libtool must be patched. If you insist on not using the AUTORECONF macro, then
you should redo everything that is done by the AUTORECONF macro in your custom
hooks. _PRE_CONFIGURE_HOOKS += LIBTOOL_PATCH_HOOK is enough. But also don't
forget to do the gettextize before the autoreconf if necessary, and to add
automake etc. to the dependencies.


> 
> Otherwise, I would suggest to add a note to the manual that in case for
> autotools packages clone from a repository, a <pkg>_AUTORECONF=YES has
> to be added the <pkg>.mk file manually.

 I'm not sure what you're asking: adding a note to the manual to say that
_AUTORECONF = YES must be added when configure is missing? Something like:

(current):
* +LIBFOO_AUTORECONF+, tells whether the package should
  be autoreconfigured or not (i.e. if the configure script and
  Makefile.in files should be re-generated by re-running autoconf,
  automake, libtool, etc.). Valid values are +YES+ and
  +NO+. By default, the value is +NO+
(add to it):
  Set this variable to YES when the configure script is missing (e.g.
  the source is fetched from a repository rather than a release tarball),
  or when configure.ac or Makefile.am is patched.

 Or did you have a different place in mind?

 Regards,
 Arnout

> 
> ---
> 
> Steps to reproduce this issue:
> 
> 1) In the override directory:
>    git clone https://github.com/medoc92/libupnpp.git
> 
> 2) Add the libupnpp override path to the local.mk in the Buildroot dir.
> 
> 3) Minimal defconfig:
> BR2_x86_64=y
> BR2_x86_corei7=y
> BR2_TOOLCHAIN_EXTERNAL=y
> BR2_TOOLCHAIN_EXTERNAL_MUSL_CROSS=y
> BR2_INIT_NONE=y
> BR2_SYSTEM_BIN_SH_NONE=y
> # BR2_PACKAGE_BUSYBOX is not set
> BR2_PACKAGE_LIBUPNPP=y
> 
> 4) Add to libupnpp.mk:
> define LIBUPNPP_RUN_AUTOGEN
> 	cd $(@D) && PATH=$(BR_PATH) ./autogen.sh
> endef
> LIBUPNPP_PRE_CONFIGURE_HOOKS += LIBUPNPP_RUN_AUTOGEN
> 
> ---
> 
> Best regards
> J?rg Krause
> 
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] Unsafe header/library path issue with <pkg>_OVERRIDE_SRCDIR

[Jörg Krause]

Hi,

On Mi, 2016-09-14 at 02:13 +0200, Arnout Vandecappelle wrote:
> 
> On 13-09-16 22:29, J?rg Krause wrote:
> > 
> > Hi,
> > 
> > I have an issue when using an autotools package cloned with git.
> > The
> > clone is added to <pkg>_OVERRIDE_SRCDIR, e.g.
> > 
> > LIBUPNPP_OVERRIDE_SRCDIR=/home/me/override/libupnpp
> > 
> > As this git clone has no configure file autoreconf needs to be run
> > first. This is currently not supported by the autotools
> > infrastructure.
> > So I have to choices:
> > 
> > 1) Add <pkg>_AUTORECONF=YES
> > 2) Add <pkg>_PRE_CONFIGURE_HOOKS which runs the packages autogen.sh
> 
> ?Or the third choice: run autogen manually (once) in your override
> source dir.
> This way the libtool patch will still be applied and your worries are
> over. And
> it completely avoids the need to patch buildroot just because you
> have an
> override sourcedir.

Yes, that would propably be the best option. However, Buildroot does
not apply the libtool patch for the override source directory build -
there is no "Patching libtool" message printed and the install process
aborts with the unsafe header/library error.

> > 
> > 
> > The first option works fine and is for sure the prefered way.
> > 
> > However, If I choose for curiosity the second option, I run into
> > unsafe
> > header/library issue for the package libupnpp when doing a "libtool
> > install" step:
> > 
> > ```
> > make DESTDIR=/home/buildroot/output/host/usr/x86_64-buildroot-
> > linux-
> > musl/sysroot install -C /home/buildroot/output/build/libupnpp/
> > ?/bin/sh ./libtool???--mode=install /usr/bin/install
> > -c???libupnpp.la
> > '/home/buildroot/output/host/usr/x86_64-buildroot-linux-
> > musl/sysroot/usr/lib'
> > 
> > libtool: warning: relinking 'libupnpp.la'
> > libtool: install: [..]
> > 
> > x86_64-linux-musl-g++: ERROR: unsafe header/library path used in
> > cross-
> > compilation: '/usr/lib'
> > libtool:???error: error: relink 'libupnpp.la' with the above
> > command
> > before installing it
> > make[2]: *** [Makefile:562: install-libLTLIBRARIES] Error 1
> > ```
> > 
> > I suppose the first option is working as Buildroot patches libtool,
> > whereas for the second option the host libtool is executed, right?
> > I've
> > read some post on different mailing lists remarking that libtool
> > has
> > some issues with cross-compiling.
> 
> ?Yes, that seems like a good assumption.
> 
> > 
> > 
> > Is it possible for Buildroot to detect if autoreconf has to be run
> > for
> > override sources?
> 
> ?Yes: use _AUTORECONF = YES. That will make sure that libtool is
> patched after
> autoreconf has run, and it will also add the required dependencies to
> the package.

Sorry, but I meant without setting AUTORECONF. I would prefer not to
alter the .mk file for each override package.

> > 
> > 
> > Or is it problem of the package and it can fixed by adding some
> > crucial
> > autoconf/libtool flags? As I did not cited the complete build log,
> > the
> > steps to reproduce the issue are described below...
> 
> ?libtool must be patched. If you insist on not using the AUTORECONF
> macro, then
> you should redo everything that is done by the AUTORECONF macro in
> your custom
> hooks. _PRE_CONFIGURE_HOOKS += LIBTOOL_PATCH_HOOK is enough. But also
> don't
> forget to do the gettextize before the autoreconf if necessary, and
> to add
> automake etc. to the dependencies.

I see! This seems to much burden for a package developer. So using some
mechanisms which takes care of all the necessary hooks is fine.

> 
> > 
> > 
> > Otherwise, I would suggest to add a note to the manual that in case
> > for
> > autotools packages clone from a repository, a <pkg>_AUTORECONF=YES
> > has
> > to be added the <pkg>.mk file manually.
> 
> ?I'm not sure what you're asking: adding a note to the manual to say
> that
> _AUTORECONF = YES must be added when configure is missing? Something
> like:
> 
> (current):
> * +LIBFOO_AUTORECONF+, tells whether the package should
> ? be autoreconfigured or not (i.e. if the configure script and
> ? Makefile.in files should be re-generated by re-running autoconf,
> ? automake, libtool, etc.). Valid values are +YES+ and
> ? +NO+. By default, the value is +NO+
> (add to it):
> ? Set this variable to YES when the configure script is missing (e.g.
> ? the source is fetched from a repository rather than a release
> tarball),
> ? or when configure.ac or Makefile.am is patched.
> 
> ?Or did you have a different place in mind?

Looks good to me, many thanks! Do you like to add it to the manual?

Best regards
J?rg Krause

[Buildroot] Unsafe header/library path issue with <pkg>_OVERRIDE_SRCDIR

[Arnout Vandecappelle]

On 14-09-16 23:30, J?rg Krause wrote:
> Hi,
> 
> On Mi, 2016-09-14 at 02:13 +0200, Arnout Vandecappelle wrote:
>>
>> On 13-09-16 22:29, J?rg Krause wrote:
>>>
>>> Hi,
>>>
>>> I have an issue when using an autotools package cloned with git.
>>> The
>>> clone is added to <pkg>_OVERRIDE_SRCDIR, e.g.
>>>
>>> LIBUPNPP_OVERRIDE_SRCDIR=/home/me/override/libupnpp
>>>
>>> As this git clone has no configure file autoreconf needs to be run
>>> first. This is currently not supported by the autotools
>>> infrastructure.
>>> So I have to choices:
>>>
>>> 1) Add <pkg>_AUTORECONF=YES
>>> 2) Add <pkg>_PRE_CONFIGURE_HOOKS which runs the packages autogen.sh
>>
>>  Or the third choice: run autogen manually (once) in your override
>> source dir.
>> This way the libtool patch will still be applied and your worries are
>> over. And
>> it completely avoids the need to patch buildroot just because you
>> have an
>> override sourcedir.
> 
> Yes, that would propably be the best option. However, Buildroot does
> not apply the libtool patch for the override source directory build -
> there is no "Patching libtool" message printed and the install process
> aborts with the unsafe header/library error.

 Ah, that's right, the libtool patch is a post-patch hook.

 Perhaps it would be better to always make it a pre-configure hook.

[snip]
>>> Is it possible for Buildroot to detect if autoreconf has to be run
>>> for
>>> override sources?
>>
>>  Yes: use _AUTORECONF = YES. That will make sure that libtool is
>> patched after
>> autoreconf has run, and it will also add the required dependencies to
>> the package.
> 
> Sorry, but I meant without setting AUTORECONF. I would prefer not to
> alter the .mk file for each override package.

 Er, your alternative was to add a PRE_CONFIGURE_HOOK, so you still need to
change the package, no?


 Regards,
 Arnout

[snip]


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF