[PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Richard Purdie]

We have a problem when for example, a glibc 2.27 based system builds some
library like libpopt-native and puts it into sstate then it is reused
on a pre glibc-2.27 system to build something which depends on popt like
rpm-native. This results in an error like:

recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'

In the past we've had this problem with new symbols like getrandom and
getentropy, here its with a more complex symbol where there is an old
version and a newer version.

We've looked into various options, basically we cannot link against our
uninative libc/ld.so since we don't have the right headers or compiler
link libraries. The compiler doesn't allow you to switch in a new set
either, even if we did want to ship them. Shipping a complete compiler,
dev headers and libs also isn't an option.

On the other hand if we follow the ld man page, it does say:

"""
The reasons for allowing undefined symbol references in shared libraries
specified at link time are that:

- A shared library specified at link time may not be the same as the one
  that is available at load time, so the symbol might actually be
  resolvable at load time.
"""

which is exactly this case. By the time the binary runs, it will use
our uninative loader and libc and the symbol will be available.

Therefore we basically have a choice, we get weird intermittent bugs,
we drop uninative entirely, or we pass this option.

If we pass the option, we can drop the other workarounds too.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/uninative.bbclass                     |  4 +++-
 meta/recipes-core/expat/expat/no_getrandom.patch   | 23 ----------------------
 meta/recipes-core/expat/expat_2.2.5.bb             |  2 --
 .../util-linux/util-linux/no_getrandom.patch       | 21 --------------------
 meta/recipes-core/util-linux/util-linux_2.31.bb    |  1 -
 .../python/python3-native_3.5.5.bb                 |  3 +--
 6 files changed, 4 insertions(+), 50 deletions(-)
 delete mode 100644 meta/recipes-core/expat/expat/no_getrandom.patch
 delete mode 100644 meta/recipes-core/util-linux/util-linux/no_getrandom.patch

diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
index 0501dde..23c5314 100644
--- a/meta/classes/uninative.bbclass
+++ b/meta/classes/uninative.bbclass
@@ -9,7 +9,7 @@ UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
 UNINATIVE_DLDIR ?= "${DL_DIR}/uninative/"
 
 # Enabling uninative will change the following variables so they need to go the parsing white list to prevent multiple recipe parsing
-BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS"
+BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS BUILD_LDFLAGS"
 
 addhandler uninative_event_fetchloader
 uninative_event_fetchloader[eventmask] = "bb.event.BuildStarted"
@@ -129,6 +129,8 @@ def enable_uninative(d):
         d.setVar("NATIVELSBSTRING", "universal%s" % oe.utils.host_gcc_version(d))
         d.appendVar("SSTATEPOSTUNPACKFUNCS", " uninative_changeinterp")
         d.appendVarFlag("SSTATEPOSTUNPACKFUNCS", "vardepvalueexclude", "| uninative_changeinterp")
+        d.appendVar("BUILD_LDFLAGS", " -Wl,--allow-shlib-undefined")
+        d.appendVarFlag("BUILD_LDFLAGS", "vardepvalueexclude", "| -Wl,--allow-shlib-undefined")
         d.prependVar("PATH", "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:")
 
 python uninative_changeinterp () {
diff --git a/meta/recipes-core/expat/expat/no_getrandom.patch b/meta/recipes-core/expat/expat/no_getrandom.patch
deleted file mode 100644
index d64f1bf..0000000
--- a/meta/recipes-core/expat/expat/no_getrandom.patch
+++ /dev/null
@@ -1,23 +0,0 @@
-The native version of expat may be used on older systems which dont have glibc 2.25
-and hence don't have getrandom() thanks to uninative. Disable the libc call and
-use the syscall instead to avoid a compatibility issue until we have 2.25 everywhere
-we support with uninative.
-
-RP
-2017/8/14
-
-Upstream-Status: Inappropriate
-
-Index: expat-2.2.3/configure.ac
-===================================================================
---- expat-2.2.3.orig/configure.ac
-+++ expat-2.2.3/configure.ac
-@@ -151,7 +151,7 @@ AC_LINK_IFELSE([AC_LANG_SOURCE([
-   #include <stdlib.h>  /* for NULL */
-   #include <sys/random.h>
-   int main() {
--    return getrandom(NULL, 0U, 0U);
-+    return getrandomBREAKME(NULL, 0U, 0U);
-   }
- ])], [
-     AC_DEFINE([HAVE_GETRANDOM], [1],
diff --git a/meta/recipes-core/expat/expat_2.2.5.bb b/meta/recipes-core/expat/expat_2.2.5.bb
index bce5929..c68a2ef 100644
--- a/meta/recipes-core/expat/expat_2.2.5.bb
+++ b/meta/recipes-core/expat/expat_2.2.5.bb
@@ -11,8 +11,6 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/expat/expat-${PV}.tar.bz2 \
            file://libtool-tag.patch \
 	  "
 
-SRC_URI_append_class-native = " file://no_getrandom.patch"
-
 SRC_URI[md5sum] = "789e297f547980fc9ecc036f9a070d49"
 SRC_URI[sha256sum] = "d9dc32efba7e74f788fcc4f212a43216fc37cf5f23f4c2339664d473353aedf6"
 
diff --git a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch b/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
deleted file mode 100644
index b9fa1ca..0000000
--- a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
+++ /dev/null
@@ -1,21 +0,0 @@
-getrandom() is only available in glibc 2.25+ and uninative may relocate 
-binaries onto systems that don't have this function. For now, force the 
-code to the older codepath until we can come up with a better solution 
-for this kind of issue.
-
-Upstream-Status: Inappropriate
-RP
-2016/8/15
-
-Index: util-linux-2.30/configure.ac
-===================================================================
---- util-linux-2.30.orig/configure.ac
-+++ util-linux-2.30/configure.ac
-@@ -399,7 +399,6 @@ AC_CHECK_FUNCS([ \
- 	getdtablesize \
- 	getexecname \
- 	getmntinfo \
--	getrandom \
- 	getrlimit \
- 	getsgnam \
- 	inotify_init \
diff --git a/meta/recipes-core/util-linux/util-linux_2.31.bb b/meta/recipes-core/util-linux/util-linux_2.31.bb
index a83919b..7f7a78e 100644
--- a/meta/recipes-core/util-linux/util-linux_2.31.bb
+++ b/meta/recipes-core/util-linux/util-linux_2.31.bb
@@ -15,7 +15,6 @@ SRC_URI += "file://configure-sbindir.patch \
             file://display_testname_for_subtest.patch \
             file://avoid_parallel_tests.patch \
 "
-SRC_URI_append_class-native = " file://no_getrandom.patch"
 SRC_URI[md5sum] = "5b6821c403c3cc6e7775f74df1882a20"
 SRC_URI[sha256sum] = "f9be7cdcf4fc5c5064a226599acdda6bdf3d86c640152ba01ea642d91108dc8a"
 
diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
index 70c1512..962ccfc 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
@@ -42,8 +42,7 @@ DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native s
 
 inherit native
 
-# uninative may be used on pre glibc 2.25 systems which don't have getentropy
-EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip ac_cv_func_getentropy=no"
+EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip"
 
 EXTRA_OEMAKE = '\
   LIBC="" \
-- 
2.7.4

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Andre McCurdy]

On Tue, Apr 17, 2018 at 9:44 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> We have a problem when for example, a glibc 2.27 based system builds some
> library like libpopt-native and puts it into sstate then it is reused
> on a pre glibc-2.27 system to build something which depends on popt like
> rpm-native. This results in an error like:
>
> recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'
>
> In the past we've had this problem with new symbols like getrandom and
> getentropy, here its with a more complex symbol where there is an old
> version and a newer version.
>
> We've looked into various options, basically we cannot link against our
> uninative libc/ld.so since we don't have the right headers or compiler
> link libraries. The compiler doesn't allow you to switch in a new set
> either, even if we did want to ship them. Shipping a complete compiler,
> dev headers and libs also isn't an option.

Maybe not an option now, but in theory wouldn't a set of native tools
statically linked with musl and downloadable from a public sstate
server solve all these kinds of issues?

> On the other hand if we follow the ld man page, it does say:
>
> """
> The reasons for allowing undefined symbol references in shared libraries
> specified at link time are that:
>
> - A shared library specified at link time may not be the same as the one
>   that is available at load time, so the symbol might actually be
>   resolvable at load time.
> """
>
> which is exactly this case. By the time the binary runs, it will use
> our uninative loader and libc and the symbol will be available.
>
> Therefore we basically have a choice, we get weird intermittent bugs,
> we drop uninative entirely, or we pass this option.
>
> If we pass the option, we can drop the other workarounds too.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/uninative.bbclass                     |  4 +++-
>  meta/recipes-core/expat/expat/no_getrandom.patch   | 23 ----------------------
>  meta/recipes-core/expat/expat_2.2.5.bb             |  2 --
>  .../util-linux/util-linux/no_getrandom.patch       | 21 --------------------
>  meta/recipes-core/util-linux/util-linux_2.31.bb    |  1 -
>  .../python/python3-native_3.5.5.bb                 |  3 +--
>  6 files changed, 4 insertions(+), 50 deletions(-)
>  delete mode 100644 meta/recipes-core/expat/expat/no_getrandom.patch
>  delete mode 100644 meta/recipes-core/util-linux/util-linux/no_getrandom.patch
>
> diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
> index 0501dde..23c5314 100644
> --- a/meta/classes/uninative.bbclass
> +++ b/meta/classes/uninative.bbclass
> @@ -9,7 +9,7 @@ UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
>  UNINATIVE_DLDIR ?= "${DL_DIR}/uninative/"
>
>  # Enabling uninative will change the following variables so they need to go the parsing white list to prevent multiple recipe parsing
> -BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS"
> +BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS BUILD_LDFLAGS"
>
>  addhandler uninative_event_fetchloader
>  uninative_event_fetchloader[eventmask] = "bb.event.BuildStarted"
> @@ -129,6 +129,8 @@ def enable_uninative(d):
>          d.setVar("NATIVELSBSTRING", "universal%s" % oe.utils.host_gcc_version(d))
>          d.appendVar("SSTATEPOSTUNPACKFUNCS", " uninative_changeinterp")
>          d.appendVarFlag("SSTATEPOSTUNPACKFUNCS", "vardepvalueexclude", "| uninative_changeinterp")
> +        d.appendVar("BUILD_LDFLAGS", " -Wl,--allow-shlib-undefined")
> +        d.appendVarFlag("BUILD_LDFLAGS", "vardepvalueexclude", "| -Wl,--allow-shlib-undefined")
>          d.prependVar("PATH", "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:")
>
>  python uninative_changeinterp () {
> diff --git a/meta/recipes-core/expat/expat/no_getrandom.patch b/meta/recipes-core/expat/expat/no_getrandom.patch
> deleted file mode 100644
> index d64f1bf..0000000
> --- a/meta/recipes-core/expat/expat/no_getrandom.patch
> +++ /dev/null
> @@ -1,23 +0,0 @@
> -The native version of expat may be used on older systems which dont have glibc 2.25
> -and hence don't have getrandom() thanks to uninative. Disable the libc call and
> -use the syscall instead to avoid a compatibility issue until we have 2.25 everywhere
> -we support with uninative.
> -
> -RP
> -2017/8/14
> -
> -Upstream-Status: Inappropriate
> -
> -Index: expat-2.2.3/configure.ac
> -===================================================================
> ---- expat-2.2.3.orig/configure.ac
> -+++ expat-2.2.3/configure.ac
> -@@ -151,7 +151,7 @@ AC_LINK_IFELSE([AC_LANG_SOURCE([
> -   #include <stdlib.h>  /* for NULL */
> -   #include <sys/random.h>
> -   int main() {
> --    return getrandom(NULL, 0U, 0U);
> -+    return getrandomBREAKME(NULL, 0U, 0U);
> -   }
> - ])], [
> -     AC_DEFINE([HAVE_GETRANDOM], [1],
> diff --git a/meta/recipes-core/expat/expat_2.2.5.bb b/meta/recipes-core/expat/expat_2.2.5.bb
> index bce5929..c68a2ef 100644
> --- a/meta/recipes-core/expat/expat_2.2.5.bb
> +++ b/meta/recipes-core/expat/expat_2.2.5.bb
> @@ -11,8 +11,6 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/expat/expat-${PV}.tar.bz2 \
>             file://libtool-tag.patch \
>           "
>
> -SRC_URI_append_class-native = " file://no_getrandom.patch"
> -
>  SRC_URI[md5sum] = "789e297f547980fc9ecc036f9a070d49"
>  SRC_URI[sha256sum] = "d9dc32efba7e74f788fcc4f212a43216fc37cf5f23f4c2339664d473353aedf6"
>
> diff --git a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch b/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
> deleted file mode 100644
> index b9fa1ca..0000000
> --- a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
> +++ /dev/null
> @@ -1,21 +0,0 @@
> -getrandom() is only available in glibc 2.25+ and uninative may relocate
> -binaries onto systems that don't have this function. For now, force the
> -code to the older codepath until we can come up with a better solution
> -for this kind of issue.
> -
> -Upstream-Status: Inappropriate
> -RP
> -2016/8/15
> -
> -Index: util-linux-2.30/configure.ac
> -===================================================================
> ---- util-linux-2.30.orig/configure.ac
> -+++ util-linux-2.30/configure.ac
> -@@ -399,7 +399,6 @@ AC_CHECK_FUNCS([ \
> -       getdtablesize \
> -       getexecname \
> -       getmntinfo \
> --      getrandom \
> -       getrlimit \
> -       getsgnam \
> -       inotify_init \
> diff --git a/meta/recipes-core/util-linux/util-linux_2.31.bb b/meta/recipes-core/util-linux/util-linux_2.31.bb
> index a83919b..7f7a78e 100644
> --- a/meta/recipes-core/util-linux/util-linux_2.31.bb
> +++ b/meta/recipes-core/util-linux/util-linux_2.31.bb
> @@ -15,7 +15,6 @@ SRC_URI += "file://configure-sbindir.patch \
>              file://display_testname_for_subtest.patch \
>              file://avoid_parallel_tests.patch \
>  "
> -SRC_URI_append_class-native = " file://no_getrandom.patch"
>  SRC_URI[md5sum] = "5b6821c403c3cc6e7775f74df1882a20"
>  SRC_URI[sha256sum] = "f9be7cdcf4fc5c5064a226599acdda6bdf3d86c640152ba01ea642d91108dc8a"
>
> diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
> index 70c1512..962ccfc 100644
> --- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
> +++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
> @@ -42,8 +42,7 @@ DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native s
>
>  inherit native
>
> -# uninative may be used on pre glibc 2.25 systems which don't have getentropy
> -EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip ac_cv_func_getentropy=no"
> +EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip"
>
>  EXTRA_OEMAKE = '\
>    LIBC="" \
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Burton, Ross]

On 17 April 2018 at 17:44, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> We have a problem when for example, a glibc 2.27 based system builds some
> library like libpopt-native and puts it into sstate then it is reused
> on a pre glibc-2.27 system to build something which depends on popt like
> rpm-native. This results in an error like:
>
> recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'

dbus-glib-native is failing on a few builders like this:

| /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm-lsb/build/build/tmp/work/x86_64-linux/dbus-glib-native/0.108-r0/build/dbus/.libs/lt-dbus-binding-tool:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found
(required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm-lsb/build/build/tmp/work/x86_64-linux/dbus-glib-native/0.108-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

Using the host libc doesn't seem right surely?

Ross

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Burton, Ross]

The dbus-glib-native one hits a few times but the only other place it
has broken so far is in cmake-native:

bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not
found (required by
TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)

Is expat using rpaths or something and picking the wrong libc?

Ross

On 17 April 2018 at 21:52, Burton, Ross <ross.burton@intel.com> wrote:
> On 17 April 2018 at 17:44, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> We have a problem when for example, a glibc 2.27 based system builds some
>> library like libpopt-native and puts it into sstate then it is reused
>> on a pre glibc-2.27 system to build something which depends on popt like
>> rpm-native. This results in an error like:
>>
>> recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'
>
> dbus-glib-native is failing on a few builders like this:
>
> | /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm-lsb/build/build/tmp/work/x86_64-linux/dbus-glib-native/0.108-r0/build/dbus/.libs/lt-dbus-binding-tool:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found
> (required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-arm-lsb/build/build/tmp/work/x86_64-linux/dbus-glib-native/0.108-r0/recipe-sysroot-native/usr/lib/libexpat.so.1)
>
> Using the host libc doesn't seem right surely?
>
> Ross

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Khem Raj]

On 4/17/18 12:15 PM, Andre McCurdy wrote:
> On Tue, Apr 17, 2018 at 9:44 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>> We have a problem when for example, a glibc 2.27 based system builds some
>> library like libpopt-native and puts it into sstate then it is reused
>> on a pre glibc-2.27 system to build something which depends on popt like
>> rpm-native. This results in an error like:
>>
>> recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'
>>
>> In the past we've had this problem with new symbols like getrandom and
>> getentropy, here its with a more complex symbol where there is an old
>> version and a newer version.
>>
>> We've looked into various options, basically we cannot link against our
>> uninative libc/ld.so since we don't have the right headers or compiler
>> link libraries. The compiler doesn't allow you to switch in a new set
>> either, even if we did want to ship them. Shipping a complete compiler,
>> dev headers and libs also isn't an option.
> 
> Maybe not an option now, but in theory wouldn't a set of native tools
> statically linked with musl and downloadable from a public sstate
> server solve all these kinds of issues?

I think this is an interesting idea, you should open a bugzilla entry 
for it so it can be followed up.

I am of the opinion that we should explore containers and abstract all 
this distro variability once for all.

> 
>> On the other hand if we follow the ld man page, it does say:
>>
>> """
>> The reasons for allowing undefined symbol references in shared libraries
>> specified at link time are that:
>>
>> - A shared library specified at link time may not be the same as the one
>>    that is available at load time, so the symbol might actually be
>>    resolvable at load time.
>> """
>>
>> which is exactly this case. By the time the binary runs, it will use
>> our uninative loader and libc and the symbol will be available.
>>
>> Therefore we basically have a choice, we get weird intermittent bugs,
>> we drop uninative entirely, or we pass this option.
>>
>> If we pass the option, we can drop the other workarounds too.
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> ---
>>   meta/classes/uninative.bbclass                     |  4 +++-
>>   meta/recipes-core/expat/expat/no_getrandom.patch   | 23 ----------------------
>>   meta/recipes-core/expat/expat_2.2.5.bb             |  2 --
>>   .../util-linux/util-linux/no_getrandom.patch       | 21 --------------------
>>   meta/recipes-core/util-linux/util-linux_2.31.bb    |  1 -
>>   .../python/python3-native_3.5.5.bb                 |  3 +--
>>   6 files changed, 4 insertions(+), 50 deletions(-)
>>   delete mode 100644 meta/recipes-core/expat/expat/no_getrandom.patch
>>   delete mode 100644 meta/recipes-core/util-linux/util-linux/no_getrandom.patch
>>
>> diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
>> index 0501dde..23c5314 100644
>> --- a/meta/classes/uninative.bbclass
>> +++ b/meta/classes/uninative.bbclass
>> @@ -9,7 +9,7 @@ UNINATIVE_TARBALL ?= "${BUILD_ARCH}-nativesdk-libc.tar.bz2"
>>   UNINATIVE_DLDIR ?= "${DL_DIR}/uninative/"
>>
>>   # Enabling uninative will change the following variables so they need to go the parsing white list to prevent multiple recipe parsing
>> -BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS"
>> +BB_HASHCONFIG_WHITELIST += "NATIVELSBSTRING SSTATEPOSTUNPACKFUNCS BUILD_LDFLAGS"
>>
>>   addhandler uninative_event_fetchloader
>>   uninative_event_fetchloader[eventmask] = "bb.event.BuildStarted"
>> @@ -129,6 +129,8 @@ def enable_uninative(d):
>>           d.setVar("NATIVELSBSTRING", "universal%s" % oe.utils.host_gcc_version(d))
>>           d.appendVar("SSTATEPOSTUNPACKFUNCS", " uninative_changeinterp")
>>           d.appendVarFlag("SSTATEPOSTUNPACKFUNCS", "vardepvalueexclude", "| uninative_changeinterp")
>> +        d.appendVar("BUILD_LDFLAGS", " -Wl,--allow-shlib-undefined")
>> +        d.appendVarFlag("BUILD_LDFLAGS", "vardepvalueexclude", "| -Wl,--allow-shlib-undefined")
>>           d.prependVar("PATH", "${STAGING_DIR}-uninative/${BUILD_ARCH}-linux${bindir_native}:")
>>
>>   python uninative_changeinterp () {
>> diff --git a/meta/recipes-core/expat/expat/no_getrandom.patch b/meta/recipes-core/expat/expat/no_getrandom.patch
>> deleted file mode 100644
>> index d64f1bf..0000000
>> --- a/meta/recipes-core/expat/expat/no_getrandom.patch
>> +++ /dev/null
>> @@ -1,23 +0,0 @@
>> -The native version of expat may be used on older systems which dont have glibc 2.25
>> -and hence don't have getrandom() thanks to uninative. Disable the libc call and
>> -use the syscall instead to avoid a compatibility issue until we have 2.25 everywhere
>> -we support with uninative.
>> -
>> -RP
>> -2017/8/14
>> -
>> -Upstream-Status: Inappropriate
>> -
>> -Index: expat-2.2.3/configure.ac
>> -===================================================================
>> ---- expat-2.2.3.orig/configure.ac
>> -+++ expat-2.2.3/configure.ac
>> -@@ -151,7 +151,7 @@ AC_LINK_IFELSE([AC_LANG_SOURCE([
>> -   #include <stdlib.h>  /* for NULL */
>> -   #include <sys/random.h>
>> -   int main() {
>> --    return getrandom(NULL, 0U, 0U);
>> -+    return getrandomBREAKME(NULL, 0U, 0U);
>> -   }
>> - ])], [
>> -     AC_DEFINE([HAVE_GETRANDOM], [1],
>> diff --git a/meta/recipes-core/expat/expat_2.2.5.bb b/meta/recipes-core/expat/expat_2.2.5.bb
>> index bce5929..c68a2ef 100644
>> --- a/meta/recipes-core/expat/expat_2.2.5.bb
>> +++ b/meta/recipes-core/expat/expat_2.2.5.bb
>> @@ -11,8 +11,6 @@ SRC_URI = "${SOURCEFORGE_MIRROR}/expat/expat-${PV}.tar.bz2 \
>>              file://libtool-tag.patch \
>>            "
>>
>> -SRC_URI_append_class-native = " file://no_getrandom.patch"
>> -
>>   SRC_URI[md5sum] = "789e297f547980fc9ecc036f9a070d49"
>>   SRC_URI[sha256sum] = "d9dc32efba7e74f788fcc4f212a43216fc37cf5f23f4c2339664d473353aedf6"
>>
>> diff --git a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch b/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
>> deleted file mode 100644
>> index b9fa1ca..0000000
>> --- a/meta/recipes-core/util-linux/util-linux/no_getrandom.patch
>> +++ /dev/null
>> @@ -1,21 +0,0 @@
>> -getrandom() is only available in glibc 2.25+ and uninative may relocate
>> -binaries onto systems that don't have this function. For now, force the
>> -code to the older codepath until we can come up with a better solution
>> -for this kind of issue.
>> -
>> -Upstream-Status: Inappropriate
>> -RP
>> -2016/8/15
>> -
>> -Index: util-linux-2.30/configure.ac
>> -===================================================================
>> ---- util-linux-2.30.orig/configure.ac
>> -+++ util-linux-2.30/configure.ac
>> -@@ -399,7 +399,6 @@ AC_CHECK_FUNCS([ \
>> -       getdtablesize \
>> -       getexecname \
>> -       getmntinfo \
>> --      getrandom \
>> -       getrlimit \
>> -       getsgnam \
>> -       inotify_init \
>> diff --git a/meta/recipes-core/util-linux/util-linux_2.31.bb b/meta/recipes-core/util-linux/util-linux_2.31.bb
>> index a83919b..7f7a78e 100644
>> --- a/meta/recipes-core/util-linux/util-linux_2.31.bb
>> +++ b/meta/recipes-core/util-linux/util-linux_2.31.bb
>> @@ -15,7 +15,6 @@ SRC_URI += "file://configure-sbindir.patch \
>>               file://display_testname_for_subtest.patch \
>>               file://avoid_parallel_tests.patch \
>>   "
>> -SRC_URI_append_class-native = " file://no_getrandom.patch"
>>   SRC_URI[md5sum] = "5b6821c403c3cc6e7775f74df1882a20"
>>   SRC_URI[sha256sum] = "f9be7cdcf4fc5c5064a226599acdda6bdf3d86c640152ba01ea642d91108dc8a"
>>
>> diff --git a/meta/recipes-devtools/python/python3-native_3.5.5.bb b/meta/recipes-devtools/python/python3-native_3.5.5.bb
>> index 70c1512..962ccfc 100644
>> --- a/meta/recipes-devtools/python/python3-native_3.5.5.bb
>> +++ b/meta/recipes-devtools/python/python3-native_3.5.5.bb
>> @@ -42,8 +42,7 @@ DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native s
>>
>>   inherit native
>>
>> -# uninative may be used on pre glibc 2.25 systems which don't have getentropy
>> -EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip ac_cv_func_getentropy=no"
>> +EXTRA_OECONF_append = " --bindir=${bindir}/${PN} --without-ensurepip"
>>
>>   EXTRA_OEMAKE = '\
>>     LIBC="" \
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Richard Purdie]

On Tue, 2018-04-17 at 12:15 -0700, Andre McCurdy wrote:
> On Tue, Apr 17, 2018 at 9:44 AM, Richard Purdie
> > We've looked into various options, basically we cannot link against
> > our uninative libc/ld.so since we don't have the right headers or
> > compiler link libraries. The compiler doesn't allow you to switch
> > in a new set either, even if we did want to ship them. Shipping a
> > complete compiler, dev headers and libs also isn't an option.
>
> Maybe not an option now, but in theory wouldn't a set of native tools
> statically linked with musl and downloadable from a public sstate
> server solve all these kinds of issues?

There are a few ways you could solve this problem in theory. The
trouble is you need to build these native tools somehow and to link
against musl, you'd need a cross build more similar to "nativesdk" than
"native" as you can no longer use the host gcc.

We've looked at shoe-horning nativesdk into native before but it gets
very very messy very quickly as they're really not equivalent. We could
of course create some new native-like variant but again, messy very
quickly, particularly trying to get the sstate checksums to match.

You can give up on the native checksums matching or coerce them somehow
but again, very messy very quickly.

uninative works surprisingly well for how "simple" it is, it does have
some rough edges.

I have found one further issue with this patch and the need to tweak it
a little further, I'll send another patch shortly related to that once
we test it a bit more. That said, if these two changes do work, they
fix one of the big flaws in uninative and should make it a lot more
workable/usable.

Cheers,

Richard

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Richard Purdie]

On Wed, 2018-04-18 at 00:49 -0700, Khem Raj wrote:
> 
> On 4/17/18 12:15 PM, Andre McCurdy wrote:
> > 
> > On Tue, Apr 17, 2018 at 9:44 AM, Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > Maybe not an option now, but in theory wouldn't a set of native
> > tools
> > statically linked with musl and downloadable from a public sstate
> > server solve all these kinds of issues?
>
> I think this is an interesting idea, you should open a bugzilla entry
> for it so it can be followed up.

Its a nice idea and I can see the attraction but I can't see how you'd
make it work in practise.

> I am of the opinion that we should explore containers and abstract
> all this distro variability once for all.

I'd like to see this one explored. The trouble is the root privs
problem for containers, I don't know if that is still an issue?

The other thing I think we should explore urgently is an sstate
artefact equivalence mapping server.

That might get us huge amounts of sstate reuse, particularly given our
reproducibility improvements and mean that the bootstrap times become
less of a factor.

That along with a public central sstate mirror for the core of the
system might make a really interesting step forward for the project.

If I had to spend time on anything, I think this would be the next big
win for the project.

Cheers,

Richard

Re: [PATCH] uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

[Richard Purdie]

On Tue, 2018-04-17 at 21:57 +0100, Burton, Ross wrote:
> The dbus-glib-native one hits a few times but the only other place it
> has broken so far is in cmake-native:
> 
> bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not
> found (required by
> TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-
> native/usr/lib/libexpat.so.1)
> 
> Is expat using rpaths or something and picking the wrong libc?

I've sent a followup patch which should address these issues, thanks.

Cheers,

Richard