[PATCH 1/3] populate_sdk_ext: Ensure buildtools doesn't corrupt OECORE_NATIVE_SYSROOT

[PATCH 1/3] populate_sdk_ext: Ensure buildtools doesn't corrupt OECORE_NATIVE_SYSROOT

[Richard Purdie]

buildtools is built as a nativesdk which needs to use OECORE_NATIVE_SYSROOT
for its own purposes and can reset it. Save and restore the value
within the eSDK so the two don't clash.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/populate_sdk_ext.bbclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index fd0da16e7ea..44d99cfb972 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -653,7 +653,10 @@ sdk_ext_postinst() {
 
 		# Make sure when the user sets up the environment, they also get
 		# the buildtools-tarball tools in their path.
+		echo "# Save and reset OECORE_NATIVE_SYSROOT as buildtools may change it" >> $env_setup_script
+		echo "SAVED=\"\$OECORE_NATIVE_SYSROOT\"" >> $env_setup_script
 		echo ". $target_sdk_dir/buildtools/environment-setup*" >> $env_setup_script
+		echo "OECORE_NATIVE_SYSROOT=\"\$SAVED\"" >> $env_setup_script
 	fi
 
 	# Allow bitbake environment setup to be ran as part of this sdk.
-- 
2.25.1

[PATCH 2/3] buildtools: Handle generic environment setup injection

[Richard Purdie]

Currently buildtools has its own hacked environment setup. We added
capability for generic script fragements but this code was never updated
to use it even if several components it contains needs it.

Remove a hardcoded variable in favour of the generic scripts to
avoid bugs in this area in future. The others should probably be
migrated to environment files in future too but there isn't such
a file for those at present.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-core/meta/buildtools-tarball.bb | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index d0f8dd7d7cb..3785941c2cf 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -66,16 +66,19 @@ create_sdk_files_append () {
 	script=${1:-${SDK_OUTPUT}/${SDKPATH}/environment-setup-${SDK_SYS}}
 	touch $script
 	echo 'export PATH=${SDKPATHNATIVE}${bindir_nativesdk}:$PATH' >> $script
-	# In order for the self-extraction script to correctly extract and set up things,
-	# we need a 'OECORE_NATIVE_SYSROOT=xxx' line in environment setup script.
-	# However, buildtools-tarball is inherently a tool set instead of a fully functional SDK,
-	# so instead of exporting the variable, we use a comment here.
-	echo '#OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
-	toolchain_create_sdk_version ${SDK_OUTPUT}/${SDKPATH}/version-${SDK_SYS}
-
+	echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
 	echo 'export GIT_SSL_CAINFO="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
 	echo 'export SSL_CERT_FILE="${SDKPATHNATIVE}${sysconfdir}/ssl/certs/ca-certificates.crt"' >>$script
-	echo 'export OPENSSL_CONF="${SDKPATHNATIVE}${sysconfdir}/ssl/openssl.cnf"' >>$script
+
+	toolchain_create_sdk_version ${SDK_OUTPUT}/${SDKPATH}/version-${SDK_SYS}
+
+	cat >> $script <<EOF
+if [ -d "\$OECORE_NATIVE_SYSROOT/environment-setup.d" ]; then
+	for envfile in \$OECORE_NATIVE_SYSROOT/environment-setup.d/*.sh; do
+		. \$envfile
+	done
+fi
+EOF
 
 	if [ "${SDKMACHINE}" = "i686" ]; then
 		echo 'export NO32LIBS="0"' >>$script
-- 
2.25.1

[PATCH 3/3] uninative: Handle PREMIRRORS generically

[Richard Purdie]

Currently uninative handles SOURCE_MIRROR_URL but not generic PREMIRRORS.
It can handle this better, attempt to iterate PREMIRRORS entries.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/uninative.bbclass | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
index 70799bbf6d9..316c0f0616c 100644
--- a/meta/classes/uninative.bbclass
+++ b/meta/classes/uninative.bbclass
@@ -56,12 +56,17 @@ python uninative_event_fetchloader() {
             # Our games with path manipulation of DL_DIR mean standard PREMIRRORS don't work
             # and we can't easily put 'chksum' into the url path from a url parameter with
             # the current fetcher url handling
-            ownmirror = d.getVar('SOURCE_MIRROR_URL')
-            if ownmirror:
-                localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} ${SOURCE_MIRROR_URL}/uninative/%s/${UNINATIVE_TARBALL}" % chksum)
+            premirrors = bb.fetch2.mirror_from_string(localdata.getVar("PREMIRRORS"))
+            for line in premirrors:
+                try:
+                    (find, replace) = line
+                except ValueError:
+                    continue
+                if find.startswith("http"):
+                    localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % (replace, chksum))
 
             srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum)
-            bb.note("Fetching uninative binary shim from %s" % srcuri)
+            bb.note("Fetching uninative binary shim %s (will check PREMIRRORS first)" % srcuri)
 
             fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
             fetcher.download()
-- 
2.25.1

Re: [OE-core] [PATCH 1/3] populate_sdk_ext: Ensure buildtools doesn't corrupt OECORE_NATIVE_SYSROOT

[Steve Sakoman]

Is this series suitable for dunfell?

Steve

On Fri, Aug 7, 2020 at 10:25 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> buildtools is built as a nativesdk which needs to use OECORE_NATIVE_SYSROOT
> for its own purposes and can reset it. Save and restore the value
> within the eSDK so the two don't clash.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/populate_sdk_ext.bbclass | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
> index fd0da16e7ea..44d99cfb972 100644
> --- a/meta/classes/populate_sdk_ext.bbclass
> +++ b/meta/classes/populate_sdk_ext.bbclass
> @@ -653,7 +653,10 @@ sdk_ext_postinst() {
>
>                 # Make sure when the user sets up the environment, they also get
>                 # the buildtools-tarball tools in their path.
> +               echo "# Save and reset OECORE_NATIVE_SYSROOT as buildtools may change it" >> $env_setup_script
> +               echo "SAVED=\"\$OECORE_NATIVE_SYSROOT\"" >> $env_setup_script
>                 echo ". $target_sdk_dir/buildtools/environment-setup*" >> $env_setup_script
> +               echo "OECORE_NATIVE_SYSROOT=\"\$SAVED\"" >> $env_setup_script
>         fi
>
>         # Allow bitbake environment setup to be ran as part of this sdk.
> --
> 2.25.1
>
> 

Re: [OE-core] [PATCH 1/3] populate_sdk_ext: Ensure buildtools doesn't corrupt OECORE_NATIVE_SYSROOT

[Richard Purdie]

On Sat, 2020-08-08 at 12:12 -1000, Steve Sakoman wrote:
> Is this series suitable for dunfell?

The uninative one is probably a good idea. The buildtools one,
definitely not. This one should be harmless but isn't necessary without
the buildtools one.

Cheers,

Richard

Re: [OE-core] [PATCH 1/3] populate_sdk_ext: Ensure buildtools doesn't corrupt OECORE_NATIVE_SYSROOT

[Steve Sakoman]

On Sun, Aug 9, 2020 at 2:19 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Sat, 2020-08-08 at 12:12 -1000, Steve Sakoman wrote:
> > Is this series suitable for dunfell?
>
> The uninative one is probably a good idea. The buildtools one,
> definitely not. This one should be harmless but isn't necessary without
> the buildtools one.

Thanks Richard!

Steve