[PATCH 1/7] toolchain-scripts: run post-relocate scripts for every environment

[PATCH 1/7] toolchain-scripts: run post-relocate scripts for every environment

[Ross Burton]

SDKs for multilib configurations have multiple environment scripts, so
re-arrange the post-relocate hook invocation so that it runs the post-relocate
hooks after sourcing each environment script.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/classes/toolchain-scripts.bbclass | 46 +++++++++++++++++-----------------
 1 file changed, 23 insertions(+), 23 deletions(-)

diff --git a/meta/classes/toolchain-scripts.bbclass b/meta/classes/toolchain-scripts.bbclass
index 6d1ba69473a..1a2ec4f3b29 100644
--- a/meta/classes/toolchain-scripts.bbclass
+++ b/meta/classes/toolchain-scripts.bbclass
@@ -128,30 +128,30 @@ toolchain_create_post_relocate_script() {
 	touch $relocate_script
 
 	cat >> $relocate_script <<EOF
-# Source top-level SDK env scripts in case they are needed for the relocate
-# scripts.
-for env_setup_script in ${env_dir}/environment-setup-*; do
-    . \$env_setup_script
-    status=\$?
-    if [ \$status != 0 ]; then
-        echo "\$0: Failed to source \$env_setup_script with status \$status"
-        exit \$status
-    fi
-done
-
 if [ -d "${SDKPATHNATIVE}/post-relocate-setup.d/" ]; then
-    for s in ${SDKPATHNATIVE}/post-relocate-setup.d/*; do
-        if [ ! -x \$s ]; then
-            continue
-        fi
-        \$s "\$1"
-        status=\$?
-        if [ \$status != 0 ]; then
-            echo "post-relocate command \"\$s \$1\" failed with status \$status" >&2
-            exit \$status
-        fi
-    done
-    rm -rf "${SDKPATHNATIVE}/post-relocate-setup.d"
+	# Source top-level SDK env scripts in case they are needed for the relocate
+	# scripts.
+	for env_setup_script in ${env_dir}/environment-setup-*; do
+		. \$env_setup_script
+		status=\$?
+		if [ \$status != 0 ]; then
+			echo "\$0: Failed to source \$env_setup_script with status \$status"
+			exit \$status
+		fi
+
+		for s in ${SDKPATHNATIVE}/post-relocate-setup.d/*; do
+			if [ ! -x \$s ]; then
+				continue
+			fi
+			\$s "\$1"
+			status=\$?
+			if [ \$status != 0 ]; then
+				echo "post-relocate command \"\$s \$1\" failed with status \$status" >&2
+				exit \$status
+			fi
+		done
+	done
+	rm -rf "${SDKPATHNATIVE}/post-relocate-setup.d"
 fi
 EOF
 }
-- 
2.11.0

[PATCH 2/7] meson: use a sensible default for libdir when cross-compiling

[Ross Burton]

If --libdir isn't passed to Meson a default is used, but as this default value
changes depending on the host (whether it is Debian-like, or has /usr/lib64)
this isn't appropriate for cross builds.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/meson/meson.inc              |  1 +
 .../meson/meson/cross-libdir.patch                 | 41 ++++++++++++++++++++++
 2 files changed, 42 insertions(+)
 create mode 100644 meta/recipes-devtools/meson/meson/cross-libdir.patch

diff --git a/meta/recipes-devtools/meson/meson.inc b/meta/recipes-devtools/meson/meson.inc
index ef8c5aefa52..d37afc3b0dc 100644
--- a/meta/recipes-devtools/meson/meson.inc
+++ b/meta/recipes-devtools/meson/meson.inc
@@ -11,6 +11,7 @@ SRC_URI = "https://github.com/mesonbuild/meson/releases/download/${PV}/meson-${P
            file://0003-native_bindir.patch \
            file://0001-python-module-do-not-manipulate-the-environment-when.patch \
            file://disable-rpath-handling.patch \
+           file://cross-libdir.patch \
            "
 SRC_URI[sha256sum] = "fb0395c4ac208eab381cd1a20571584bdbba176eb562a7efa9cb17cace0e1551"
 SRC_URI[md5sum] = "3c35b91e8040901034809576d54007c6"
diff --git a/meta/recipes-devtools/meson/meson/cross-libdir.patch b/meta/recipes-devtools/meson/meson/cross-libdir.patch
new file mode 100644
index 00000000000..2bd4fb3f9e3
--- /dev/null
+++ b/meta/recipes-devtools/meson/meson/cross-libdir.patch
@@ -0,0 +1,41 @@
+Upstream-Status: Backport
+Signed-off-by: Ross Burton <ross.burton@intel.com>
+
+From ce24f0d6a9be22ba61d80cd254e0409de2eaffd7 Mon Sep 17 00:00:00 2001
+From: Jussi Pakkanen <jpakkane@gmail.com>
+Date: Thu, 27 Dec 2018 23:43:35 +0200
+Subject: [PATCH] Default libdir is "lib" when cross compiling. Closes #2535.
+
+---
+ docs/markdown/snippets/crosslib.md |  7 +++++++
+ mesonbuild/coredata.py             |  8 ++++++++
+ run_unittests.py                   | 13 +++++++++++++
+ 3 files changed, 28 insertions(+)
+ create mode 100644 docs/markdown/snippets/crosslib.md
+
+diff --git a/mesonbuild/coredata.py b/mesonbuild/coredata.py
+index d70c23038c..4e2f3e008c 100644
+--- a/mesonbuild/coredata.py
++++ b/mesonbuild/coredata.py
+@@ -299,6 +299,7 @@ def __init__(self, options):
+         # Only to print a warning if it changes between Meson invocations.
+         self.pkgconf_envvar = os.environ.get('PKG_CONFIG_PATH', '')
+         self.config_files = self.__load_config_files(options.native_file)
++        self.libdir_cross_fixup()
+ 
+     @staticmethod
+     def __load_config_files(filenames):
+@@ -348,6 +349,13 @@ def __load_cross_file(filename):
+ 
+         raise MesonException('Cannot find specified cross file: ' + filename)
+ 
++    def libdir_cross_fixup(self):
++        # By default set libdir to "lib" when cross compiling since
++        # getting the "system default" is always wrong on multiarch
++        # platforms as it gets a value like lib/x86_64-linux-gnu.
++        if self.cross_file is not None:
++            self.builtins['libdir'].value = 'lib'
++
+     def sanitize_prefix(self, prefix):
+         if not os.path.isabs(prefix):
+             raise MesonException('prefix value {!r} must be an absolute path'
-- 
2.11.0

[PATCH 3/7] meson: fix nativesdk-meson for multilib SDKs

[Ross Burton]

Multilib SDKs differ only in the environment variables set, so nativesdk-meson's
setup script needs to write a cross file for each environment.

Rename the shipped meson.cross to meson.cross.template, as it cannot be used
directly.  Now that post-relocate scripts are called once for each environment,
the generated meson.cross can be prefixed with TARGET_PREFIX to ensure it is
unique.

Finally rewrite the setup script to use string.Template to perform the expansion
instead of hand-coding the logic.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/meson/meson/meson-setup.py   | 69 ++++++----------------
 meta/recipes-devtools/meson/meson/meson-wrapper    |  2 +-
 .../meson/nativesdk-meson_0.49.0.bb                | 41 +++++--------
 3 files changed, 36 insertions(+), 76 deletions(-)

diff --git a/meta/recipes-devtools/meson/meson/meson-setup.py b/meta/recipes-devtools/meson/meson/meson-setup.py
index a9749eae9d4..808e2a062f6 100755
--- a/meta/recipes-devtools/meson/meson/meson-setup.py
+++ b/meta/recipes-devtools/meson/meson/meson-setup.py
@@ -1,62 +1,31 @@
 #!/usr/bin/env python3
 
 import os
+import string
 import sys
 
-def bail(msg):
-    print(msg, file=sys.stderr)
-    sys.exit(1)
-
-_MARKER = '@@'
-def transform_line(line):
-    # Substitute any special markers of this form:
-    # @@ENV@@
-    # with the value of ENV, split into meson array syntax.
-    start = line.find(_MARKER)
-    if start == -1:
-        return line
-
-    end = line.rfind(_MARKER)
-    if end == start:
-        return line
-
-    # Lookup value of the env var.
-    var = line[start+len(_MARKER):end]
-    try:
-        val = os.environ[var]
-    except KeyError:
-        bail('cannot generate meson.cross; env var %s not set' % var)
+class Template(string.Template):
+    delimiter = "@"
 
-    # Transform into meson array.
-    val = ["'%s'" % x for x in val.split()]
-    val = ', '.join(val)
-    val = '[%s]' % val
+class Environ():
+    def __getitem__(self, name):
+        val = os.environ[name]
+        val = ["'%s'" % x for x in val.split()]
+        val = ', '.join(val)
+        val = '[%s]' % val
+        return val
 
-    before = line[:start]
-    after = line[end+len(_MARKER):]
-
-    return '%s%s%s' % (before, val, after)
-
-# Make sure this is really an SDK extraction environment.
 try:
     sysroot = os.environ['OECORE_NATIVE_SYSROOT']
 except KeyError:
-    bail('OECORE_NATIVE_SYSROOT env var must be set')
-
-cross_file = os.path.join(sysroot, 'usr/share/meson/meson.cross')
-tmp_cross_file = '%s.tmp' % cross_file
+    print("Not in environment setup, bailing")
+    sys.exit(1)
 
-# Read through and transform the current meson.cross.
-lines = []
-with open(cross_file, 'r') as f:
-    for line in f:
-        lines.append(transform_line(line))
+template_file = os.path.join(sysroot, 'usr/share/meson/meson.cross.template')
+cross_file = os.path.join(sysroot, 'usr/share/meson/%smeson.cross' % os.environ["TARGET_PREFIX"])
 
-# Write the transformed result to a tmp file and atomically rename it. In case
-# we crash during the file write, we don't want an invalid meson.cross file.
-with open(tmp_cross_file, 'w') as f:
-    for line in lines:
-        f.write(line)
-    f.flush()
-    os.fdatasync(f.fileno())
-os.rename(tmp_cross_file, cross_file)
+with open(template_file) as in_file:
+    template = in_file.read()
+    output = Template(template).substitute(Environ())
+    with open(cross_file, "w") as out_file:
+        out_file.write(output)
diff --git a/meta/recipes-devtools/meson/meson/meson-wrapper b/meta/recipes-devtools/meson/meson/meson-wrapper
index b2e00da513f..d4ffe60f9a1 100755
--- a/meta/recipes-devtools/meson/meson/meson-wrapper
+++ b/meta/recipes-devtools/meson/meson/meson-wrapper
@@ -10,5 +10,5 @@ fi
 unset CC CXX CPP LD AR NM STRIP
 
 exec "$OECORE_NATIVE_SYSROOT/usr/bin/meson.real" \
-     --cross-file "$OECORE_NATIVE_SYSROOT/usr/share/meson/meson.cross" \
+     --cross-file "${OECORE_NATIVE_SYSROOT}/usr/share/meson/${TARGET_PREFIX}meson.cross" \
      "$@"
diff --git a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
index 721ee8c8f12..55c57775e06 100644
--- a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
+++ b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
@@ -5,9 +5,6 @@ inherit nativesdk
 SRC_URI += "file://meson-setup.py \
             file://meson-wrapper"
 
-def meson_array(var, d):
-    return "', '".join(d.getVar(var).split()).join(("'", "'"))
-
 # both are required but not used by meson
 MESON_SDK_ENDIAN = "bogus-endian"
 MESON_TARGET_ENDIAN = "bogus-endian"
@@ -17,32 +14,31 @@ MESON_C_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_CFLAGS}"
 MESON_CPP_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_CXXFLAGS}"
 MESON_LINK_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_LDFLAGS}"
 
-# This logic is similar but not identical to that in meson.bbclass, since it's
-# generating for an SDK rather than a cross-compile. Important differences are:
+# The cross file logic is similar but not identical to that in meson.bbclass,
+# since it's generating for an SDK rather than a cross-compile. Important
+# differences are:
 # - We can't set vars like CC, CXX, etc. yet because they will be filled in with
 #   real paths by meson-setup.sh when the SDK is extracted.
 # - Some overrides aren't needed, since the SDK injects paths that take care of
 #   them.
-addtask write_config before do_install
-do_write_config[vardeps] += "MESON_C_ARGS MESON_CPP_ARGS MESON_LINK_ARGS CC CXX LD AR NM STRIP READELF"
-do_write_config() {
-    # This needs to be Py to split the args into single-element lists
-    cat >${WORKDIR}/meson.cross <<EOF
+do_install_append() {
+    install -d ${D}${datadir}/meson
+    cat >${D}${datadir}/meson/meson.cross.template <<EOF
 [binaries]
-c = @@CC@@
-cpp = @@CXX@@
-ar = @@AR@@
-nm = @@NM@@
-ld = @@LD@@
-strip = @@STRIP@@
+c = @CC
+cpp = @CXX
+ar = @AR
+nm = @NM
+ld = @LD
+strip = @STRIP
 pkgconfig = 'pkg-config'
 
 [properties]
 needs_exe_wrapper = true
-c_args = @@CFLAGS@@
-c_link_args = @@LDFLAGS@@
-cpp_args = @@CPPFLAGS@@
-cpp_link_args = @@LDFLAGS@@
+c_args = @CFLAGS
+c_link_args = @LDFLAGS
+cpp_args = @CPPFLAGS
+cpp_link_args = @LDFLAGS
 
 [host_machine]
 system = '${SDK_OS}'
@@ -50,11 +46,6 @@ cpu_family = '${SDK_ARCH}'
 cpu = '${SDK_ARCH}'
 endian = '${MESON_SDK_ENDIAN}'
 EOF
-}
-
-do_install_append() {
-    install -d ${D}${datadir}/meson
-    install -m 0644 ${WORKDIR}/meson.cross ${D}${datadir}/meson/
 
     install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d
     install -m 0755 ${WORKDIR}/meson-setup.py ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
-- 
2.11.0

[PATCH 4/7] meson: write correct host endian into SDK cross file

[Ross Burton]

Meson doesn't ignore this but will emit a warning, so write the correct value.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
index 55c57775e06..1549357a554 100644
--- a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
+++ b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
@@ -1,13 +1,20 @@
 include meson.inc
 
 inherit nativesdk
+inherit siteinfo
 
 SRC_URI += "file://meson-setup.py \
             file://meson-wrapper"
 
-# both are required but not used by meson
-MESON_SDK_ENDIAN = "bogus-endian"
-MESON_TARGET_ENDIAN = "bogus-endian"
+def meson_endian(prefix, d):
+    arch, os = d.getVar(prefix + "_ARCH"), d.getVar(prefix + "_OS")
+    sitedata = siteinfo_data_for_machine(arch, os, d)
+    if "endian-little" in sitedata:
+        return "little"
+    elif "endian-big" in sitedata:
+        return "big"
+    else:
+        bb.fatal("Cannot determine endianism for %s-%s" % (arch, os))
 
 MESON_TOOLCHAIN_ARGS = "${BUILDSDK_CC_ARCH}${TOOLCHAIN_OPTIONS}"
 MESON_C_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_CFLAGS}"
@@ -44,7 +51,7 @@ cpp_link_args = @LDFLAGS
 system = '${SDK_OS}'
 cpu_family = '${SDK_ARCH}'
 cpu = '${SDK_ARCH}'
-endian = '${MESON_SDK_ENDIAN}'
+endian = '${@meson_endian("SDK", d)}'
 EOF
 
     install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d
-- 
2.11.0

[PATCH 5/7] oeqa/sdk/cases: clean up DL_DIR handling

[Ross Burton]

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/lib/oeqa/sdk/cases/assimp.py          | 3 +--
 meta/lib/oeqa/sdk/cases/buildcpio.py       | 3 +--
 meta/lib/oeqa/sdk/cases/buildgalculator.py | 3 +--
 meta/lib/oeqa/sdk/cases/buildlzip.py       | 3 +--
 4 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oeqa/sdk/cases/assimp.py b/meta/lib/oeqa/sdk/cases/assimp.py
index 21c91068815..595f3834894 100644
--- a/meta/lib/oeqa/sdk/cases/assimp.py
+++ b/meta/lib/oeqa/sdk/cases/assimp.py
@@ -20,8 +20,7 @@ class BuildAssimp(OESDKTestCase):
     def test_assimp(self):
         import tempfile
         with tempfile.TemporaryDirectory(prefix="assimp", dir=self.tc.sdk_dir) as testdir:
-            dl_dir = self.td.get('DL_DIR', None)
-            tarball = self.fetch(testdir, dl_dir, "https://github.com/assimp/assimp/archive/v4.1.0.tar.gz")
+            tarball = self.fetch(testdir, self.td["DL_DIR"], "https://github.com/assimp/assimp/archive/v4.1.0.tar.gz")
             subprocess.check_output(["tar", "xf", tarball, "-C", testdir])
 
             sourcedir = os.path.join(testdir, "assimp-4.1.0") 
diff --git a/meta/lib/oeqa/sdk/cases/buildcpio.py b/meta/lib/oeqa/sdk/cases/buildcpio.py
index ff3e1347d37..9504ee8e053 100644
--- a/meta/lib/oeqa/sdk/cases/buildcpio.py
+++ b/meta/lib/oeqa/sdk/cases/buildcpio.py
@@ -13,8 +13,7 @@ class BuildCpioTest(OESDKTestCase):
     """
     def test_cpio(self):
         with tempfile.TemporaryDirectory(prefix="cpio-", dir=self.tc.sdk_dir) as testdir:
-            dl_dir = self.td.get('DL_DIR', None)
-            tarball = self.fetch(testdir, dl_dir, "https://ftp.gnu.org/gnu/cpio/cpio-2.12.tar.gz")
+            tarball = self.fetch(testdir, self.td["DL_DIR"], "https://ftp.gnu.org/gnu/cpio/cpio-2.12.tar.gz")
 
             dirs = {}
             dirs["source"] = os.path.join(testdir, "cpio-2.12")
diff --git a/meta/lib/oeqa/sdk/cases/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py
index 164a695637a..47d7580fafd 100644
--- a/meta/lib/oeqa/sdk/cases/buildgalculator.py
+++ b/meta/lib/oeqa/sdk/cases/buildgalculator.py
@@ -20,8 +20,7 @@ class GalculatorTest(OESDKTestCase):
 
     def test_galculator(self):
         with tempfile.TemporaryDirectory(prefix="galculator", dir=self.tc.sdk_dir) as testdir:
-            dl_dir = self.td.get('DL_DIR', None)
-            tarball = self.fetch(testdir, dl_dir, "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2")
+            tarball = self.fetch(testdir, self.td["DL_DIR"], "http://galculator.mnim.org/downloads/galculator-2.1.4.tar.bz2")
 
             dirs = {}
             dirs["source"] = os.path.join(testdir, "galculator-2.1.4")
diff --git a/meta/lib/oeqa/sdk/cases/buildlzip.py b/meta/lib/oeqa/sdk/cases/buildlzip.py
index a7ca239fb49..b7483bfea51 100644
--- a/meta/lib/oeqa/sdk/cases/buildlzip.py
+++ b/meta/lib/oeqa/sdk/cases/buildlzip.py
@@ -9,8 +9,7 @@ class BuildLzipTest(OESDKTestCase):
     """
     def test_lzip(self):
         with tempfile.TemporaryDirectory(prefix="lzip", dir=self.tc.sdk_dir) as testdir:
-            dl_dir = self.td.get('DL_DIR', None)
-            tarball = self.fetch(testdir, dl_dir, "http://downloads.yoctoproject.org/mirror/sources/lzip-1.19.tar.gz")
+            tarball = self.fetch(testdir, self.td["DL_DIR"], "http://downloads.yoctoproject.org/mirror/sources/lzip-1.19.tar.gz")
 
             dirs = {}
             dirs["source"] = os.path.join(testdir, "lzip-1.19")
-- 
2.11.0

[PATCH 6/7] oeqa/sdk/assimp: cleanup

[Ross Burton]

Unify style with the other tests.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/lib/oeqa/sdk/cases/assimp.py | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/meta/lib/oeqa/sdk/cases/assimp.py b/meta/lib/oeqa/sdk/cases/assimp.py
index 595f3834894..a6000103993 100644
--- a/meta/lib/oeqa/sdk/cases/assimp.py
+++ b/meta/lib/oeqa/sdk/cases/assimp.py
@@ -1,5 +1,7 @@
-import os, subprocess, unittest
-import bb
+import os
+import subprocess
+import tempfile
+import unittest
 from oeqa.sdk.case import OESDKTestCase
 
 from oeqa.utils.subprocesstweak import errors_have_output
@@ -10,25 +12,25 @@ class BuildAssimp(OESDKTestCase):
     Test case to build a project using cmake.
     """
 
-    td_vars = ['DATETIME', 'TARGET_OS', 'TARGET_ARCH']
-
     def setUp(self):
         if not (self.tc.hasHostPackage("nativesdk-cmake") or
                 self.tc.hasHostPackage("cmake-native")):
             raise unittest.SkipTest("Needs cmake")
 
     def test_assimp(self):
-        import tempfile
         with tempfile.TemporaryDirectory(prefix="assimp", dir=self.tc.sdk_dir) as testdir:
             tarball = self.fetch(testdir, self.td["DL_DIR"], "https://github.com/assimp/assimp/archive/v4.1.0.tar.gz")
-            subprocess.check_output(["tar", "xf", tarball, "-C", testdir])
 
-            sourcedir = os.path.join(testdir, "assimp-4.1.0") 
-            builddir = os.path.join(testdir, "build")
-            installdir = os.path.join(testdir, "install")
-            bb.utils.mkdirhier(builddir)
+            dirs = {}
+            dirs["source"] = os.path.join(testdir, "assimp-4.1.0")
+            dirs["build"] = os.path.join(testdir, "build")
+            dirs["install"] = os.path.join(testdir, "install")
+
+            subprocess.check_output(["tar", "xf", tarball, "-C", testdir])
+            self.assertTrue(os.path.isdir(dirs["source"]))
+            os.makedirs(dirs["build"])
 
-            self._run("cd %s && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON %s " % (builddir, sourcedir))
-            self._run("cmake --build %s -- -j" % builddir)
-            self._run("cmake --build %s --target install -- DESTDIR=%s" % (builddir, installdir))
-            self.check_elf(os.path.join(installdir, "usr", "local", "lib", "libassimp.so.4.1.0"))
+            self._run("cd {build} && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON {source}".format(**dirs))
+            self._run("cmake --build {build} -- -j".format(**dirs))
+            self._run("cmake --build {build} --target install -- DESTDIR={install}".format(**dirs))
+            self.check_elf(os.path.join(dirs["install"], "usr", "local", "lib", "libassimp.so.4.1.0"))
-- 
2.11.0

[PATCH 7/7] oeqa/sdk: add test to exercise Meson

[Ross Burton]

(From OE-Core rev: 29359493e391d68a5a6b4fa4d09ffdc1fe6db620)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/sdk/cases/buildepoxy.py | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
 create mode 100644 meta/lib/oeqa/sdk/cases/buildepoxy.py

diff --git a/meta/lib/oeqa/sdk/cases/buildepoxy.py b/meta/lib/oeqa/sdk/cases/buildepoxy.py
new file mode 100644
index 00000000000..ef24b4f4ac1
--- /dev/null
+++ b/meta/lib/oeqa/sdk/cases/buildepoxy.py
@@ -0,0 +1,35 @@
+import os
+import subprocess
+import tempfile
+import unittest
+
+from oeqa.sdk.case import OESDKTestCase
+from oeqa.utils.subprocesstweak import errors_have_output
+errors_have_output()
+
+class EpoxyTest(OESDKTestCase):
+    """
+    Test that Meson builds correctly.
+    """
+    def setUp(self):
+        if not (self.tc.hasHostPackage("nativesdk-meson")):
+            raise unittest.SkipTest("GalculatorTest class: SDK doesn't contain Meson")
+
+    def test_epoxy(self):
+        with tempfile.TemporaryDirectory(prefix="epoxy", dir=self.tc.sdk_dir) as testdir:
+            tarball = self.fetch(testdir, self.td["DL_DIR"], "https://github.com/anholt/libepoxy/releases/download/1.5.3/libepoxy-1.5.3.tar.xz")
+
+            dirs = {}
+            dirs["source"] = os.path.join(testdir, "libepoxy-1.5.3")
+            dirs["build"] = os.path.join(testdir, "build")
+            dirs["install"] = os.path.join(testdir, "install")
+
+            subprocess.check_output(["tar", "xf", tarball, "-C", testdir])
+            self.assertTrue(os.path.isdir(dirs["source"]))
+            os.makedirs(dirs["build"])
+
+            self._run("meson -Degl=no -Dglx=no -Dx11=false {build} {source}".format(**dirs))
+            self._run("ninja -C {build} -v".format(**dirs))
+            self._run("DESTDIR={install} ninja -C {build} -v install".format(**dirs))
+
+            self.check_elf(os.path.join(dirs["install"], "usr", "local", "lib", "libepoxy.so"))
-- 
2.11.0

Re: [PATCH 7/7] oeqa/sdk: add test to exercise Meson

[Joshua Watt]

On Mon, 2019-01-07 at 16:38 +0000, Ross Burton wrote:
> (From OE-Core rev: 29359493e391d68a5a6b4fa4d09ffdc1fe6db620)
> 
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/lib/oeqa/sdk/cases/buildepoxy.py | 35
> +++++++++++++++++++++++++++++++++++
>  1 file changed, 35 insertions(+)
>  create mode 100644 meta/lib/oeqa/sdk/cases/buildepoxy.py
> 
> diff --git a/meta/lib/oeqa/sdk/cases/buildepoxy.py
> b/meta/lib/oeqa/sdk/cases/buildepoxy.py
> new file mode 100644
> index 00000000000..ef24b4f4ac1
> --- /dev/null
> +++ b/meta/lib/oeqa/sdk/cases/buildepoxy.py
> @@ -0,0 +1,35 @@
> +import os
> +import subprocess
> +import tempfile
> +import unittest
> +
> +from oeqa.sdk.case import OESDKTestCase
> +from oeqa.utils.subprocesstweak import errors_have_output
> +errors_have_output()
> +
> +class EpoxyTest(OESDKTestCase):

FWIW: I know that is in the same style as the assimp test, but I always
thought the test should have been called the "cmake" test, since that
is what we actually care about (not specifically that it can build
assimp; any cmake package could have been used AFAIK).

Perhaps this should break that trend and be called the "meson" test?

> +    """
> +    Test that Meson builds correctly.
> +    """
> +    def setUp(self):
> +        if not (self.tc.hasHostPackage("nativesdk-meson")):
> +            raise unittest.SkipTest("GalculatorTest class: SDK
> doesn't contain Meson")
> +
> +    def test_epoxy(self):
> +        with tempfile.TemporaryDirectory(prefix="epoxy",
> dir=self.tc.sdk_dir) as testdir:
> +            tarball = self.fetch(testdir, self.td["DL_DIR"], "
> https://github.com/anholt/libepoxy/releases/download/1.5.3/libepoxy-1.5.3.tar.xz
> ")
> +
> +            dirs = {}
> +            dirs["source"] = os.path.join(testdir, "libepoxy-1.5.3")
> +            dirs["build"] = os.path.join(testdir, "build")
> +            dirs["install"] = os.path.join(testdir, "install")
> +
> +            subprocess.check_output(["tar", "xf", tarball, "-C",
> testdir])
> +            self.assertTrue(os.path.isdir(dirs["source"]))
> +            os.makedirs(dirs["build"])
> +
> +            self._run("meson -Degl=no -Dglx=no -Dx11=false {build}
> {source}".format(**dirs))
> +            self._run("ninja -C {build} -v".format(**dirs))
> +            self._run("DESTDIR={install} ninja -C {build} -v
> install".format(**dirs))
> +
> +            self.check_elf(os.path.join(dirs["install"], "usr",
> "local", "lib", "libepoxy.so"))
> -- 
> 2.11.0
> 
-- 
Joshua Watt <JPEWhacker@gmail.com>

Re: [PATCH 7/7] oeqa/sdk: add test to exercise Meson

[Burton, Ross]

On Mon, 7 Jan 2019 at 16:45, Joshua Watt <jpewhacker@gmail.com> wrote:
> FWIW: I know that is in the same style as the assimp test, but I always
> thought the test should have been called the "cmake" test, since that
> is what we actually care about (not specifically that it can build
> assimp; any cmake package could have been used AFAIK).
>
> Perhaps this should break that trend and be called the "meson" test?

My worry was that in the future we'd add another recipe that used
cmake/meson to exercise some other codepaths, and then we'd have the
cmake.py and cmake2.py.

At least by using the upstream name, it's clear what is actually being tested.

Ross

✗ patchtest: failure for "toolchain-scripts: run post-re..." and 6 more

[Patchwork]

== Series Details ==

Series: "toolchain-scripts: run post-re..." and 6 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/15560/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 65c419b8c4)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [PATCH 3/7] meson: fix nativesdk-meson for multilib SDKs

[Martin Kelly]

On 1/7/19 8:38 AM, Ross Burton wrote:
> Multilib SDKs differ only in the environment variables set, so nativesdk-meson's
> setup script needs to write a cross file for each environment.
> 
> Rename the shipped meson.cross to meson.cross.template, as it cannot be used
> directly.  Now that post-relocate scripts are called once for each environment,
> the generated meson.cross can be prefixed with TARGET_PREFIX to ensure it is
> unique.
> 
> Finally rewrite the setup script to use string.Template to perform the expansion
> instead of hand-coding the logic.
> 
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>   meta/recipes-devtools/meson/meson/meson-setup.py   | 69 ++++++----------------
>   meta/recipes-devtools/meson/meson/meson-wrapper    |  2 +-
>   .../meson/nativesdk-meson_0.49.0.bb                | 41 +++++--------
>   3 files changed, 36 insertions(+), 76 deletions(-)
> 
> diff --git a/meta/recipes-devtools/meson/meson/meson-setup.py b/meta/recipes-devtools/meson/meson/meson-setup.py
> index a9749eae9d4..808e2a062f6 100755
> --- a/meta/recipes-devtools/meson/meson/meson-setup.py
> +++ b/meta/recipes-devtools/meson/meson/meson-setup.py
> @@ -1,62 +1,31 @@
>   #!/usr/bin/env python3
>   
>   import os
> +import string
>   import sys
>   
> -def bail(msg):
> -    print(msg, file=sys.stderr)
> -    sys.exit(1)
> -
> -_MARKER = '@@'
> -def transform_line(line):
> -    # Substitute any special markers of this form:
> -    # @@ENV@@
> -    # with the value of ENV, split into meson array syntax.
> -    start = line.find(_MARKER)
> -    if start == -1:
> -        return line
> -
> -    end = line.rfind(_MARKER)
> -    if end == start:
> -        return line
> -
> -    # Lookup value of the env var.
> -    var = line[start+len(_MARKER):end]
> -    try:
> -        val = os.environ[var]
> -    except KeyError:
> -        bail('cannot generate meson.cross; env var %s not set' % var)
> +class Template(string.Template):
> +    delimiter = "@"
>   
> -    # Transform into meson array.
> -    val = ["'%s'" % x for x in val.split()]
> -    val = ', '.join(val)
> -    val = '[%s]' % val
> +class Environ():
> +    def __getitem__(self, name):
> +        val = os.environ[name]
> +        val = ["'%s'" % x for x in val.split()]
> +        val = ', '.join(val)
> +        val = '[%s]' % val
> +        return val
>   
> -    before = line[:start]
> -    after = line[end+len(_MARKER):]
> -
> -    return '%s%s%s' % (before, val, after)
> -
> -# Make sure this is really an SDK extraction environment.
>   try:
>       sysroot = os.environ['OECORE_NATIVE_SYSROOT']
>   except KeyError:
> -    bail('OECORE_NATIVE_SYSROOT env var must be set')
> -
> -cross_file = os.path.join(sysroot, 'usr/share/meson/meson.cross')
> -tmp_cross_file = '%s.tmp' % cross_file
> +    print("Not in environment setup, bailing")

This should print to sys.stderr as the bail function did.

> +    sys.exit(1)
>   
> -# Read through and transform the current meson.cross.
> -lines = []
> -with open(cross_file, 'r') as f:
> -    for line in f:
> -        lines.append(transform_line(line))
> +template_file = os.path.join(sysroot, 'usr/share/meson/meson.cross.template')
> +cross_file = os.path.join(sysroot, 'usr/share/meson/%smeson.cross' % os.environ["TARGET_PREFIX"])
>   
> -# Write the transformed result to a tmp file and atomically rename it. In case
> -# we crash during the file write, we don't want an invalid meson.cross file.
> -with open(tmp_cross_file, 'w') as f:
> -    for line in lines:
> -        f.write(line)
> -    f.flush()
> -    os.fdatasync(f.fileno())
> -os.rename(tmp_cross_file, cross_file)
> +with open(template_file) as in_file:
> +    template = in_file.read()
> +    output = Template(template).substitute(Environ())
> +    with open(cross_file, "w") as out_file:
> +        out_file.write(output)

I like using string.Template instead of custom logic, but shouldn't we 
keep the atomic rename logic? If someone hits control-C or the 
extraction process crashes during the write, we don't want a corrupt 
file due to a partial write.

> diff --git a/meta/recipes-devtools/meson/meson/meson-wrapper b/meta/recipes-devtools/meson/meson/meson-wrapper
> index b2e00da513f..d4ffe60f9a1 100755
> --- a/meta/recipes-devtools/meson/meson/meson-wrapper
> +++ b/meta/recipes-devtools/meson/meson/meson-wrapper
> @@ -10,5 +10,5 @@ fi
>   unset CC CXX CPP LD AR NM STRIP
>   
>   exec "$OECORE_NATIVE_SYSROOT/usr/bin/meson.real" \
> -     --cross-file "$OECORE_NATIVE_SYSROOT/usr/share/meson/meson.cross" \
> +     --cross-file "${OECORE_NATIVE_SYSROOT}/usr/share/meson/${TARGET_PREFIX}meson.cross" \
>        "$@"
> diff --git a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
> index 721ee8c8f12..55c57775e06 100644
> --- a/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
> +++ b/meta/recipes-devtools/meson/nativesdk-meson_0.49.0.bb
> @@ -5,9 +5,6 @@ inherit nativesdk
>   SRC_URI += "file://meson-setup.py \
>               file://meson-wrapper"
>   
> -def meson_array(var, d):
> -    return "', '".join(d.getVar(var).split()).join(("'", "'"))
> -
>   # both are required but not used by meson
>   MESON_SDK_ENDIAN = "bogus-endian"
>   MESON_TARGET_ENDIAN = "bogus-endian"
> @@ -17,32 +14,31 @@ MESON_C_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_CFLAGS}"
>   MESON_CPP_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_CXXFLAGS}"
>   MESON_LINK_ARGS = "${MESON_TOOLCHAIN_ARGS} ${BUILDSDK_LDFLAGS}"
>   
> -# This logic is similar but not identical to that in meson.bbclass, since it's
> -# generating for an SDK rather than a cross-compile. Important differences are:
> +# The cross file logic is similar but not identical to that in meson.bbclass,
> +# since it's generating for an SDK rather than a cross-compile. Important
> +# differences are:
>   # - We can't set vars like CC, CXX, etc. yet because they will be filled in with
>   #   real paths by meson-setup.sh when the SDK is extracted.
>   # - Some overrides aren't needed, since the SDK injects paths that take care of
>   #   them.
> -addtask write_config before do_install
> -do_write_config[vardeps] += "MESON_C_ARGS MESON_CPP_ARGS MESON_LINK_ARGS CC CXX LD AR NM STRIP READELF"
> -do_write_config() {
> -    # This needs to be Py to split the args into single-element lists
> -    cat >${WORKDIR}/meson.cross <<EOF
> +do_install_append() {
> +    install -d ${D}${datadir}/meson
> +    cat >${D}${datadir}/meson/meson.cross.template <<EOF
>   [binaries]
> -c = @@CC@@
> -cpp = @@CXX@@
> -ar = @@AR@@
> -nm = @@NM@@
> -ld = @@LD@@
> -strip = @@STRIP@@
> +c = @CC
> +cpp = @CXX
> +ar = @AR
> +nm = @NM
> +ld = @LD
> +strip = @STRIP
>   pkgconfig = 'pkg-config'
>   
>   [properties]
>   needs_exe_wrapper = true
> -c_args = @@CFLAGS@@
> -c_link_args = @@LDFLAGS@@
> -cpp_args = @@CPPFLAGS@@
> -cpp_link_args = @@LDFLAGS@@
> +c_args = @CFLAGS
> +c_link_args = @LDFLAGS
> +cpp_args = @CPPFLAGS
> +cpp_link_args = @LDFLAGS
>   

I have no problem with it, but I'm curious why we use @CC instead of 
@@CC@@ style here. Is there an issue with @@CC@@ style?

>   [host_machine]
>   system = '${SDK_OS}'
> @@ -50,11 +46,6 @@ cpu_family = '${SDK_ARCH}'
>   cpu = '${SDK_ARCH}'
>   endian = '${MESON_SDK_ENDIAN}'
>   EOF
> -}
> -
> -do_install_append() {
> -    install -d ${D}${datadir}/meson
> -    install -m 0644 ${WORKDIR}/meson.cross ${D}${datadir}/meson/
>   
>       install -d ${D}${SDKPATHNATIVE}/post-relocate-setup.d
>       install -m 0755 ${WORKDIR}/meson-setup.py ${D}${SDKPATHNATIVE}/post-relocate-setup.d/
> 

Re: [PATCH 3/7] meson: fix nativesdk-meson for multilib SDKs

[Burton, Ross]

On Mon, 7 Jan 2019 at 19:03, Martin Kelly <mkelly@xevo.com> wrote:
> > +with open(template_file) as in_file:
> > +    template = in_file.read()
> > +    output = Template(template).substitute(Environ())
> > +    with open(cross_file, "w") as out_file:
> > +        out_file.write(output)
>
> I like using string.Template instead of custom logic, but shouldn't we
> keep the atomic rename logic? If someone hits control-C or the
> extraction process crashes during the write, we don't want a corrupt
> file due to a partial write.

If someone control-c's during SDK unpack, the SDK isn't usable.

> I have no problem with it, but I'm curious why we use @CC instead of
> @@CC@@ style here. Is there an issue with @@CC@@ style?

Because that's how string.Template works.

Ross

Re: [PATCH 3/7] meson: fix nativesdk-meson for multilib SDKs

[Martin Kelly]

On 1/7/19 12:02 PM, Burton, Ross wrote:
> On Mon, 7 Jan 2019 at 19:03, Martin Kelly <mkelly@xevo.com> wrote:
>>> +with open(template_file) as in_file:
>>> +    template = in_file.read()
>>> +    output = Template(template).substitute(Environ())
>>> +    with open(cross_file, "w") as out_file:
>>> +        out_file.write(output)
>>
>> I like using string.Template instead of custom logic, but shouldn't we
>> keep the atomic rename logic? If someone hits control-C or the
>> extraction process crashes during the write, we don't want a corrupt
>> file due to a partial write.
> 
> If someone control-c's during SDK unpack, the SDK isn't usable.
> 

I don't think it will break cleanly though; instead, some things will 
work and other things won't. I was trying to avoid cryptic error 
messages such as meson hitting a syntax error. However, I agree that 
this should be addressed globally. Separate from this change, maybe we 
should drop a semaphore at the end of extraction and check it in the env 
setup script, so we can display a nice message if something didn't complete.

Re: [PATCH 1/7] toolchain-scripts: run post-relocate scripts for every environment

[Burton, Ross]

On Mon, 7 Jan 2019 at 16:38, Ross Burton <ross.burton@intel.com> wrote:
> SDKs for multilib configurations have multiple environment scripts, so
> re-arrange the post-relocate hook invocation so that it runs the post-relocate
> hooks after sourcing each environment script.

I think this change - whilst the right thing - exposes a problem that
icecc doesn't work properly in multilib SDKs.  Can I ask you to
investigate and sort this, as you submitted the icecc rewrite
recently?

Ross

Re: [PATCH 1/7] toolchain-scripts: run post-relocate scripts for every environment

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Tue, Jan 8, 2019, 5:02 AM Burton, Ross <ross.burton@intel.com wrote:

> On Mon, 7 Jan 2019 at 16:38, Ross Burton <ross.burton@intel.com> wrote:
> > SDKs for multilib configurations have multiple environment scripts, so
> > re-arrange the post-relocate hook invocation so that it runs the
> post-relocate
> > hooks after sourcing each environment script.
>
> I think this change - whilst the right thing - exposes a problem that
> icecc doesn't work properly in multilib SDKs.  Can I ask you to
> investigate and sort this, as you submitted the icecc rewrite
> recently?
>

Yep, I'll take a look.


> Ross
>

[-- Attachment #2: Type: text/html, Size: 1237 bytes --]

Re: [PATCH 1/7] toolchain-scripts: run post-relocate scripts for every environment

[Burton, Ross]

On Tue, 8 Jan 2019 at 13:10, Joshua Watt <jpewhacker@gmail.com> wrote:
>> I think this change - whilst the right thing - exposes a problem that
>> icecc doesn't work properly in multilib SDKs.  Can I ask you to
>> investigate and sort this, as you submitted the icecc rewrite
>> recently?
>
> Yep, I'll take a look.

Thanks, hopefully you spotted some bug mail earlier.

Ross