All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/9] Extensible SDK
@ 2015-02-23 17:00 Paul Eggleton
  2015-02-23 17:00 ` [PATCH 1/9] gen-lockedsig-cache: Allow cross-filesystem copies Paul Eggleton
                   ` (8 more replies)
  0 siblings, 9 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

At long last, here is the first mergeable version of what I'm now
calling the "extensible SDK". The initial proposal for this was posted
last year:

https://lists.yoctoproject.org/pipermail/yocto/2014-August/021044.html

Other than some names there hasn't been all that much change since then,
although things have changed quite a lot behind the scenes. In any case,
the basic idea is (example from poky):

 1) Create an extensible SDK:

    bitbake -c populate_sdk_ext core-image-minimal

 2) Install the SDK:

    ./poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-1.7.sh

    (usual SDK installation procedure; however do not run the installer
     as root or under sudo - it should be installed as the user you intend
     to run it with)

 3) Run the environment script:
    
    . ./environment-setup-core2-64-poky-linux

 4) Add your application to the build:
   
    devtool add myapp /path/to/myapp/source

 5) Build it:

    devtool build myapp

 6) Deploy the application to your target:

    devtool deploy-target myapp root@192.168.7.2

Now your application is ready to test on the target. "devtool build"
puts items into the SDK's sysroot as well so you can also add new
libraries to the installed SDK as needed.

There are still some rough edges to be worked out, and there is no
capability to update the SDK without reinstalling yet, but the basic
functionality is there.


The following changes since commit c4de42aadd4c8a4a8f16c25e7dcdefef79daf030:

  pulseaudio: upgrade to 6.0 (2015-02-23 08:08:05 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/extensible-sdk
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/extensible-sdk

Paul Eggleton (1):
  sstatesig: Add ability to filter list of tasks for locked-sigs.inc

Randy Witt (8):
  gen-lockedsig-cache: Allow cross-filesystem copies.
  copy_buildsystem.py: Add a way to copy buildsystem to a directory.
  copy_buildsystem.py: Add methods to copy shared state.
  toolchain-scripts: Add parameters to toolchain_create_sdk_env_script.
  toolchain-shar-template.sh: Make relocation optional.
  uninative-tarball: Actually use bzip2 for compression.
  scripts/oe-buildenv-internal: add means of skipping SDK check during
    setup
  populate_sdk_ext: add extensible SDK

 meta/classes/image.bbclass                         |   2 +-
 meta/classes/populate_sdk_base.bbclass             |  10 +-
 meta/classes/populate_sdk_ext.bbclass              | 217 +++++++++++++++++++++
 meta/classes/toolchain-scripts.bbclass             |  15 +-
 ...-shar-template.sh => toolchain-shar-extract.sh} |  53 +----
 meta/files/toolchain-shar-relocate.sh              |  50 +++++
 meta/lib/oe/copy_buildsystem.py                    | 102 ++++++++++
 meta/lib/oe/sstatesig.py                           |   5 +-
 meta/recipes-core/meta/meta-environment-extsdk.bb  |  12 ++
 meta/recipes-core/meta/uninative-tarball.bb        |   2 +-
 scripts/gen-lockedsig-cache                        |  10 +-
 scripts/oe-buildenv-internal                       |   2 +-
 12 files changed, 414 insertions(+), 66 deletions(-)
 create mode 100644 meta/classes/populate_sdk_ext.bbclass
 rename meta/files/{toolchain-shar-template.sh => toolchain-shar-extract.sh} (64%)
 create mode 100644 meta/files/toolchain-shar-relocate.sh
 create mode 100644 meta/lib/oe/copy_buildsystem.py
 create mode 100644 meta/recipes-core/meta/meta-environment-extsdk.bb

-- 
1.9.3



^ permalink raw reply	[flat|nested] 17+ messages in thread
* [PATCH 1/9] gen-lockedsig-cache: Allow cross-filesystem copies.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 2/9] sstatesig: Add ability to filter list of tasks for locked-sigs.inc Paul Eggleton
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

Since this previously always tried to use hardlinks you couldn't have
the source and destination be on different devices. This change allows
for that and also prevents failure in situations where the files already
existed.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
---
 scripts/gen-lockedsig-cache | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/scripts/gen-lockedsig-cache b/scripts/gen-lockedsig-cache
index dfb282e..c93b2c0 100755
--- a/scripts/gen-lockedsig-cache
+++ b/scripts/gen-lockedsig-cache
@@ -35,6 +35,12 @@ for s in sigs:
 
 for f in files:
     dst = f.replace(sys.argv[2], sys.argv[3])
-    mkdir(os.path.dirname(dst))
-    os.link(f, dst)
+    destdir = os.path.dirname(dst)
+    mkdir(destdir)
 
+    if os.path.exists(dst):
+        os.remove(dst)
+    if (os.stat(f).st_dev == os.stat(destdir).st_dev):
+        os.link(f, dst)
+    else:
+        shutil.copyfile(f, dst)
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 2/9] sstatesig: Add ability to filter list of tasks for locked-sigs.inc
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
  2015-02-23 17:00 ` [PATCH 1/9] gen-lockedsig-cache: Allow cross-filesystem copies Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 3/9] copy_buildsystem.py: Add a way to copy buildsystem to a directory Paul Eggleton
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/lib/oe/sstatesig.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index af7617e..62e75c2 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -137,13 +137,16 @@ class SignatureGeneratorOEBasicHash(bb.siggen.SignatureGeneratorBasicHash):
             return
         super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigtask(fn, task, stampbase, runtime)
 
-    def dump_lockedsigs(self, sigfile=None):
+    def dump_lockedsigs(self, sigfile=None, taskfilter=None):
         if not sigfile:
             sigfile = os.getcwd() + "/locked-sigs.inc"
 
         bb.plain("Writing locked sigs to %s" % sigfile)
         types = {}
         for k in self.runtaskdeps:
+            if taskfilter:
+                if not k in taskfilter:
+                    continue
             fn = k.rsplit(".",1)[0]
             t = self.lockedhashfn[fn].split(" ")[1].split(":")[5]
             t = 't-' + t.replace('_', '-')
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 3/9] copy_buildsystem.py: Add a way to copy buildsystem to a directory.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
  2015-02-23 17:00 ` [PATCH 1/9] gen-lockedsig-cache: Allow cross-filesystem copies Paul Eggleton
  2015-02-23 17:00 ` [PATCH 2/9] sstatesig: Add ability to filter list of tasks for locked-sigs.inc Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 4/9] copy_buildsystem.py: Add methods to copy shared state Paul Eggleton
                   ` (5 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

This file provides a way to take bitbake and the layers in the
current build and copy them to a target specified.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/lib/oe/copy_buildsystem.py | 70 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)
 create mode 100644 meta/lib/oe/copy_buildsystem.py

diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
new file mode 100644
index 0000000..5a4d8c3
--- /dev/null
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -0,0 +1,70 @@
+# This class should provide easy access to the different aspects of the
+# buildsystem such as layers, bitbake location, etc.
+import stat
+import shutil
+
+def _smart_copy(src, dest):
+    # smart_copy will choose the correct function depending on whether the
+    # source is a file or a directory.
+    mode = os.stat(src).st_mode
+    if stat.S_ISDIR(mode):
+        shutil.copytree(src, dest, symlinks=True)
+    else:
+        shutil.copyfile(src, dest)
+        shutil.copymode(src, dest)
+
+class BuildSystem(object):
+    def __init__(self, d):
+        self.d = d
+        self.layerdirs = d.getVar('BBLAYERS', True).split()
+
+    def copy_bitbake_and_layers(self, destdir):
+        # Copy in all metadata layers + bitbake (as repositories)
+        layers_copied = []
+        bb.utils.mkdirhier(destdir)
+        layers = list(self.layerdirs)
+
+        corebase = self.d.getVar('COREBASE', True)
+        layers.append(corebase)
+
+        corebase_files = self.d.getVar('COREBASE_FILES', True).split()
+
+        # bitbake belongs in corebase so make sure it goes there
+        if "bitbake" not in corebase_files:
+            corebase_files.append("bitbake")
+        corebase_files = [corebase + '/' +x for x in corebase_files]
+
+        for layer in layers:
+            layerconf = os.path.join(layer, 'conf', 'layer.conf')
+            if os.path.exists(layerconf):
+                with open(layerconf, 'r') as f:
+                    if f.readline().startswith("# ### workspace layer auto-generated by devtool ###"):
+                        bb.warn("Skipping local workspace layer %s" % layer)
+                        continue
+
+            # If the layer was already under corebase, leave it there
+            # since layers such as meta have issues when moved.
+            layerdestpath = destdir
+            if corebase == os.path.dirname(layer):
+                layerdestpath += '/' + os.path.basename(corebase)
+            layerdestpath += '/' + os.path.basename(layer)
+
+            layer_relative = os.path.relpath(layerdestpath,
+                                             destdir)
+            layers_copied.append(layer_relative)
+
+            # Treat corebase as special since it typically will contain
+            # build directories or other custom items.
+            if corebase == layer:
+                bb.utils.mkdirhier(layerdestpath)
+                for f in corebase_files:
+                    f_basename = os.path.basename(f)
+                    destname = os.path.join(layerdestpath, f_basename)
+                    _smart_copy(f, destname)
+            else:
+                if os.path.exists(layerdestpath):
+                    bb.note("Skipping layer %s, already handled" % layer)
+                else:
+                    _smart_copy(layer, layerdestpath)
+
+        return layers_copied
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 4/9] copy_buildsystem.py: Add methods to copy shared state.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (2 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 3/9] copy_buildsystem.py: Add a way to copy buildsystem to a directory Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 5/9] toolchain-scripts: Add parameters to toolchain_create_sdk_env_script Paul Eggleton
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

Added the helper functions necessary to copy the sstate from the
current build, and generate the file to "lock" it.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/lib/oe/copy_buildsystem.py | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
index 5a4d8c3..cf7fada 100644
--- a/meta/lib/oe/copy_buildsystem.py
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -68,3 +68,35 @@ class BuildSystem(object):
                     _smart_copy(layer, layerdestpath)
 
         return layers_copied
+
+def generate_locked_sigs(sigfile, d):
+    bb.utils.mkdirhier(os.path.dirname(sigfile))
+    depd = d.getVar('BB_TASKDEPDATA', True)
+    tasks = ['%s.%s' % (v[2], v[1]) for v in depd.itervalues()]
+    bb.parse.siggen.dump_lockedsigs(sigfile, tasks)
+
+def prune_lockedsigs(allowed_tasks, excluded_targets, lockedsigs, pruned_output):
+    with open(lockedsigs, 'r') as infile:
+        bb.utils.mkdirhier(os.path.dirname(pruned_output))
+        with open(pruned_output, 'w') as f:
+            invalue = False
+            for line in infile:
+                if invalue:
+                    if line.endswith('\\\n'):
+                        splitval = line.strip().split(':')
+                        if splitval[1] in allowed_tasks and not splitval[0] in excluded_targets:
+                            f.write(line)
+                    else:
+                        f.write(line)
+                        invalue = False
+                elif line.startswith('SIGGEN_LOCKEDSIGS'):
+                    invalue = True
+                    f.write(line)
+
+def create_locked_sstate_cache(lockedsigs, input_sstate_cache, output_sstate_cache, d, fixedlsbstring=""):
+    bb.note('Generating sstate-cache...')
+
+    bb.process.run("gen-lockedsig-cache %s %s %s" % (lockedsigs, input_sstate_cache, output_sstate_cache))
+    if fixedlsbstring:
+        os.rename(output_sstate_cache + '/' + d.getVar('NATIVELSBSTRING', True),
+        output_sstate_cache + '/' + fixedlsbstring)
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 5/9] toolchain-scripts: Add parameters to toolchain_create_sdk_env_script.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (3 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 4/9] copy_buildsystem.py: Add methods to copy shared state Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 6/9] toolchain-shar-template.sh: Make relocation optional Paul Eggleton
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

To add some flexibility to setting up the paths for the toolchain,
add some parameters. This initial use will be in order to point at
the buildsystem toolchain copied in by copy_buildsystem.py.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
---
 meta/classes/toolchain-scripts.bbclass | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/meta/classes/toolchain-scripts.bbclass b/meta/classes/toolchain-scripts.bbclass
index 9836db4..ea6aff0 100644
--- a/meta/classes/toolchain-scripts.bbclass
+++ b/meta/classes/toolchain-scripts.bbclass
@@ -7,6 +7,9 @@ REAL_MULTIMACH_TARGET_SYS ?= "${MULTIMACH_TARGET_SYS}"
 # This function creates an environment-setup-script for use in a deployable SDK
 toolchain_create_sdk_env_script () {
 	# Create environment setup script
+	sdkpathnative=${7:-${SDKPATHNATIVE}}
+	prefix=${6:-${prefix_nativesdk}}
+	bindir=${5:-${bindir_nativesdk}}
 	libdir=${4:-${libdir}}
 	sysroot=${3:-${SDKTARGETSYSROOT}}
 	multimach_target_sys=${2:-${REAL_MULTIMACH_TARGET_SYS}}
@@ -16,17 +19,17 @@ toolchain_create_sdk_env_script () {
 	echo 'export SDKTARGETSYSROOT='"$sysroot" >> $script
 	EXTRAPATH=""
 	for i in ${CANADIANEXTRAOS}; do
-		EXTRAPATH="$EXTRAPATH:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_ARCH}${TARGET_VENDOR}-$i"
+		EXTRAPATH="$EXTRAPATH:$sdkpathnative$bindir/${TARGET_ARCH}${TARGET_VENDOR}-$i"
 	done
-	echo 'export PATH=${SDKPATHNATIVE}${bindir_nativesdk}:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_SYS}'$EXTRAPATH':$PATH' >> $script
-	echo 'export CCACHE_PATH=${SDKPATHNATIVE}${bindir_nativesdk}:${SDKPATHNATIVE}${bindir_nativesdk}/${TARGET_SYS}'$EXTRAPATH':$CCACHE_PATH' >> $script
+	echo "export PATH=$sdkpathnative$bindir:$sdkpathnative$bindir/${TARGET_SYS}"$EXTRAPATH':$PATH' >> $script
+	echo 'export CCACHE_PATH=$sdkpathnative$bindir:$sdkpathnative$bindir/${TARGET_SYS}'$EXTRAPATH':$CCACHE_PATH' >> $script
 	echo 'export PKG_CONFIG_SYSROOT_DIR=$SDKTARGETSYSROOT' >> $script
 	echo 'export PKG_CONFIG_PATH=$SDKTARGETSYSROOT'"$libdir"'/pkgconfig' >> $script
 	echo 'export CONFIG_SITE=${SDKPATH}/site-config-'"${multimach_target_sys}" >> $script
-	echo 'export OECORE_NATIVE_SYSROOT="${SDKPATHNATIVE}"' >> $script
+	echo "export OECORE_NATIVE_SYSROOT=\"$sdkpathnative\"" >> $script
 	echo 'export OECORE_TARGET_SYSROOT="$SDKTARGETSYSROOT"' >> $script
-	echo 'export OECORE_ACLOCAL_OPTS="-I ${SDKPATHNATIVE}/usr/share/aclocal"' >> $script
-	echo 'export PYTHONHOME=${SDKPATHNATIVE}${prefix_nativesdk}' >> $script
+	echo "export OECORE_ACLOCAL_OPTS=\"-I $sdkpathnative/usr/share/aclocal\"" >> $script
+	echo "export PYTHONHOME=$sdkpathnative$prefix" >> $script
 
 	toolchain_shared_env_script
 }
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 6/9] toolchain-shar-template.sh: Make relocation optional.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (4 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 5/9] toolchain-scripts: Add parameters to toolchain_create_sdk_env_script Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 7/9] uninative-tarball: Actually use bzip2 for compression Paul Eggleton
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

If the buildsystem is copied into the sdk and its toolchain is to
be used, then the relocation provided in toolchain-shar-template.sh
isn't needed and will actually fail.

So break the relocation aspect out and essentially make it another
SDK_POST_INSTALL_COMMAND script.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
---
 meta/classes/populate_sdk_base.bbclass             | 10 +++-
 ...-shar-template.sh => toolchain-shar-extract.sh} | 53 +---------------------
 meta/files/toolchain-shar-relocate.sh              | 50 ++++++++++++++++++++
 3 files changed, 59 insertions(+), 54 deletions(-)
 rename meta/files/{toolchain-shar-template.sh => toolchain-shar-extract.sh} (64%)
 create mode 100644 meta/files/toolchain-shar-relocate.sh

diff --git a/meta/classes/populate_sdk_base.bbclass b/meta/classes/populate_sdk_base.bbclass
index e3adacb..5c07693 100644
--- a/meta/classes/populate_sdk_base.bbclass
+++ b/meta/classes/populate_sdk_base.bbclass
@@ -52,6 +52,7 @@ EXCLUDE_FROM_WORLD = "1"
 
 SDK_PACKAGING_FUNC ?= "create_shar"
 SDK_POST_INSTALL_COMMAND ?= ""
+SDK_RELOCATE_AFTER_INSTALL ?= "1"
 
 SDK_MANIFEST = "${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.manifest"
 python write_target_sdk_manifest () {
@@ -116,9 +117,14 @@ fakeroot tar_sdk() {
 
 fakeroot create_shar() {
 	# copy in the template shar extractor script
-	cp ${COREBASE}/meta/files/toolchain-shar-template.sh ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
+	cp ${COREBASE}/meta/files/toolchain-shar-extract.sh ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
 
-	cat << "EOF" > ${T}/post_install_command
+	rm -f ${T}/post_install_command
+
+	if [ ${SDK_RELOCATE_AFTER_INSTALL} -eq 1 ] ; then
+		cp ${COREBASE}/meta/files/toolchain-shar-relocate.sh ${T}/post_install_command
+	fi
+	cat << "EOF" >> ${T}/post_install_command
 ${SDK_POST_INSTALL_COMMAND}
 EOF
 	sed -i -e '/@SDK_POST_INSTALL_COMMAND@/r ${T}/post_install_command' ${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.sh
diff --git a/meta/files/toolchain-shar-template.sh b/meta/files/toolchain-shar-extract.sh
similarity index 64%
rename from meta/files/toolchain-shar-template.sh
rename to meta/files/toolchain-shar-extract.sh
index 151b973..516aa3a 100644
--- a/meta/files/toolchain-shar-template.sh
+++ b/meta/files/toolchain-shar-extract.sh
@@ -139,63 +139,12 @@ for env_setup_script in `ls $target_sdk_dir/environment-setup-*`; do
 	$SUDO_EXEC sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g" -i $env_setup_script
 done
 
-# fix dynamic loader paths in all ELF SDK binaries
-native_sysroot=$($SUDO_EXEC cat $env_setup_script |grep 'OECORE_NATIVE_SYSROOT='|cut -d'=' -f2|tr -d '"')
-dl_path=$($SUDO_EXEC find $native_sysroot/lib -name "ld-linux*")
-if [ "$dl_path" = "" ] ; then
-	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
-	exit 1
-fi
-executable_files=$($SUDO_EXEC find $native_sysroot -type f \
-	\( -perm -0100 -o -perm -0010 -o -perm -0001 \) -printf "'%h/%f' ")
-
-tdir=`mktemp -d`
-if [ x$tdir = x ] ; then
-   echo "SDK relocate failed, could not create a temporary directory"
-   exit 1
-fi
-echo "#!/bin/bash" > $tdir/relocate_sdk.sh
-echo exec ${env_setup_script%/*}/relocate_sdk.py $target_sdk_dir $dl_path $executable_files >> $tdir/relocate_sdk.sh
-$SUDO_EXEC mv $tdir/relocate_sdk.sh ${env_setup_script%/*}/relocate_sdk.sh
-$SUDO_EXEC chmod 755 ${env_setup_script%/*}/relocate_sdk.sh
-rm -rf $tdir
-if [ $relocate = 1 ] ; then
-	$SUDO_EXEC ${env_setup_script%/*}/relocate_sdk.sh
-	if [ $? -ne 0 ]; then
-		echo "SDK could not be set up. Relocate script failed. Abort!"
-		exit 1
-	fi
-fi
-
-# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc
-for replace in "$target_sdk_dir -maxdepth 1" "$native_sysroot"; do
-	$SUDO_EXEC find $replace -type f -exec file '{}' \; | \
-		grep ":.*\(ASCII\|script\|source\).*text" | \
-		awk -F':' '{printf "\"%s\"\n", $1}' | \
-		grep -v "$target_sdk_dir/environment-setup-*" | \
-		$SUDO_EXEC xargs -n32 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
-done
-
-# change all symlinks pointing to @SDKPATH@
-for l in $($SUDO_EXEC find $native_sysroot -type l); do
-	$SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
-done
-
-# find out all perl scripts in $native_sysroot and modify them replacing the
-# host perl with SDK perl.
-for perl_script in $($SUDO_EXEC find $native_sysroot -type f -exec grep -l "^#!.*perl" '{}' \;); do
-	$SUDO_EXEC sed -i -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" -e \
-		"s: /usr/bin/perl: /usr/bin/env perl:g" $perl_script
-done
-
-echo done
-
 @SDK_POST_INSTALL_COMMAND@
 
 # delete the relocating script, so that user is forced to re-run the installer
 # if he/she wants another location for the sdk
 if [ $savescripts = 0 ] ; then
-	$SUDO_EXEC rm ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
+	$SUDO_EXEC rm -f ${env_setup_script%/*}/relocate_sdk.py ${env_setup_script%/*}/relocate_sdk.sh
 fi
 
 echo "SDK has been successfully set up and is ready to be used."
diff --git a/meta/files/toolchain-shar-relocate.sh b/meta/files/toolchain-shar-relocate.sh
new file mode 100644
index 0000000..dfb8e16
--- /dev/null
+++ b/meta/files/toolchain-shar-relocate.sh
@@ -0,0 +1,50 @@
+# fix dynamic loader paths in all ELF SDK binaries
+native_sysroot=$($SUDO_EXEC cat $env_setup_script |grep 'OECORE_NATIVE_SYSROOT='|cut -d'=' -f2|tr -d '"')
+dl_path=$($SUDO_EXEC find $native_sysroot/lib -name "ld-linux*")
+if [ "$dl_path" = "" ] ; then
+	echo "SDK could not be set up. Relocate script unable to find ld-linux.so. Abort!"
+	exit 1
+fi
+executable_files=$($SUDO_EXEC find $native_sysroot -type f \
+	\( -perm -0100 -o -perm -0010 -o -perm -0001 \) -printf "'%h/%f' ")
+
+tdir=`mktemp -d`
+if [ x$tdir = x ] ; then
+   echo "SDK relocate failed, could not create a temporary directory"
+   exit 1
+fi
+echo "#!/bin/bash" > $tdir/relocate_sdk.sh
+echo exec ${env_setup_script%/*}/relocate_sdk.py $target_sdk_dir $dl_path $executable_files >> $tdir/relocate_sdk.sh
+$SUDO_EXEC mv $tdir/relocate_sdk.sh ${env_setup_script%/*}/relocate_sdk.sh
+$SUDO_EXEC chmod 755 ${env_setup_script%/*}/relocate_sdk.sh
+rm -rf $tdir
+if [ $relocate = 1 ] ; then
+	$SUDO_EXEC ${env_setup_script%/*}/relocate_sdk.sh
+	if [ $? -ne 0 ]; then
+		echo "SDK could not be set up. Relocate script failed. Abort!"
+		exit 1
+	fi
+fi
+
+# replace @SDKPATH@ with the new prefix in all text files: configs/scripts/etc
+for replace in "$target_sdk_dir -maxdepth 1" "$native_sysroot"; do
+	$SUDO_EXEC find $replace -type f -exec file '{}' \; | \
+		grep ":.*\(ASCII\|script\|source\).*text" | \
+		awk -F':' '{printf "\"%s\"\n", $1}' | \
+		grep -v "$target_sdk_dir/environment-setup-*" | \
+		$SUDO_EXEC xargs -n32 sed -i -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:g"
+done
+
+# change all symlinks pointing to @SDKPATH@
+for l in $($SUDO_EXEC find $native_sysroot -type l); do
+	$SUDO_EXEC ln -sfn $(readlink $l|$SUDO_EXEC sed -e "s:$DEFAULT_INSTALL_DIR:$target_sdk_dir:") $l
+done
+
+# find out all perl scripts in $native_sysroot and modify them replacing the
+# host perl with SDK perl.
+for perl_script in $($SUDO_EXEC find $native_sysroot -type f -exec grep -l "^#!.*perl" '{}' \;); do
+	$SUDO_EXEC sed -i -e "s:^#! */usr/bin/perl.*:#! /usr/bin/env perl:g" -e \
+		"s: /usr/bin/perl: /usr/bin/env perl:g" $perl_script
+done
+
+echo done
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 7/9] uninative-tarball: Actually use bzip2 for compression.
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (5 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 6/9] toolchain-shar-template.sh: Make relocation optional Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 8/9] scripts/oe-buildenv-internal: add means of skipping SDK check during setup Paul Eggleton
  2015-02-23 17:00 ` [PATCH 9/9] populate_sdk_ext: add extensible SDK Paul Eggleton
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

uninative.bbclass uses -xjf for decompression so actually run the data
through bzip2.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
---
 meta/recipes-core/meta/uninative-tarball.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/meta/uninative-tarball.bb b/meta/recipes-core/meta/uninative-tarball.bb
index e17685a..53435f2 100644
--- a/meta/recipes-core/meta/uninative-tarball.bb
+++ b/meta/recipes-core/meta/uninative-tarball.bb
@@ -44,5 +44,5 @@ fakeroot tar_sdk() {
 	rm sysroots -rf
 	patchelf --set-interpreter ${@''.join('a' for n in xrange(1024))} ./${BUILD_SYS}/usr/bin/patchelf
 	mv ./${BUILD_SYS}/usr/bin/patchelf ./${BUILD_SYS}/usr/bin/patchelf-uninative
-	tar ${SDKTAROPTS} -c --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
+	tar ${SDKTAROPTS} -c -j --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
 }
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 8/9] scripts/oe-buildenv-internal: add means of skipping SDK check during setup
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (6 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 7/9] uninative-tarball: Actually use bzip2 for compression Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:00 ` [PATCH 9/9] populate_sdk_ext: add extensible SDK Paul Eggleton
  8 siblings, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

The oe-buildenv-internal script checks if the user is already in an sdk
environment and errors if true. Add a way to skip this check.

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 scripts/oe-buildenv-internal | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index bba6f8f..9ed2721 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -24,7 +24,7 @@ if [ -z "$OEROOT" ]; then
     return 1
 fi
 
-if [ ! -z "$OECORE_SDK_VERSION" ]; then
+if [ -z "$OE_SKIP_SDK_CHECK" -a ! -z "$OECORE_SDK_VERSION" ]; then
     echo >&2 "Error: The OE SDK/ADT was detected as already being present in this shell environment. Please use a clean shell when sourcing this environment script."
     return 1
 fi
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
                   ` (7 preceding siblings ...)
  2015-02-23 17:00 ` [PATCH 8/9] scripts/oe-buildenv-internal: add means of skipping SDK check during setup Paul Eggleton
@ 2015-02-23 17:00 ` Paul Eggleton
  2015-02-23 17:41   ` Otavio Salvador
  8 siblings, 1 reply; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:00 UTC (permalink / raw)
  To: openembedded-core

From: Randy Witt <randy.e.witt@linux.intel.com>

This bbclass will create an SDK with a copy of bitbake and the metadata
and sstate for the target specified for the task. The idea is to let
"system" developers both work on applications and then test adding them
to an image without having to switch between workspaces or having to
download separate items.

Rather than running bitbake directly however, the primary way of running
builds within the extensible SDK is to use the "devtool" command. The
rest of the build system is fixed via locked shared state signatures,
and thus only the recipes you have added get built.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/image.bbclass                        |   2 +-
 meta/classes/populate_sdk_ext.bbclass             | 217 ++++++++++++++++++++++
 meta/recipes-core/meta/meta-environment-extsdk.bb |  12 ++
 3 files changed, 230 insertions(+), 1 deletion(-)
 create mode 100644 meta/classes/populate_sdk_ext.bbclass
 create mode 100644 meta/recipes-core/meta/meta-environment-extsdk.bb

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 893eb40..89eb5f3 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -1,6 +1,6 @@
 inherit rootfs_${IMAGE_PKGTYPE}
 
-inherit populate_sdk_base
+inherit populate_sdk_ext
 
 TOOLCHAIN_TARGET_TASK += "${PACKAGE_INSTALL}"
 TOOLCHAIN_TARGET_TASK_ATTEMPTONLY += "${PACKAGE_INSTALL_ATTEMPTONLY}"
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
new file mode 100644
index 0000000..ec1cff0
--- /dev/null
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -0,0 +1,217 @@
+# Extensible SDK
+
+inherit populate_sdk_base
+
+# NOTE: normally you cannot use task overrides for this kind of thing - this
+# only works because of get_sdk_ext_rdepends()
+
+TOOLCHAIN_HOST_TASK_task-populate-sdk-ext = " \
+    meta-environment-extsdk-${MACHINE} \
+    "
+
+TOOLCHAIN_TARGET_TASK_task-populate-sdk-ext = ""
+
+SDK_RDEPENDS_append_task-populate-sdk-ext = " ${SDK_TARGETS}"
+
+SDK_RELOCATE_AFTER_INSTALL_task-populate-sdk-ext = "0"
+
+SDK_META_CONF_WHITELIST ?= "MACHINE DISTRO PACKAGE_CLASSES"
+
+SDK_TARGETS ?= "${PN}"
+OE_INIT_ENV_SCRIPT ?= "oe-init-build-env"
+
+# The files from COREBASE that you want preserved in the COREBASE copied
+# into the sdk. This allows someone to have their own setup scripts in
+# COREBASE be preserved as well as untracked files.
+COREBASE_FILES ?= " \
+    oe-init-build-env \
+    oe-init-build-env-memres \
+    scripts \
+    LICENSE \
+    .templateconf \
+"
+
+SDK_DIR_task-populate-sdk-ext = "${WORKDIR}/sdk-ext"
+B_task-populate-sdk-ext = "${SDK_DIR}"
+TOOLCHAIN_OUTPUTNAME_task-populate-sdk-ext = "${SDK_NAME}-toolchain-ext-${SDK_VERSION}"
+
+python copy_buildsystem () {
+    import re
+    import oe.copy_buildsystem
+
+    oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True)
+
+    conf_bbpath = ''
+    conf_initpath = ''
+    core_meta_subdir = ''
+
+    # Copy in all metadata layers + bitbake (as repositories)
+    buildsystem = oe.copy_buildsystem.BuildSystem(d)
+    baseoutpath = d.getVar('SDK_OUTPUT', True) + '/' + d.getVar('SDKPATH', True)
+    layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath + '/layers')
+
+    sdkbblayers = []
+    corebase = os.path.basename(d.getVar('COREBASE', True))
+    for layer in layers_copied:
+        if corebase == os.path.basename(layer):
+            conf_bbpath = os.path.join('layers', layer, 'bitbake')
+        else:
+            sdkbblayers.append(layer)
+
+    for path in os.listdir(baseoutpath + '/layers'):
+        relpath = os.path.join('layers', path, oe_init_env_script)
+        if os.path.exists(os.path.join(baseoutpath, relpath)):
+            conf_initpath = relpath
+
+        relpath = os.path.join('layers', path, 'scripts', 'devtool')
+        if os.path.exists(os.path.join(baseoutpath, relpath)):
+            scriptrelpath = os.path.dirname(relpath)
+
+        relpath = os.path.join('layers', path, 'meta')
+        if os.path.exists(os.path.join(baseoutpath, relpath, 'lib', 'oe')):
+            core_meta_subdir = relpath
+
+    d.setVar('oe_init_build_env_path', conf_initpath)
+    d.setVar('scriptrelpath', scriptrelpath)
+
+    # Write out config file for devtool
+    import ConfigParser
+    config = ConfigParser.SafeConfigParser()
+    config.add_section('General')
+    config.set('General', 'bitbake_subdir', conf_bbpath)
+    config.set('General', 'init_path', conf_initpath)
+    config.set('General', 'core_meta_subdir', core_meta_subdir)
+    bb.utils.mkdirhier(os.path.join(baseoutpath, 'conf'))
+    with open(os.path.join(baseoutpath, 'conf', 'devtool.conf'), 'w') as f:
+        config.write(f)
+
+    # Create a layer for new recipes / appends
+    bb.process.run("devtool --basepath %s create-workspace --create-only %s" % (baseoutpath, os.path.join(baseoutpath, 'workspace')))
+
+    # Create bblayers.conf
+    bb.utils.mkdirhier(baseoutpath + '/conf')
+    with open(baseoutpath + '/conf/bblayers.conf', 'w') as f:
+        f.write('LCONF_VERSION = "%s"\n\n' % d.getVar('LCONF_VERSION'))
+        f.write('BBPATH = "$' + '{TOPDIR}"\n')
+        f.write('SDKBASEMETAPATH = "$' + '{TOPDIR}"\n')
+        f.write('BBLAYERS := " \\\n')
+        for layerrelpath in sdkbblayers:
+            f.write('    $' + '{SDKBASEMETAPATH}/layers/%s \\\n' % layerrelpath)
+        f.write('    $' + '{SDKBASEMETAPATH}/workspace \\\n')
+        f.write('    "\n')
+
+    # Create local.conf
+    with open(baseoutpath + '/conf/local.conf', 'w') as f:
+        f.write('INHERIT += "%s"\n\n' % 'uninative')
+        f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION'))
+
+        # This is a bit of a hack, but we really don't want these dependencies
+        # (we're including them in the SDK as nativesdk- versions instead)
+        f.write('POKYQEMUDEPS_forcevariable = ""\n\n')
+        f.write('EXTRA_IMAGEDEPENDS_remove = "qemu-native qemu-helper-native"\n\n')
+
+        # Another hack, but we want the native part of sstate to be kept the same
+        # regardless of the host distro
+        fixedlsbstring = 'SDK-Fixed'
+        f.write('NATIVELSBSTRING_forcevariable = "%s"\n\n' % fixedlsbstring)
+
+        # Ensure locked sstate cache objects are re-used without error
+        f.write('SIGGEN_LOCKEDSIGS_CHECK_LEVEL = "warn"\n\n')
+
+        for varname in d.getVar('SDK_META_CONF_WHITELIST', True).split():
+            f.write('%s = "%s"\n' % (varname, d.getVar(varname, True)))
+        f.write('require conf/locked-sigs.inc\n')
+        f.write('require conf/work-config.inc\n')
+
+    sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
+    oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
+
+    # Filter the locked signatures file to just the sstate tasks we are interested in
+    allowed_tasks = ['do_populate_lic', 'do_populate_sysroot', 'do_packagedata', 'do_package_write_ipk', 'do_package_write_rpm', 'do_package_write_deb', 'do_package_qa', 'do_deploy']
+    excluded_targets = d.getVar('SDK_TARGETS', True)
+    lockedsigs_pruned = baseoutpath + '/conf/locked-sigs.inc'
+    oe.copy_buildsystem.prune_lockedsigs(allowed_tasks,
+                                         excluded_targets,
+                                         sigfile,
+                                         lockedsigs_pruned)
+
+    sstate_out = baseoutpath + '/sstate-cache'
+    bb.utils.remove(sstate_out, True)
+    oe.copy_buildsystem.create_locked_sstate_cache(lockedsigs_pruned,
+                                                   d.getVar('SSTATE_DIR', True),
+                                                   sstate_out, d,
+                                                   fixedlsbstring)
+
+    # Create a dummy config file for additional settings
+    with open(baseoutpath + '/conf/work-config.inc', 'w') as f:
+        pass
+}
+
+install_tools() {
+	install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}
+	ln -sr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/devtool ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/devtool
+	ln -sr ${SDK_OUTPUT}/${SDKPATH}/${scriptrelpath}/recipetool ${SDK_OUTPUT}/${SDKPATHNATIVE}${bindir_nativesdk}/recipetool
+	touch ${SDK_OUTPUT}/${SDKPATH}/.devtoolbase
+
+	install ${SDK_DEPLOY}/${DISTRO}-${TCLIBC}-${SDK_ARCH}-buildtools-tarball-${TUNE_PKGARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}.sh ${SDK_OUTPUT}/${SDKPATH}
+
+	install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2 ${SDK_OUTPUT}/${SDKPATH}
+}
+
+# FIXME this preparation should be done as part of the SDK construction
+sdk_ext_postinst() {
+	printf "\nExtracting buildtools...\n"
+	cd $target_sdk_dir
+	printf "buildtools\ny" | ./*buildtools-tarball* > /dev/null
+
+	# Make sure when the user sets up the environment, they also get
+	# the buildtools-tarball tools in their path.
+	echo ". $target_sdk_dir/buildtools/environment-setup*" >> $target_sdk_dir/environment-setup*
+
+	# Allow bitbake environment setup to be ran as part of this sdk.
+	echo "export OE_SKIP_SDK_CHECK=1" >> $target_sdk_dir/environment-setup*
+
+	# A bit of another hack, but we need this in the path only for devtool
+	# so put it at the end of $PATH.
+	echo "export PATH=\$PATH:$target_sdk_dir/sysroots/${SDK_SYS}/${bindir_nativesdk}" >> $target_sdk_dir/environment-setup*
+
+	# For now this is where uninative.bbclass expects the tarball
+	mv *-nativesdk-libc.tar.* $target_sdk_dir/`dirname ${oe_init_build_env_path}`
+
+	printf "Preparing build system...\n"
+	# dash which is /bin/sh on Ubuntu will not preserve the
+	# current working directory when first ran, nor will it set $1 when
+	# sourcing a script. That is why this has to look so ugly.
+	sh -c ". buildtools/environment-setup* > /dev/null && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir > /dev/null && bitbake ${SDK_TARGETS} > /dev/null" || { echo "SDK preparation failed" ; exit 1 ; }
+	echo done
+}
+
+SDK_POST_INSTALL_COMMAND_task-populate-sdk-ext = "${sdk_ext_postinst}"
+
+SDK_POSTPROCESS_COMMAND_prepend_task-populate-sdk-ext = "copy_buildsystem; install_tools; "
+
+fakeroot python do_populate_sdk_ext() {
+    bb.build.exec_func("do_populate_sdk", d)
+}
+
+def get_sdk_ext_rdepends(d):
+    localdata = d.createCopy()
+    localdata.appendVar('OVERRIDES', ':task-populate-sdk-ext')
+    bb.data.update_data(localdata)
+    return localdata.getVarFlag('do_populate_sdk', 'rdepends', True)
+
+do_populate_sdk_ext[dirs] = "${@d.getVarFlag('do_populate_sdk', 'dirs', False)}"
+do_populate_sdk_ext[depends] += "${@d.getVarFlag('do_populate_sdk', 'depends', False)}"
+do_populate_sdk_ext[rdepends] = "${@get_sdk_ext_rdepends(d)}"
+do_populate_sdk_ext[recrdeptask] += "${@d.getVarFlag('do_populate_sdk', 'recrdeptask', False)}"
+
+
+do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
+
+do_populate_sdk_ext[rdepends] += "${@' '.join([x + ':do_build' for x in d.getVar('SDK_TARGETS', True).split()])}"
+do_populate_sdk_ext[recrdeptask] += "do_populate_lic do_package_qa do_populate_sysroot do_deploy"
+
+# Make sure codes change in copy_buildsystem can result in rebuilt
+do_populate_sdk_ext[vardeps] += "copy_buildsystem"
+
+addtask populate_sdk_ext
diff --git a/meta/recipes-core/meta/meta-environment-extsdk.bb b/meta/recipes-core/meta/meta-environment-extsdk.bb
new file mode 100644
index 0000000..d9e5961
--- /dev/null
+++ b/meta/recipes-core/meta/meta-environment-extsdk.bb
@@ -0,0 +1,12 @@
+# meta-environment for extensible SDK
+
+require meta-environment.bb
+
+PN = "meta-environment-extsdk-${MACHINE}"
+
+create_sdk_files_append() {
+	local sysroot=${SDKPATH}/${@os.path.relpath(d.getVar('STAGING_DIR_TARGET', True), d.getVar('TOPDIR', True))}
+	local sdkpathnative=${SDKPATH}/${@os.path.relpath(d.getVar('STAGING_DIR_NATIVE',True), d.getVar('TOPDIR', True))}
+
+	toolchain_create_sdk_env_script '' '' $sysroot '' ${bindir_native} ${prefix_native} $sdkpathnative
+}
-- 
1.9.3



^ permalink raw reply related	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 17:00 ` [PATCH 9/9] populate_sdk_ext: add extensible SDK Paul Eggleton
@ 2015-02-23 17:41   ` Otavio Salvador
  2015-02-23 17:54     ` Paul Eggleton
  0 siblings, 1 reply; 17+ messages in thread
From: Otavio Salvador @ 2015-02-23 17:41 UTC (permalink / raw)
  To: Paul Eggleton; +Cc: Patches and discussions about the oe-core layer

On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> From: Randy Witt <randy.e.witt@linux.intel.com>
>
> This bbclass will create an SDK with a copy of bitbake and the metadata
> and sstate for the target specified for the task. The idea is to let
> "system" developers both work on applications and then test adding them
> to an image without having to switch between workspaces or having to
> download separate items.
>
> Rather than running bitbake directly however, the primary way of running
> builds within the extensible SDK is to use the "devtool" command. The
> rest of the build system is fixed via locked shared state signatures,
> and thus only the recipes you have added get built.
>
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
> Signed-off-by: Chen Qi <Qi.Chen@windriver.com>

Why another class? it could be tuned using SDKIMAGE_FEATURES

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 17:41   ` Otavio Salvador
@ 2015-02-23 17:54     ` Paul Eggleton
  2015-02-23 18:00       ` Otavio Salvador
  0 siblings, 1 reply; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 17:54 UTC (permalink / raw)
  To: Otavio Salvador; +Cc: Patches and discussions about the oe-core layer

On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
> On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > From: Randy Witt <randy.e.witt@linux.intel.com>
> > 
> > This bbclass will create an SDK with a copy of bitbake and the metadata
> > and sstate for the target specified for the task. The idea is to let
> > "system" developers both work on applications and then test adding them
> > to an image without having to switch between workspaces or having to
> > download separate items.
> > 
> > Rather than running bitbake directly however, the primary way of running
> > builds within the extensible SDK is to use the "devtool" command. The
> > rest of the build system is fixed via locked shared state signatures,
> > and thus only the recipes you have added get built.
> > 
> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> > Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> 
> Why another class? it could be tuned using SDKIMAGE_FEATURES

If you look at what the class does it would be a bit messy to do it that way - 
I wanted to get something working with minimal impact. I don't doubt
that it could be implemented as an SDKIMAGE_FEATURES item though
with extra work.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 17:54     ` Paul Eggleton
@ 2015-02-23 18:00       ` Otavio Salvador
  2015-02-23 18:06         ` Richard Purdie
  0 siblings, 1 reply; 17+ messages in thread
From: Otavio Salvador @ 2015-02-23 18:00 UTC (permalink / raw)
  To: Paul Eggleton; +Cc: Patches and discussions about the oe-core layer

On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
>> On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
>> <paul.eggleton@linux.intel.com> wrote:
>> > From: Randy Witt <randy.e.witt@linux.intel.com>
>> >
>> > This bbclass will create an SDK with a copy of bitbake and the metadata
>> > and sstate for the target specified for the task. The idea is to let
>> > "system" developers both work on applications and then test adding them
>> > to an image without having to switch between workspaces or having to
>> > download separate items.
>> >
>> > Rather than running bitbake directly however, the primary way of running
>> > builds within the extensible SDK is to use the "devtool" command. The
>> > rest of the build system is fixed via locked shared state signatures,
>> > and thus only the recipes you have added get built.
>> >
>> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>> > Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
>> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>>
>> Why another class? it could be tuned using SDKIMAGE_FEATURES
>
> If you look at what the class does it would be a bit messy to do it that way -
> I wanted to get something working with minimal impact. I don't doubt
> that it could be implemented as an SDKIMAGE_FEATURES item though
> with extra work.

As far this does not gets merged I am fine with that. This is clearly
a WIP code and shouldn't be merged as is.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 18:00       ` Otavio Salvador
@ 2015-02-23 18:06         ` Richard Purdie
  2015-02-23 18:11           ` Otavio Salvador
  0 siblings, 1 reply; 17+ messages in thread
From: Richard Purdie @ 2015-02-23 18:06 UTC (permalink / raw)
  To: Otavio Salvador
  Cc: Paul Eggleton, Patches and discussions about the oe-core layer

On Mon, 2015-02-23 at 15:00 -0300, Otavio Salvador wrote:
> On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
> >> On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
> >> <paul.eggleton@linux.intel.com> wrote:
> >> > From: Randy Witt <randy.e.witt@linux.intel.com>
> >> >
> >> > This bbclass will create an SDK with a copy of bitbake and the metadata
> >> > and sstate for the target specified for the task. The idea is to let
> >> > "system" developers both work on applications and then test adding them
> >> > to an image without having to switch between workspaces or having to
> >> > download separate items.
> >> >
> >> > Rather than running bitbake directly however, the primary way of running
> >> > builds within the extensible SDK is to use the "devtool" command. The
> >> > rest of the build system is fixed via locked shared state signatures,
> >> > and thus only the recipes you have added get built.
> >> >
> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> >> > Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
> >> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> >>
> >> Why another class? it could be tuned using SDKIMAGE_FEATURES
> >
> > If you look at what the class does it would be a bit messy to do it that way -
> > I wanted to get something working with minimal impact. I don't doubt
> > that it could be implemented as an SDKIMAGE_FEATURES item though
> > with extra work.
> 
> As far this does not gets merged I am fine with that. This is clearly
> a WIP code and shouldn't be merged as is.

This code adds in an alternative SDK format and it drives that using a
different task target. Right now its hard for people to see where things
are going, this puts it in easy reach whilst allowing the current SDK to
continue to work too.

I think a choice of two different task targets here makes sense to drive
this configuration rather than SDKIMAGE_FEATURES and I agree with Paul
that adding it the other way would be complex, error prone and
confusing.

Cheers,

Richard




^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 18:06         ` Richard Purdie
@ 2015-02-23 18:11           ` Otavio Salvador
  2015-02-23 18:29             ` Paul Eggleton
  2015-02-23 22:33             ` Richard Purdie
  0 siblings, 2 replies; 17+ messages in thread
From: Otavio Salvador @ 2015-02-23 18:11 UTC (permalink / raw)
  To: Richard Purdie
  Cc: Paul Eggleton, Patches and discussions about the oe-core layer

On Mon, Feb 23, 2015 at 3:06 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Mon, 2015-02-23 at 15:00 -0300, Otavio Salvador wrote:
>> On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton
>> <paul.eggleton@linux.intel.com> wrote:
>> > On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
>> >> On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
>> >> <paul.eggleton@linux.intel.com> wrote:
>> >> > From: Randy Witt <randy.e.witt@linux.intel.com>
>> >> >
>> >> > This bbclass will create an SDK with a copy of bitbake and the metadata
>> >> > and sstate for the target specified for the task. The idea is to let
>> >> > "system" developers both work on applications and then test adding them
>> >> > to an image without having to switch between workspaces or having to
>> >> > download separate items.
>> >> >
>> >> > Rather than running bitbake directly however, the primary way of running
>> >> > builds within the extensible SDK is to use the "devtool" command. The
>> >> > rest of the build system is fixed via locked shared state signatures,
>> >> > and thus only the recipes you have added get built.
>> >> >
>> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>> >> > Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
>> >> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
>> >>
>> >> Why another class? it could be tuned using SDKIMAGE_FEATURES
>> >
>> > If you look at what the class does it would be a bit messy to do it that way -
>> > I wanted to get something working with minimal impact. I don't doubt
>> > that it could be implemented as an SDKIMAGE_FEATURES item though
>> > with extra work.
>>
>> As far this does not gets merged I am fine with that. This is clearly
>> a WIP code and shouldn't be merged as is.
>
> This code adds in an alternative SDK format and it drives that using a
> different task target. Right now its hard for people to see where things
> are going, this puts it in easy reach whilst allowing the current SDK to
> continue to work too.
>
> I think a choice of two different task targets here makes sense to drive
> this configuration rather than SDKIMAGE_FEATURES and I agree with Paul
> that adding it the other way would be complex, error prone and
> confusing.

Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.

Sorry ... it is clear that people wish this feature (and I also do)
and appreciate the hard work Paul has put on this (yes this is indeed
a good amount of work) but the extra task does not seem to fit how the
system is designed.

The SDKIMAGE_FEATURES[1] is described as:


Equivalent to IMAGE_FEATURES. However, this variable applies to the
SDK generated from an image using the following command:

     $ bitbake -c populate_sdk imagename

and it makes a lot of sense to have a 'extended' or 'devtool' or
another name for this feature.

1. http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SDKIMAGE_FEATURES

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 18:11           ` Otavio Salvador
@ 2015-02-23 18:29             ` Paul Eggleton
  2015-02-23 22:33             ` Richard Purdie
  1 sibling, 0 replies; 17+ messages in thread
From: Paul Eggleton @ 2015-02-23 18:29 UTC (permalink / raw)
  To: Otavio Salvador; +Cc: openembedded-core

On Monday 23 February 2015 15:11:38 Otavio Salvador wrote:
> On Mon, Feb 23, 2015 at 3:06 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Mon, 2015-02-23 at 15:00 -0300, Otavio Salvador wrote:
> >> On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton
> >> 
> >> <paul.eggleton@linux.intel.com> wrote:
> >> > On Monday 23 February 2015 14:41:34 Otavio Salvador wrote:
> >> >> On Mon, Feb 23, 2015 at 2:00 PM, Paul Eggleton
> >> >> 
> >> >> <paul.eggleton@linux.intel.com> wrote:
> >> >> > From: Randy Witt <randy.e.witt@linux.intel.com>
> >> >> > 
> >> >> > This bbclass will create an SDK with a copy of bitbake and the
> >> >> > metadata
> >> >> > and sstate for the target specified for the task. The idea is to let
> >> >> > "system" developers both work on applications and then test adding
> >> >> > them
> >> >> > to an image without having to switch between workspaces or having to
> >> >> > download separate items.
> >> >> > 
> >> >> > Rather than running bitbake directly however, the primary way of
> >> >> > running
> >> >> > builds within the extensible SDK is to use the "devtool" command.
> >> >> > The
> >> >> > rest of the build system is fixed via locked shared state
> >> >> > signatures,
> >> >> > and thus only the recipes you have added get built.
> >> >> > 
> >> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> >> >> > Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
> >> >> > Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
> >> >> 
> >> >> Why another class? it could be tuned using SDKIMAGE_FEATURES
> >> > 
> >> > If you look at what the class does it would be a bit messy to do it
> >> > that way - I wanted to get something working with minimal impact. I
> >> > don't doubt that it could be implemented as an SDKIMAGE_FEATURES item
> >> > though
> >> > with extra work.
> >> 
> >> As far this does not gets merged I am fine with that. This is clearly
> >> a WIP code and shouldn't be merged as is.
> > 
> > This code adds in an alternative SDK format and it drives that using a
> > different task target. Right now its hard for people to see where things
> > are going, this puts it in easy reach whilst allowing the current SDK to
> > continue to work too.
> > 
> > I think a choice of two different task targets here makes sense to drive
> > this configuration rather than SDKIMAGE_FEATURES and I agree with Paul
> > that adding it the other way would be complex, error prone and
> > confusing.
> 
> Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.
>
> Sorry ... it is clear that people wish this feature (and I also do)
> and appreciate the hard work Paul has put on this (yes this is indeed
> a good amount of work) but the extra task does not seem to fit how the
> system is designed.
> 
> The SDKIMAGE_FEATURES[1] is described as:
> 
> Equivalent to IMAGE_FEATURES. However, this variable applies to the
> SDK generated from an image using the following command:
> 
>      $ bitbake -c populate_sdk imagename
> 
> and it makes a lot of sense to have a 'extended' or 'devtool' or
> another name for this feature.
> 
> 1.
> http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SDK
> IMAGE_FEATURES

I see where you are coming from, but again if we look at what the class does, 
it's not just adding some packages. It fundamentally changes how the SDK works 
- you don't actually have a nativesdk toolchain (in fact you don't have any 
nativesdk packages at all) and you don't have anything in the target sysroot 
either - because the build system sysroots pretty much contain everything 
already. It's not simply an add-on or a post-processing of the SDK.

With the way it's been implemented, you get a separate way of building it and 
a separate SDK installer, and therefore it's a little clearer as to what you 
are building/installing. At the moment, with this being very new 
functionality, that seems to me to be the right way to do things for now. If 
this turns out to work really well for everyone then great, we can look at 
integrating it more tightly.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre


^ permalink raw reply	[flat|nested] 17+ messages in thread
* Re: [PATCH 9/9] populate_sdk_ext: add extensible SDK
  2015-02-23 18:11           ` Otavio Salvador
  2015-02-23 18:29             ` Paul Eggleton
@ 2015-02-23 22:33             ` Richard Purdie
  1 sibling, 0 replies; 17+ messages in thread
From: Richard Purdie @ 2015-02-23 22:33 UTC (permalink / raw)
  To: Otavio Salvador
  Cc: Paul Eggleton, Patches and discussions about the oe-core layer

On Mon, 2015-02-23 at 15:11 -0300, Otavio Salvador wrote:
> On Mon, Feb 23, 2015 at 3:06 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Mon, 2015-02-23 at 15:00 -0300, Otavio Salvador wrote:
> >> On Mon, Feb 23, 2015 at 2:54 PM, Paul Eggleton
> > This code adds in an alternative SDK format and it drives that using a
> > different task target. Right now its hard for people to see where things
> > are going, this puts it in easy reach whilst allowing the current SDK to
> > continue to work too.
> >
> > I think a choice of two different task targets here makes sense to drive
> > this configuration rather than SDKIMAGE_FEATURES and I agree with Paul
> > that adding it the other way would be complex, error prone and
> > confusing.
> 
> Well so why we don't add populate_sdk+dev-pkgs+dbg-pkgs.
> 
> Sorry ... it is clear that people wish this feature (and I also do)
> and appreciate the hard work Paul has put on this (yes this is indeed
> a good amount of work) but the extra task does not seem to fit how the
> system is designed.
> 
> The SDKIMAGE_FEATURES[1] is described as:
> 
> 
> Equivalent to IMAGE_FEATURES. However, this variable applies to the
> SDK generated from an image using the following command:
> 
>      $ bitbake -c populate_sdk imagename
> 
> and it makes a lot of sense to have a 'extended' or 'devtool' or
> another name for this feature.

I see this as that SDKIMAGE_FEATURES controls the content of the SDK,
not the actual construction of it so I'm afraid I don't agree.

Cheers,

Richard






^ permalink raw reply	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2015-02-23 22:33 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-02-23 17:00 [PATCH 0/9] Extensible SDK Paul Eggleton
2015-02-23 17:00 ` [PATCH 1/9] gen-lockedsig-cache: Allow cross-filesystem copies Paul Eggleton
2015-02-23 17:00 ` [PATCH 2/9] sstatesig: Add ability to filter list of tasks for locked-sigs.inc Paul Eggleton
2015-02-23 17:00 ` [PATCH 3/9] copy_buildsystem.py: Add a way to copy buildsystem to a directory Paul Eggleton
2015-02-23 17:00 ` [PATCH 4/9] copy_buildsystem.py: Add methods to copy shared state Paul Eggleton
2015-02-23 17:00 ` [PATCH 5/9] toolchain-scripts: Add parameters to toolchain_create_sdk_env_script Paul Eggleton
2015-02-23 17:00 ` [PATCH 6/9] toolchain-shar-template.sh: Make relocation optional Paul Eggleton
2015-02-23 17:00 ` [PATCH 7/9] uninative-tarball: Actually use bzip2 for compression Paul Eggleton
2015-02-23 17:00 ` [PATCH 8/9] scripts/oe-buildenv-internal: add means of skipping SDK check during setup Paul Eggleton
2015-02-23 17:00 ` [PATCH 9/9] populate_sdk_ext: add extensible SDK Paul Eggleton
2015-02-23 17:41   ` Otavio Salvador
2015-02-23 17:54     ` Paul Eggleton
2015-02-23 18:00       ` Otavio Salvador
2015-02-23 18:06         ` Richard Purdie
2015-02-23 18:11           ` Otavio Salvador
2015-02-23 18:29             ` Paul Eggleton
2015-02-23 22:33             ` Richard Purdie

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.