[PATCH 0/6] Improve oe-init-build-env/-memres

[PATCH 0/6] Improve oe-init-build-env/-memres

[Peter Kjellerstedt]

I started looking at the oe-init-build-env (and
oe-init-build-env-memres) scripts since I wanted to add support for
specifying $OEROOT from the outside. This is because we want to wrap
those scripts with our own while keeping the official names (so anyone
reading the official Poky documentation will not be too confused).

While fixing the above, I had noticed a number of inconsistencies in
the scripts, so I continued to fix those. I also improved the code
that adds paths to $PATH, and the code that adds variable names to
$BB_ENV_EXTRAWHITE.

//Peter

The following changes since commit 8debfea81e69d038bd2d56314b272cb74f5582ed:

  local.conf.sample: Disable prelink by default (2016-03-13 22:09:05 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib pkj/oe-init-build-env
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/oe-init-build-env

Peter Kjellerstedt (6):
  oe-init-build-env*: Allow $OEROOT to be predefined
  oe-init-build-env*: Update/correct comment about specifying arguments
  oe-init-build-env*: Remove unnecessary differences between the scripts
  oe-buildenv-internal: Add paths to $PATH individually
  oe-buildenv-internal: Add variables individually to BB_ENV_EXTRAWHITE
  oe-buildenv-internal: Some clean up

 oe-init-build-env            | 45 ++++++++++++------------
 oe-init-build-env-memres     | 54 ++++++++++++++--------------
 scripts/oe-buildenv-internal | 83 ++++++++++++++++++++++++++++----------------
 3 files changed, 102 insertions(+), 80 deletions(-)

-- 
2.1.0

[PATCH 1/6] oe-init-build-env*: Allow $OEROOT to be predefined

[Peter Kjellerstedt]

The current implementation of oe-init-build-env and
oe-init-build-env-memres requires that they are sourced from the
directory that will be known as $OEROOT. This makes it hard to write a
wrapper script with the same name as the original OE script which,
e.g., sources the original OE script from a sub-directory.

With this change, $OEROOT can be predefined when oe-init-build-env or
oe-init-build-env-memres is sourced, allowing the original OE scripts
to be anywhere.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 oe-init-build-env        | 16 ++++++++--------
 oe-init-build-env-memres | 16 ++++++++--------
 2 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/oe-init-build-env b/oe-init-build-env
index b7e2918..f511b57 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -26,30 +26,30 @@
 # to sourcing this script.
 #
 if [ -n "$BASH_SOURCE" ]; then
-   OEROOT="`dirname $BASH_SOURCE`"
+    THIS_SCRIPT=$BASH_SOURCE
 elif [ -n "$ZSH_NAME" ]; then
-   OEROOT="`dirname $0`"
+    THIS_SCRIPT=$0
 else
-   OEROOT="`pwd`"
+    THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
    unset BBSERVER
 fi
-THIS_SCRIPT=$OEROOT/oe-init-build-env
 
 if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
    exit 1
 fi
 
-OEROOT=`readlink -f "$OEROOT"`
+if [ -z "$OEROOT" ]; then
+    OEROOT=$(dirname "$THIS_SCRIPT")
+    OEROOT=$(readlink -f "$OEROOT")
+fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
      TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
      [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
-unset OEROOT
-unset BBPATH
-unset THIS_SCRIPT
+unset THIS_SCRIPT OEROOT BBPATH
 
 # Shutdown any bitbake server if the BBSERVER variable is not set
 if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
diff --git a/oe-init-build-env-memres b/oe-init-build-env-memres
index c1dc4fe..6dd2b13 100755
--- a/oe-init-build-env-memres
+++ b/oe-init-build-env-memres
@@ -34,30 +34,30 @@ else
 fi
 
 if [ -n "$BASH_SOURCE" ]; then
-  OEROOT="`dirname $BASH_SOURCE`"
+    THIS_SCRIPT=$BASH_SOURCE
 elif [ -n "$ZSH_NAME" ]; then
-  OEROOT="`dirname $0`"
+    THIS_SCRIPT=$0
 else
-  OEROOT="`pwd`"
+    THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
   unset BBSERVER
 fi
 
-THIS_SCRIPT=$OEROOT/oe-init-build-env-memres
 if [ -z "$ZSH_NAME" ] && [ "x$0" = "x$THIS_SCRIPT" ]; then
    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
    exit 1
 fi
 
-OEROOT=`readlink -f "$OEROOT"`
+if [ -z "$OEROOT" ]; then
+    OEROOT=$(dirname "$THIS_SCRIPT")
+    OEROOT=$(readlink -f "$OEROOT")
+fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
     $OEROOT/scripts/oe-setup-builddir && \
     [ -n "$BUILDDIR" ] && cd $BUILDDIR
-unset OEROOT
-unset BBPATH
-unset THIS_SCRIPT
+unset THIS_SCRIPT OEROOT BBPATH
 
 res=1
 if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null ; then
-- 
2.1.0

[PATCH 2/6] oe-init-build-env*: Update/correct comment about specifying arguments

[Peter Kjellerstedt]

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 oe-init-build-env        | 8 ++++----
 oe-init-build-env-memres | 8 ++++----
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/oe-init-build-env b/oe-init-build-env
index f511b57..baab827 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -19,11 +19,11 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 #
-# Normally this is called as '. ./oe-init-build-env builddir'
+# Normally this is called as '. ./oe-init-build-env <builddir>'
 #
-# This works in most shells (not dash), but not all of them pass arg1 when
-# being sourced.   To workaround the shell limitation use "set arg1" prior 
-# to sourcing this script.
+# This works in most shells (not dash), but not all of them pass the arguments
+# when being sourced.  To workaround the shell limitation use "set <builddir>"
+# prior to sourcing this script.
 #
 if [ -n "$BASH_SOURCE" ]; then
     THIS_SCRIPT=$BASH_SOURCE
diff --git a/oe-init-build-env-memres b/oe-init-build-env-memres
index 6dd2b13..b0c7421 100755
--- a/oe-init-build-env-memres
+++ b/oe-init-build-env-memres
@@ -19,11 +19,11 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 #
-# Normally this is called as '. ./oe-init-build-env builddir <portnumber> <builddir>'
+# Normally this is called as '. ./oe-init-build-env-memres <portnumber> <builddir>'
 #
-# This works in most shells (not dash), but not all of them pass arg1 when
-# being sourced.   To workaround the shell limitation use "set arg1" prior 
-# to sourcing this script.
+# This works in most shells (not dash), but not all of them pass the arguments
+# when being sourced.  To workaround the shell limitation use "set <portnumber>
+# <builddir>" prior to sourcing this script.
 #
 if [ -z "$1" ]; then
     echo "No port specified, using dynamically selected port"
-- 
2.1.0

[PATCH 3/6] oe-init-build-env*: Remove unnecessary differences between the scripts

[Peter Kjellerstedt]

While at it, also fix:
* consistent indentation (four spaces)
* unset temporary variables
* use $(...) instead of `...`

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 oe-init-build-env        | 21 ++++++++++-----------
 oe-init-build-env-memres | 30 +++++++++++++++---------------
 2 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/oe-init-build-env b/oe-init-build-env
index baab827..53873e6 100755
--- a/oe-init-build-env
+++ b/oe-init-build-env
@@ -33,12 +33,12 @@ else
     THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
-   unset BBSERVER
+    unset BBSERVER
 fi
 
 if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
-   echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
-   exit 1
+    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
+    exit 1
 fi
 
 if [ -z "$OEROOT" ]; then
@@ -47,16 +47,15 @@ if [ -z "$OEROOT" ]; then
 fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
-     TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
-     [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
+    TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
+    [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
 unset THIS_SCRIPT OEROOT BBPATH
 
 # Shutdown any bitbake server if the BBSERVER variable is not set
-if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
-    grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
-    if [ $? = 0 ] ; then
-	echo "Shutting down bitbake memory resident server with bitbake -m"
-	BBSERVER=`cat bitbake.lock` bitbake -m
+if [ -z "$BBSERVER" ] && [ -f bitbake.lock ]; then
+    grep ":" bitbake.lock > /dev/null && BBSERVER=$(cat bitbake.lock) bitbake --status-only
+    if [ $? = 0 ]; then
+        echo "Shutting down bitbake memory resident server with bitbake -m"
+        BBSERVER=$(cat bitbake.lock) bitbake -m
     fi
 fi
-
diff --git a/oe-init-build-env-memres b/oe-init-build-env-memres
index b0c7421..08d255c 100755
--- a/oe-init-build-env-memres
+++ b/oe-init-build-env-memres
@@ -41,12 +41,12 @@ else
     THIS_SCRIPT="$(pwd)/oe-init-build-env"
 fi
 if [ -n "$BBSERVER" ]; then
-  unset BBSERVER
+    unset BBSERVER
 fi
 
-if [ -z "$ZSH_NAME" ] && [ "x$0" = "x$THIS_SCRIPT" ]; then
-   echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
-   exit 1
+if [ -z "$ZSH_NAME" ] && [ "$0" = "$THIS_SCRIPT" ]; then
+    echo "Error: This script needs to be sourced. Please run as '. $THIS_SCRIPT'"
+    exit 1
 fi
 
 if [ -z "$OEROOT" ]; then
@@ -55,30 +55,30 @@ if [ -z "$OEROOT" ]; then
 fi
 export OEROOT
 . $OEROOT/scripts/oe-buildenv-internal && \
-    $OEROOT/scripts/oe-setup-builddir && \
-    [ -n "$BUILDDIR" ] && cd $BUILDDIR
+    TEMPLATECONF="$TEMPLATECONF" $OEROOT/scripts/oe-setup-builddir && \
+    [ -n "$BUILDDIR" ] && cd "$BUILDDIR"
 unset THIS_SCRIPT OEROOT BBPATH
 
 res=1
-if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null ; then
-    BBSERVER=`cat bitbake.lock` bitbake --status-only
+if [ -e bitbake.lock ] && grep : bitbake.lock > /dev/null; then
+    BBSERVER=$(cat bitbake.lock) bitbake --status-only
     res=$?
 fi
 
-if [ $res != 0 ] ; then
+if [ $res != 0 ]; then
     bitbake --server-only -t xmlrpc -B localhost:$port
 fi
 
-if [ $port = -1 ] ; then
+if [ $port = -1 ]; then
     export BBSERVER=localhost:-1
     echo "Bitbake server started on demand as needed, use bitbake -m to shut it down"
 else
-    export BBSERVER=`cat bitbake.lock`
+    export BBSERVER=$(cat bitbake.lock)
 
-    if [ $res = 0 ] ; then
-	echo "Using existing bitbake server at: $BBSERVER, use bitbake -m to shut it down"
+    if [ $res = 0 ]; then
+        echo "Using existing bitbake server at: $BBSERVER, use bitbake -m to shut it down"
     else
-	echo "Bitbake server started at: $BBSERVER, use bitbake -m to shut it down"
+        echo "Bitbake server started at: $BBSERVER, use bitbake -m to shut it down"
     fi
-    unset res
 fi
+unset port res
-- 
2.1.0

[PATCH 4/6] oe-buildenv-internal: Add paths to $PATH individually

[Peter Kjellerstedt]

Instead of assuming that the path to the scripts directory always is
in $PATH directly before the bitbake directory, treat them as separate
paths and add them individually to $PATH.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 scripts/oe-buildenv-internal | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index 457763b..cf33c49 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -95,9 +95,14 @@ if ! (test -d "$BITBAKEDIR"); then
 fi
 
 # Make sure our paths are at the beginning of $PATH
-NEWPATHS="${OEROOT}/scripts:$BITBAKEDIR/bin:"
-PATH=$NEWPATHS$(echo $PATH | sed -e "s|:$NEWPATHS|:|g" -e "s|^$NEWPATHS||")
-unset BITBAKEDIR NEWPATHS
+for newpath in "$BITBAKEDIR/bin" "$OEROOT/scripts"; do
+    # Remove any existences of $newpath from $PATH
+    PATH=$(echo $PATH | sed -re "s#:$newpath(:|$)#\1#g" -e "s#^$newpath:##")
+
+    # Add $newpath to $PATH
+    PATH="$newpath:$PATH"
+done
+unset BITBAKEDIR newpath
 
 # Used by the runqemu script
 export BUILDDIR
-- 
2.1.0

[PATCH 5/6] oe-buildenv-internal: Add variables individually to BB_ENV_EXTRAWHITE

[Peter Kjellerstedt]

Instead of adding all variables to BB_ENV_EXTRAWHITE as one, treat
them separately and add them one by one as needed.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 scripts/oe-buildenv-internal | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index cf33c49..6a63399 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -108,13 +108,29 @@ unset BITBAKEDIR newpath
 export BUILDDIR
 export PATH
 
-BB_ENV_EXTRAWHITE_OE="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
+add_extrawhite() {
+    # If the current shell is zsh, then temporarily set it to emulate sh in this
+    # function so that the for and case statements below work as expected.
+    [ -z "$ZSH_NAME" ] || emulate -L sh
+
+    local extrawhite="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
 HTTPS_PROXY https_proxy FTP_PROXY ftp_proxy FTPS_PROXY ftps_proxy ALL_PROXY \
 all_proxy NO_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY \
 SDKMACHINE BB_NUMBER_THREADS BB_NO_NETWORK PARALLEL_MAKE GIT_PROXY_COMMAND \
 SOCKS5_PASSWD SOCKS5_USER SCREENDIR STAMPS_DIR"
 
-echo "$BB_ENV_EXTRAWHITE" | grep -q "${BB_ENV_EXTRAWHITE_OE}"
-if [ $? != 0 ]; then
-     export BB_ENV_EXTRAWHITE="${BB_ENV_EXTRAWHITE_OE} $BB_ENV_EXTRAWHITE"
-fi
+    local var
+    for var in $extrawhite; do
+        case " $BB_ENV_EXTRAWHITE " in
+            *[[:blank:]]$var[[:blank:]]*)
+                ;;
+            *)
+                BB_ENV_EXTRAWHITE="${BB_ENV_EXTRAWHITE:+$BB_ENV_EXTRAWHITE }$var"
+                ;;
+        esac
+    done
+}
+
+add_extrawhite
+unset -f add_extrawhite
+export BB_ENV_EXTRAWHITE
-- 
2.1.0

[PATCH 6/6] oe-buildenv-internal: Some clean up

[Peter Kjellerstedt]

* Consistent indentation (four spaces)
* Use [ -z ...] and [ -n ... ] where possible
* Unset temporary variables
* Use $(...) instead of `...`
* Avoid an unnecessary call to expr

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
---
 scripts/oe-buildenv-internal | 46 +++++++++++++++++++++++---------------------
 1 file changed, 24 insertions(+), 22 deletions(-)

diff --git a/scripts/oe-buildenv-internal b/scripts/oe-buildenv-internal
index 6a63399..14af872 100755
--- a/scripts/oe-buildenv-internal
+++ b/scripts/oe-buildenv-internal
@@ -24,7 +24,7 @@ if [ -z "$OEROOT" ]; then
     return 1
 fi
 
-if [ -z "$OE_SKIP_SDK_CHECK" -a ! -z "$OECORE_SDK_VERSION" ]; then
+if [ -z "$OE_SKIP_SDK_CHECK" ] && [ -n "$OECORE_SDK_VERSION" ]; then
     echo >&2 "Error: The OE SDK/ADT was detected as already being present in this shell environment. Please use a clean shell when sourcing this environment script."
     return 1
 fi
@@ -33,24 +33,26 @@ fi
 # sanity.bbclass because bitbake's source code doesn't even pass
 # parsing stage when used with python v3, so we catch it here so we
 # can offer a meaningful error message.
-py_v3_check=`/usr/bin/env python --version 2>&1 | grep "Python 3"`
-if [ "$py_v3_check" != "" ]; then
-	echo >&2 "Bitbake is not compatible with python v3"
-	echo >&2 "Please set up python v2 as your default python interpreter"
-	return 1
+py_v3_check=$(/usr/bin/env python --version 2>&1 | grep "Python 3")
+if [ -n "$py_v3_check" ]; then
+    echo >&2 "Bitbake is not compatible with python v3"
+    echo >&2 "Please set up python v2 as your default python interpreter"
+    return 1
 fi
+unset py_v3_check
 
 # Similarly, we now have code that doesn't parse correctly with older
 # versions of Python, and rather than fixing that and being eternally
 # vigilant for any other new feature use, just check the version here.
-py_v26_check=`python -c 'import sys; print sys.version_info >= (2,7,3)'`
+py_v26_check=$(python -c 'import sys; print sys.version_info >= (2,7,3)')
 if [ "$py_v26_check" != "True" ]; then
-	echo >&2 "BitBake requires Python 2.7.3 or later"
-	return 1
+    echo >&2 "BitBake requires Python 2.7.3 or later"
+    return 1
 fi
+unset py_v26_check
 
-if [ "x$BDIR" = "x" ]; then
-    if [ "x$1" = "x" ]; then
+if [ -z "$BDIR" ]; then
+    if [ -z "$1" ]; then
         BDIR="build"
     else
         BDIR="$1"
@@ -62,34 +64,34 @@ if [ "x$BDIR" = "x" ]; then
         # Remove any possible trailing slashes. This is used to work around
         # buggy readlink in Ubuntu 10.04 that doesn't ignore trailing slashes
         # and hence "readlink -f new_dir_to_be_created/" returns empty.
-        BDIR=`echo $BDIR | sed -re 's|/+$||'`
+        BDIR=$(echo $BDIR | sed -re 's|/+$||')
 
-        BDIR=`readlink -f "$BDIR"`
+        BDIR=$(readlink -f "$BDIR")
         if [ -z "$BDIR" ]; then
-            PARENTDIR=`dirname "$1"`
+            PARENTDIR=$(dirname "$1")
             echo >&2 "Error: the directory $PARENTDIR does not exist?"
             return 1
         fi
     fi
-    if [ "x$2" != "x" ]; then
+    if [ -n "$2" ]; then
         BITBAKEDIR="$2"
     fi
 fi
-if expr "$BDIR" : '/.*' > /dev/null ; then
+if [ "${BDIR#/}" != "$BDIR" ]; then
     BUILDDIR="$BDIR"
 else
-    BUILDDIR="`pwd`/$BDIR"
+    BUILDDIR="$(pwd)/$BDIR"
 fi
 unset BDIR
 
-if [ "x$BITBAKEDIR" = "x" ]; then
-    BITBAKEDIR="$OEROOT/bitbake$BBEXTRA/"
+if [ -z "$BITBAKEDIR" ]; then
+    BITBAKEDIR="$OEROOT/bitbake$BBEXTRA"
 fi
 
-BITBAKEDIR=`readlink -f "$BITBAKEDIR"`
-BUILDDIR=`readlink -f "$BUILDDIR"`
+BITBAKEDIR=$(readlink -f "$BITBAKEDIR")
+BUILDDIR=$(readlink -f "$BUILDDIR")
 
-if ! (test -d "$BITBAKEDIR"); then
+if [ ! -d "$BITBAKEDIR" ]; then
     echo >&2 "Error: The bitbake directory ($BITBAKEDIR) does not exist!  Please ensure a copy of bitbake exists at this location"
     return 1
 fi
-- 
2.1.0