[PATCH 0/7] Buildhistory improvements

[PATCH 0/7] Buildhistory improvements

[Paul Eggleton]

This includes a revised version of Martin's patch to use the real
package content size rather than the archive size (it now avoids
the double collection of installed packages, works with multilib and
I also made a few other minor tweaks). I've already run the revised
version past him before sending this out.

Also included are a few other improvements to complete items from my
todo list.


The following changes since commit 57662d4f813d5795cac1529633db80a09efdb089:

  meta-skeleton: Add busybox config fragment example (2013-08-13 23:03:44 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/buildhistory-8
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/buildhistory-8

Martin Jansa (1):
  classes/buildhistory: record size of installed package not compressed
    archive

Paul Eggleton (6):
  classes/buildhistory: remove default of BUILDHISTORY_CHECKVERBACKWARDS
  classes/buildhistory: show command line in commit message
  classes/buildhistory: tag last 3 build revisions
  classes/buildhistory: avoid mangling names in dot graphs for images
  buildhistory-diff: improve command-line handling
  buildhistory-collect-srcrevs: match new buildhistory-diff command line
    parsing

 meta/classes/buildhistory.bbclass     | 42 ++++++++++++++++---------
 meta/classes/package_rpm.bbclass      |  2 ++
 meta/classes/populate_sdk_deb.bbclass |  6 ++--
 meta/classes/populate_sdk_ipk.bbclass |  6 ++--
 meta/classes/populate_sdk_rpm.bbclass |  2 +-
 scripts/buildhistory-collect-srcrevs  | 17 ++++++----
 scripts/buildhistory-diff             | 54 +++++++++++++++++++++++---------
 scripts/oe-pkgdata-util               | 59 ++++++++++++++++++++++++++++++++---
 scripts/opkg-query-helper.py          |  2 +-
 9 files changed, 143 insertions(+), 47 deletions(-)

-- 
1.8.1.2

[PATCH 1/7] classes/buildhistory: remove default of BUILDHISTORY_CHECKVERBACKWARDS

[Paul Eggleton]

Since this variable is no longer used, we don't need to set a default
for it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/buildhistory.bbclass | 1 -
 1 file changed, 1 deletion(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index c32c7aa..1c49831 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -16,7 +16,6 @@ BUILDHISTORY_IMAGE_FILES ?= "/etc/passwd /etc/group"
 BUILDHISTORY_COMMIT ?= "0"
 BUILDHISTORY_COMMIT_AUTHOR ?= "buildhistory <buildhistory@${DISTRO}>"
 BUILDHISTORY_PUSH_REPO ?= ""
-BUILDHISTORY_CHECKVERBACKWARDS ?= "1"
 
 # Must inherit package first before changing PACKAGEFUNCS
 inherit package
-- 
1.8.1.2

[PATCH 2/7] classes/buildhistory: record size of installed package not compressed archive

[Paul Eggleton]

From: Martin Jansa <martin.jansa@gmail.com>

* usually it's more important to know how much space will each
  package take on target device then size of compressed package
* example for libewebkit0 with 4 different architectures, interesting
  that om_gta02 .ipk is bigger but it's smaller when installed

  before:
  MACHINE     DEFAULTTUNE       SIZE (.ipk file)
  om_gta04    cortexa8t-neon    15996 KiB libewebkit0
  qemux86_64  x86-64            16992 KiB libewebkit0
  spitz       xscale            16148 KiB libewebkit0
  om_gta02    arm920t           16260 KiB libewebkit0

  after:
  MACHINE     DEFAULTTUNE       SIZE (installed)
  om_gta04    cortexa8t-neon    60544 KiB libewebkit0
  qemux86_64  x86-64            63720 KiB libewebkit0
  spitz       xscale            60588 KiB libewebkit0
  om_gta02    arm920t           56268 KiB libewebkit0

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/buildhistory.bbclass     | 12 ++++---
 meta/classes/package_rpm.bbclass      |  2 ++
 meta/classes/populate_sdk_deb.bbclass |  6 ++--
 meta/classes/populate_sdk_ipk.bbclass |  6 ++--
 meta/classes/populate_sdk_rpm.bbclass |  2 +-
 scripts/oe-pkgdata-util               | 59 ++++++++++++++++++++++++++++++++---
 scripts/opkg-query-helper.py          |  2 +-
 7 files changed, 72 insertions(+), 17 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 1c49831..b2e5cc5 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -337,12 +337,14 @@ buildhistory_get_installed() {
 
 	# Produce installed package sizes list
 	printf "" > $1/installed-package-sizes.tmp
-	cat $pkgcache | while read pkg pkgfile
+	cat $pkgcache | while read pkg pkgfile pkgarch
 	do
-		if [ -f $pkgfile ] ; then
-			pkgsize=`du -k $pkgfile | head -n1 | awk '{ print $1 }'`
-			echo $pkgsize $pkg >> $1/installed-package-sizes.tmp
-		fi
+		for vendor in ${TARGET_VENDOR} ${MULTILIB_VENDORS} ; do
+			size=`oe-pkgdata-util read-value ${TMPDIR}/pkgdata $vendor-${TARGET_OS} "PKGSIZE" ${pkg}_${pkgarch}`
+			if [ "$size" != "" ] ; then
+				echo "$size $pkg" >> $1/installed-package-sizes.tmp
+			fi
+		done
 	done
 	cat $1/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > $1/installed-package-sizes.txt
 	rm $1/installed-package-sizes.tmp
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 324d83f..53377a4 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -127,6 +127,8 @@ translate_smart_to_oe() {
 		#echo "$pkg -> $new_pkg" >&2
 		if [ "$arg1" = "arch" ]; then
 			echo $new_pkg $new_arch $other
+		elif [ "$arg1" = "file" ]; then
+			echo $new_pkg $other $new_arch
 		else
 			echo $new_pkg $other
 		fi
diff --git a/meta/classes/populate_sdk_deb.bbclass b/meta/classes/populate_sdk_deb.bbclass
index 3e123ac..ec116ab 100644
--- a/meta/classes/populate_sdk_deb.bbclass
+++ b/meta/classes/populate_sdk_deb.bbclass
@@ -75,13 +75,13 @@ list_installed_packages() {
 		# Here we want the PACKAGE_ARCH not the deb architecture
 		${DPKG_QUERY_COMMAND} -W -f='${Package} ${PackageArch}\n'
 	elif [ "$1" = "file" ] ; then
-		${DPKG_QUERY_COMMAND} -W -f='${Package} ${Package}_${Version}_${Architecture}.deb\n' | while read pkg pkgfile
+		${DPKG_QUERY_COMMAND} -W -f='${Package} ${Package}_${Version}_${Architecture}.deb ${PackageArch}\n' | while read pkg pkgfile pkgarch
 		do
 			fullpath=`find ${DEPLOY_DIR_DEB} -name "$pkgfile" || true`
 			if [ "$fullpath" = "" ] ; then
-				echo "$pkg $pkgfile"
+				echo "$pkg $pkgfile $pkgarch"
 			else
-				echo "$pkg $fullpath"
+				echo "$pkg $fullpath $pkgarch"
 			fi
 		done
 	else
diff --git a/meta/classes/populate_sdk_ipk.bbclass b/meta/classes/populate_sdk_ipk.bbclass
index 4e14d9a..04c71af 100644
--- a/meta/classes/populate_sdk_ipk.bbclass
+++ b/meta/classes/populate_sdk_ipk.bbclass
@@ -61,13 +61,13 @@ list_installed_packages() {
 	if [ "$1" = "arch" ] ; then
 		opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -a
 	elif [ "$1" = "file" ] ; then
-		opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -f | while read pkg pkgfile
+		opkg-cl ${OPKG_ARGS} status | opkg-query-helper.py -f | while read pkg pkgfile pkgarch
 		do
 			fullpath=`find ${DEPLOY_DIR_IPK} -name "$pkgfile" || true`
 			if [ "$fullpath" = "" ] ; then
-				echo "$pkg $pkgfile"
+				echo "$pkg $pkgfile $pkgarch"
 			else
-				echo "$pkg $fullpath"
+				echo "$pkg $fullpath $pkgarch"
 			fi
 		done
 	else
diff --git a/meta/classes/populate_sdk_rpm.bbclass b/meta/classes/populate_sdk_rpm.bbclass
index 219cd18..dd5f39a 100644
--- a/meta/classes/populate_sdk_rpm.bbclass
+++ b/meta/classes/populate_sdk_rpm.bbclass
@@ -161,7 +161,7 @@ list_installed_packages() {
 	if [ "$1" = "arch" ]; then
 		${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH}\n]" | translate_smart_to_oe arch
 	elif [ "$1" = "file" ]; then
-		${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH} %{PACKAGEORIGIN}\n]" | translate_smart_to_oe
+		${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH} %{PACKAGEORIGIN}\n]" | translate_smart_to_oe file
 	else
 		${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH}\n]" | translate_smart_to_oe
 	fi
diff --git a/scripts/oe-pkgdata-util b/scripts/oe-pkgdata-util
index 629b2d5..c63f87d 100755
--- a/scripts/oe-pkgdata-util
+++ b/scripts/oe-pkgdata-util
@@ -20,9 +20,12 @@
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 #
 #
-# Currently only has one function - mapping of packages to their dev/dbg/doc/locale etc. 
-# counterparts ("glob" command). Could be extended in future to perform other useful querying
-# functions on the pkgdata though.
+# Currently only has two functions:
+# 1) glob - mapping of packages to their dev/dbg/doc/locale etc. counterparts.
+# 2) read-value - mapping of packagenames to their location in
+#    pkgdata and then returns value of selected variable (e.g. PKGSIZE)
+# Could be extended in future to perform other useful querying functions on the
+# pkgdata though.
 #
 
 import sys
@@ -32,7 +35,8 @@ import fnmatch
 import re
 
 def usage():
-    print("syntax: pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"");
+    print("syntax: oe-pkgdata-util glob [-d] <pkgdatadir> <vendor-os> <pkglist> \"<globs>\"\n \
+                    read-value [-d] <pkgdatadir> <vendor-os> <value-name> \"<package-name>_<package_architecture>\"");
 
 
 
@@ -151,7 +155,52 @@ def glob(args):
 
     print("\n".join(mappedpkgs))
 
+def read_value(args):
+    if len(args) < 4:
+        usage()
+        sys.exit(1)
+
+    pkgdata_dir = args[0]
+    target_suffix = args[1]
+    var = args[2]
+    packages = args[3].split()
 
+    if target_suffix.startswith("-"):
+        target_suffix = target_suffix[1:]
+
+    def readvar(pkgdata_file, var):
+        val = ""
+        with open(pkgdata_file, 'r') as f:
+            for line in f:
+                if line.startswith(var + ":"):
+                    val = line.split(': ')[1].rstrip()
+        return val
+
+    if debug:
+        print "read-value('%s', '%s', '%s' '%s'" % (pkgdata_dir, target_suffix, var, packages)
+    for package in packages:
+        pkg_split = package.split('_')
+        pkg_name = pkg_split[0]
+        pkg_arch = '_'.join(pkg_split[1:])
+        if debug:
+            print "package: name: '%s', arch: '%s'" % (pkg_name, pkg_arch)
+        multimach_target_sys = "%s-%s" % (pkg_arch, target_suffix)
+        revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+        if debug:
+            print(revlink)
+        if not os.path.exists(revlink):
+            # [YOCTO #4227] try to drop -gnueabi from TARGET_OS
+            multimach_target_sys = '-'.join(multimach_target_sys.split('-')[:-1])
+            revlink = os.path.join(pkgdata_dir, multimach_target_sys, "runtime-reverse", pkg_name)
+            if debug:
+                print(revlink)
+        if os.path.exists(revlink):
+            mappedpkg = os.path.basename(os.readlink(revlink))
+            qvar = var
+            if qvar == "PKGSIZE":
+                # append packagename
+                qvar = "%s_%s" % (var, mappedpkg)
+            print(readvar(revlink, qvar))
 
 # Too lazy to use getopt
 debug = False
@@ -173,6 +222,8 @@ if len(args) < 1:
 
 if args[0] == "glob":
     glob(args[1:])
+elif args[0] == "read-value":
+    read_value(args[1:])
 else:
     usage()
     sys.exit(1)
diff --git a/scripts/opkg-query-helper.py b/scripts/opkg-query-helper.py
index b52284b..fa6c44f 100755
--- a/scripts/opkg-query-helper.py
+++ b/scripts/opkg-query-helper.py
@@ -59,7 +59,7 @@ for line in fileinput.input(args):
                     ver = line.split(": ")[1]
                 elif line.startswith("Architecture:"):
                     arch = line.split(": ")[1]
-                    print("%s %s_%s_%s.ipk" % (pkg,pkg,ver,arch))
+                    print("%s %s_%s_%s.ipk %s" % (pkg,pkg,ver,arch,arch))
             else:
                 if line.startswith("Depends:"):
                     depval = line.split(": ")[1]
-- 
1.8.1.2

[PATCH 3/7] classes/buildhistory: show command line in commit message

[Paul Eggleton]

Record the bitbake command line in the commit message as a further
context indication.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/buildhistory.bbclass | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index b2e5cc5..6fb1ba6 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -495,6 +495,14 @@ def buildhistory_get_sdkvars(d):
     return outputvars(sdkvars, listvars, d)
 
 
+def buildhistory_get_cmdline(d):
+    if sys.argv[0].endswith('bin/bitbake'):
+        bincmd = 'bitbake'
+    else:
+        bincmd = sys.argv[0]
+    return '%s %s' % (bincmd, ' '.join(sys.argv[1:]))
+
+
 buildhistory_commit() {
 	if [ ! -d ${BUILDHISTORY_DIR} ] ; then
 		# Code above that creates this dir never executed, so there can't be anything to commit
@@ -514,18 +522,19 @@ END
 		# Check if there are new/changed files to commit (other than metadata-revs)
 		repostatus=`git status --porcelain | grep -v " metadata-revs$"`
 		HOSTNAME=`hostname 2>/dev/null || echo unknown`
+		CMDLINE="${@buildhistory_get_cmdline(d)}"
 		if [ "$repostatus" != "" ] ; then
 			git add -A .
 			# porcelain output looks like "?? packages/foo/bar"
 			# Ensure we commit metadata-revs with the first commit
 			for entry in `echo "$repostatus" | awk '{print $2}' | awk -F/ '{print $1}' | sort | uniq` ; do
-				git commit $entry metadata-revs -m "$entry: Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
+				git commit $entry metadata-revs -m "$entry: Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" -m "cmd: $CMDLINE" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
 			done
 			if [ "${BUILDHISTORY_PUSH_REPO}" != "" ] ; then
 				git push -q ${BUILDHISTORY_PUSH_REPO}
 			fi
 		else
-			git commit ${BUILDHISTORY_DIR}/ --allow-empty -m "No changes: Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
+			git commit ${BUILDHISTORY_DIR}/ --allow-empty -m "No changes: Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" -m "cmd: $CMDLINE" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
 		fi) || true
 }
 
-- 
1.8.1.2

[PATCH 4/7] classes/buildhistory: tag last 3 build revisions

[Paul Eggleton]

This makes it easier to obtain the difference from the last three
builds particularly where a single build might account for more than one
commit in the buildhistory git repository (e.g. if package and image
changes occurred).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/buildhistory.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 6fb1ba6..e3f9201 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -518,6 +518,10 @@ END
 		# Initialise the repo if necessary
 		if [ ! -d .git ] ; then
 			git init -q
+		else
+			git tag -f build-minus-3 build-minus-2 > /dev/null 2>&1 || true
+			git tag -f build-minus-2 build-minus-1 > /dev/null 2>&1 || true
+			git tag -f build-minus-1 > /dev/null 2>&1 || true
 		fi
 		# Check if there are new/changed files to commit (other than metadata-revs)
 		repostatus=`git status --porcelain | grep -v " metadata-revs$"`
-- 
1.8.1.2

[PATCH 5/7] classes/buildhistory: avoid mangling names in dot graphs for images

[Paul Eggleton]

We were mangling names here to avoid characters that are invalid for dot
node names, but if you just quote all names that isn't necessary and we
retain the original naming, allowing easy searching of the graph files.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/buildhistory.bbclass | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index e3f9201..080f2c7 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -325,10 +325,10 @@ buildhistory_get_installed() {
 	fi
 
 	# Produce dependency graph
-	# First, filter out characters that cause issues for dot
-	rootfs_list_installed_depends | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' > $1/depends.tmp
+	# First, quote each name to handle characters that cause issues for dot
+	rootfs_list_installed_depends | sed 's:\([^| ]*\):"\1":g' > $1/depends.tmp
 	# Change delimiter from pipe to -> and set style for recommend lines
-	sed -i -e 's:|: -> :' -e 's:\[REC\]:[style=dotted]:' -e 's:$:;:' $1/depends.tmp
+	sed -i -e 's:|: -> :' -e 's:"\[REC\]":[style=dotted]:' -e 's:$:;:' $1/depends.tmp
 	# Add header, sorted and de-duped contents and footer and then delete the temp file
 	printf "digraph depends {\n    node [shape=plaintext]\n" > $1/depends.dot
 	cat $1/depends.tmp | sort | uniq >> $1/depends.dot
@@ -354,10 +354,10 @@ buildhistory_get_installed() {
 
 	if [ "$2" != "sdk" ] ; then
 		# Produce some cut-down graphs (for readability)
-		grep -v kernel_image $1/depends.dot | grep -v kernel_2 | grep -v kernel_3 > $1/depends-nokernel.dot
+		grep -v kernel_image $1/depends.dot | grep -v kernel-2 | grep -v kernel-3 > $1/depends-nokernel.dot
 		grep -v libc6 $1/depends-nokernel.dot | grep -v libgcc > $1/depends-nokernel-nolibc.dot
-		grep -v update_ $1/depends-nokernel-nolibc.dot > $1/depends-nokernel-nolibc-noupdate.dot
-		grep -v kernel_module $1/depends-nokernel-nolibc-noupdate.dot > $1/depends-nokernel-nolibc-noupdate-nomodules.dot
+		grep -v update- $1/depends-nokernel-nolibc.dot > $1/depends-nokernel-nolibc-noupdate.dot
+		grep -v kernel-module $1/depends-nokernel-nolibc-noupdate.dot > $1/depends-nokernel-nolibc-noupdate-nomodules.dot
 	fi
 
 	# add complementary package information
-- 
1.8.1.2

[PATCH 6/7] buildhistory-diff: improve command-line handling

[Paul Eggleton]

Improve command-line argument handling of buildhistory-diff to make it
easier to use.

* Default buildhistory directory to buildhistory/ under the current
  directory and require an option to set it (since most users will
  likely run buildhistory-diff from the build directory and keep
  BUILDHISTORY_DIR at its default location)
* Default from-revision to "build-minus-1" to get the difference from
  the previous build with no arguments
* Allow from/to revisions to be specified by from..to (since git accepts
  this form).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 scripts/buildhistory-diff | 54 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 39 insertions(+), 15 deletions(-)

diff --git a/scripts/buildhistory-diff b/scripts/buildhistory-diff
index 30c2c08..b82240d 100755
--- a/scripts/buildhistory-diff
+++ b/scripts/buildhistory-diff
@@ -7,6 +7,7 @@
 
 import sys
 import os
+import optparse
 from distutils.version import LooseVersion
 
 # Ensure PythonGit is installed (buildhistory_analysis needs it)
@@ -17,14 +18,30 @@ except ImportError:
     sys.exit(1)
 
 def main():
+    parser = optparse.OptionParser(
+        description = "Reports significant differences in the buildhistory repository.",
+        usage = """
+    %prog [options] [from-revision [to-revision]]
+(if not specified, from-revision defaults to build-minus-1, and to-revision defaults to HEAD)""")
+
+    parser.add_option("-p", "--buildhistory-dir",
+            help = "Specify path to buildhistory directory (defaults to buildhistory/ under cwd)",
+            action="store", dest="buildhistory_dir", default='buildhistory/')
+
+    options, args = parser.parse_args(sys.argv)
+
+    if len(args) > 3:
+        sys.stderr.write('Invalid argument(s) specified: %s\n\n' % ' '.join(args[3:]))
+        parser.print_help()
+        sys.exit(1)
+
     if LooseVersion(git.__version__) < '0.3.1':
         sys.stderr.write("Version of GitPython is too old, please install GitPython (python-git) 0.3.1 or later in order to use this script\n")
         sys.exit(1)
 
-    if len(sys.argv) < 3 or '--help' in sys.argv:
-        print("Report significant differences in the buildhistory repository")
-        print("Syntax: %s <buildhistory-path> <since-revision> [to-revision]" % os.path.basename(sys.argv[0]))
-        print("If to-revision is not specified, it defaults to HEAD")
+    if not os.path.exists(options.buildhistory_dir):
+        sys.stderr.write('Buildhistory directory "%s" does not exist\n\n' % options.buildhistory_dir)
+        parser.print_help()
         sys.exit(1)
 
     # Set path to OE lib dir so we can import the buildhistory_analysis module
@@ -47,21 +64,28 @@ def main():
     sys.path[0:0] = [newpath, bitbakepath + '/lib']
     import oe.buildhistory_analysis
 
-    buildhistory_dir = sys.argv[1]
-    if not os.path.exists(buildhistory_dir):
-        sys.stderr.write('Specified buildhistory directory "%s" does not exist\n' % buildhistory_dir)
-        sys.exit(1)
-
-    if len(sys.argv) > 3:
-        torev = sys.argv[3]
-    else:
-        torev = 'HEAD'
+    fromrev = 'build-minus-1'
+    torev = 'HEAD'
+    if len(args) > 1:
+        if len(args) == 2 and '..' in args[1]:
+            revs = args[1].split('..')
+            fromrev = revs[0]
+            if revs[1]:
+                torev = revs[1]
+        else:
+            fromrev = args[1]
+    if len(args) > 2:
+        torev = args[2]
 
     import gitdb
     try:
-        changes = oe.buildhistory_analysis.process_changes(buildhistory_dir, sys.argv[2], torev)
+        changes = oe.buildhistory_analysis.process_changes(options.buildhistory_dir, fromrev, torev)
     except gitdb.exc.BadObject as e:
-        sys.stderr.write('Specified git revision "%s" is not valid\n' % e.args[0])
+        if len(args) == 1:
+            sys.stderr.write("Unable to find previous build revision in buildhistory repository\n\n")
+            parser.print_help()
+        else:
+            sys.stderr.write('Specified git revision "%s" is not valid\n' % e.args[0])
         sys.exit(1)
 
     for chg in changes:
-- 
1.8.1.2

[PATCH 7/7] buildhistory-collect-srcrevs: match new buildhistory-diff command line parsing

[Paul Eggleton]

* Default buildhistory directory to buildhistory/ under the current
  directory and require an option to set it
* Show a description in the help output

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 scripts/buildhistory-collect-srcrevs | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/scripts/buildhistory-collect-srcrevs b/scripts/buildhistory-collect-srcrevs
index 7f65c90..58a2708 100755
--- a/scripts/buildhistory-collect-srcrevs
+++ b/scripts/buildhistory-collect-srcrevs
@@ -34,8 +34,9 @@ logger = logger_create()
 
 def main():
     parser = optparse.OptionParser(
+        description = "Collects the recorded SRCREV values from buildhistory and reports on them.",
         usage = """
-    %prog [options] <buildhistory-dir>""")
+    %prog [options]""")
 
     parser.add_option("-a", "--report-all",
             help = "Report all SRCREV values, not just ones where AUTOREV has been used",
@@ -43,16 +44,20 @@ def main():
     parser.add_option("-f", "--forcevariable",
             help = "Use forcevariable override for all output lines",
             action="store_true", dest="forcevariable")
+    parser.add_option("-p", "--buildhistory-dir",
+            help = "Specify path to buildhistory directory (defaults to buildhistory/ under cwd)",
+            action="store", dest="buildhistory_dir", default='buildhistory/')
 
     options, args = parser.parse_args(sys.argv)
 
-    if len(args) != 2:
+    if len(args) > 1:
+        sys.stderr.write('Invalid argument(s) specified: %s\n\n' % ' '.join(args[1:]))
         parser.print_help()
         sys.exit(1)
 
-    buildhistory_dir = args[1]
-    if not os.path.exists(buildhistory_dir):
-        logger.error('specified buildhistory path %s could not be found' % buildhistory_dir)
+    if not os.path.exists(options.buildhistory_dir):
+        sys.stderr.write('Buildhistory directory "%s" does not exist\n\n' % options.buildhistory_dir)
+        parser.print_help()
         sys.exit(1)
 
     if options.forcevariable:
@@ -61,7 +66,7 @@ def main():
         forcevariable = ''
 
     lastdir = ''
-    for root, dirs, files in os.walk(buildhistory_dir):
+    for root, dirs, files in os.walk(options.buildhistory_dir):
         if '.git' in dirs:
             dirs.remove('.git')
         for fn in files:
-- 
1.8.1.2