[RFC PATCH v2 0/1] Add mechanism to run QA checks on the image once it's built

[RFC PATCH v2 0/1] Add mechanism to run QA checks on the image once it's built

[Joshua Lock]

Here's v2 of a patch to add a mechanism to call a series of functions to
validate a constructed image.

Changes since v1:
* support shell functions
* don't abuse NotImplementedError

Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit b17f91ed06a604e3d356fe17756bfe2ca61594b7:

  tune-ppce500mc.inc: pass -mcpu=e500mc for ppce500mc kernel compile (2016-07-10 14:12:07 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib joshuagl/imageqa
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=joshuagl/imageqa

Joshua Lock (1):
  image: add do_image_qa task to run QA checks on the constructed image

 meta/classes/image.bbclass | 30 ++++++++++++++++++++++++++++++
 meta/lib/oe/utils.py       | 13 +++++++++++++
 2 files changed, 43 insertions(+)

--
2.7.4

[RFC PATCH v2 1/1] image: add do_image_qa task to run QA checks on the constructed image

[Joshua Lock]

This task runs all functions in IMAGE_QA_COMMANDS after the image
construction has completed in order to validate the resulting image.

Image sanity checks should either be Python functions which raise
bb.build.FuncFailed on failure or shell functions with return a
non-zero exit code.

Python functions may instead raise an oe.utils.ImageQAFailed
Exception which takes an extra argument, a description of the
failure.

   python image_check_python_ok () {
       if True:
           raise bb.build.FuncFailed('This check always fails')
       else:
           bb.note("Nothing to see here")
   }

   image_check_shell_ok () {
       if true
           exit 1
       else
           exit 0
       fi
   }

[YOCTO #9448]

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
 meta/classes/image.bbclass | 30 ++++++++++++++++++++++++++++++
 meta/lib/oe/utils.py       | 13 +++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 30dfd64..8bda94a 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -275,6 +275,36 @@ do_image_complete[dirs] = "${TOPDIR}"
 do_image_complete[umask] = "022"
 addtask do_image_complete after do_image before do_build
 
+# Add image-level QA/sanity checks to IMAGE_QA_COMMANDS
+#
+# IMAGE_QA_COMMANDS += " \
+#     image_check_everything_ok \
+# "
+# This task runs all functions in IMAGE_QA_COMMANDS after the image
+# construction has completed in order to validate the resulting image.
+fakeroot python do_image_qa () {
+    from oe.utils import ImageQAFailed
+
+    qa_cmds = (d.getVar('IMAGE_QA_COMMANDS', True) or '').split()
+    qamsg = ""
+
+    for cmd in qa_cmds:
+        try:
+            bb.build.exec_func(cmd, d)
+        except oe.utils.ImageQAFailed as e:
+            qamsg = qamsg + '\tImage QA function %s failed: %s\n' % (e.name, e.description)
+        except bb.build.FuncFailed as e:
+            qamsg = qamsg + '\Image QA function %s failed' % e.name
+            if e.logfile:
+                qamsg = qamsg + ' (log file is located at %s)' % e.logfile
+            qamsg = qamsg + '\n'
+
+    if qamsg:
+        imgname = d.getVar('IMAGE_NAME', True)
+        bb.fatal("QA errors found whilst validating image: %s\n%s" % (imgname, qamsg))
+}
+addtask do_image_qa after do_image_complete before do_build
+
 #
 # Write environment variables used by wic
 # to tmp/sysroots/<machine>/imgdata/<image>.env
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index cecddc6..19db540 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -304,3 +304,16 @@ def write_ld_so_conf(d):
     with open(ldsoconf, "w") as f:
         f.write(d.getVar("base_libdir", True) + '\n')
         f.write(d.getVar("libdir", True) + '\n')
+
+class ImageQAFailed(bb.build.FuncFailed):
+    def __init__(self, description, name=None, logfile=None):
+        self.description = description
+        self.name = name
+        self.logfile=logfile
+
+    def __str__(self):
+        msg = 'Function failed: %s' % self.name
+        if self.description:
+            msg = msg + ' (%s)' % self.description
+
+        return msg
-- 
2.7.4

Re: [RFC PATCH v2 1/1] image: add do_image_qa task to run QA checks on the constructed image

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

On 14 July 2016 at 14:36, Joshua Lock <joshua.g.lock@intel.com> wrote:

> +            qamsg = qamsg + '\Image QA function %s failed' % e.name
>

I fixed this typo when merging to mut.

Ross

[-- Attachment #2: Type: text/html, Size: 683 bytes --]