[PATCH v2 0/9] Some fixes for the testimage class

[PATCH v2 0/9] Some fixes for the testimage class

[Stefan Stanacar]

Hello,

These are some fixes for the testimage.bbclass and tests and
also two new tests.

Changed in v2:
   - fixed the syslog tests for systemd builds
   - new patch (9) which fixes the dependency check between tests

Cheers,
Stefan


The following changes since commit 74e403705e25afec656b5e40921771a9c29bdc40:

  yocto-kernel: make BBLAYERS parsing more robust (2013-07-13 18:23:39 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib stefans/oeqa9

for you to fetch changes up to 4d06d05ef481537369dd646b114f617a2bc7f8a4:

  lib/oeqa: fix dependecy check (2013-07-15 17:37:57 +0300)

----------------------------------------------------------------
Alexandru Palalau (1):
      lib/oeqa/runtime: add tests for syslog and df

Stefan Stanacar (8):
      lib/oeqa/runtime/dmesg.py: whitelist dmesg message for systemd net interface name
      lib/oeqa/utils/sshcontrol.py: fix passing command to subprocess
      testimage.bbclass, lib/oeqa/utils/qemurunner.py: make boot timeout configurable and increase it
      lib/oeqa/runtime/multilib.py: use readelf instead of file
      lib/oeqa/oetest.py: provide a ps command for all tests
      classes/testimage.bbclass: use a copy of rootfs for tests
      testimage.bbclass, lib/oeqa: add headers and comments
      lib/oeqa: fix dependecy check

 meta/classes/testimage.bbclass    | 43 +++++++++++++++++++++++++++++++---
 meta/lib/oeqa/oetest.py           | 14 ++++++++---
 meta/lib/oeqa/runtime/connman.py  |  5 ----
 meta/lib/oeqa/runtime/df.py       | 13 +++++++++++
 meta/lib/oeqa/runtime/dmesg.py    |  2 +-
 meta/lib/oeqa/runtime/multilib.py |  9 ++++---
 meta/lib/oeqa/runtime/smart.py    |  1 +
 meta/lib/oeqa/runtime/syslog.py   | 37 +++++++++++++++++++++++++++++
 meta/lib/oeqa/utils/decorators.py | 12 +++++++++-
 meta/lib/oeqa/utils/qemurunner.py | 12 +++++-----
 meta/lib/oeqa/utils/sshcontrol.py | 49 +++++++++++++++++++++------------------
 11 files changed, 152 insertions(+), 45 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/df.py
 create mode 100644 meta/lib/oeqa/runtime/syslog.py

Alexandru Palalau (1):
  lib/oeqa/runtime: add tests for syslog and df

Stefan Stanacar (8):
  lib/oeqa/runtime/dmesg.py: whitelist dmesg message for systemd net
    interface name
  lib/oeqa/utils/sshcontrol.py: fix passing command to subprocess
  testimage.bbclass, lib/oeqa/utils/qemurunner.py: make boot timeout
    configurable and increase it
  lib/oeqa/runtime/multilib.py: use readelf instead of file
  lib/oeqa/oetest.py: provide a ps command for all tests
  classes/testimage.bbclass: use a copy of rootfs for tests
  testimage.bbclass, lib/oeqa: add headers and comments
  lib/oeqa: fix dependecy check

 meta/classes/testimage.bbclass    | 43 +++++++++++++++++++++++++++++++---
 meta/lib/oeqa/oetest.py           | 14 ++++++++---
 meta/lib/oeqa/runtime/connman.py  |  5 ----
 meta/lib/oeqa/runtime/df.py       | 13 +++++++++++
 meta/lib/oeqa/runtime/dmesg.py    |  2 +-
 meta/lib/oeqa/runtime/multilib.py |  9 ++++---
 meta/lib/oeqa/runtime/smart.py    |  1 +
 meta/lib/oeqa/runtime/syslog.py   | 37 +++++++++++++++++++++++++++++
 meta/lib/oeqa/utils/decorators.py | 12 +++++++++-
 meta/lib/oeqa/utils/qemurunner.py | 12 +++++-----
 meta/lib/oeqa/utils/sshcontrol.py | 49 +++++++++++++++++++++------------------
 11 files changed, 152 insertions(+), 45 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/df.py
 create mode 100644 meta/lib/oeqa/runtime/syslog.py

-- 
1.8.3.1

[PATCH v2 1/9] lib/oeqa/runtime/dmesg.py: whitelist dmesg message for systemd net interface name

[Stefan Stanacar]

Messages such as:
"systemd-udevd[79]: error changing net interface name eth0 to enp0s3: Device or resource busy"
are harmless as systemd can't rename interfaces in qemu (the interface is already active).
Alternatively, passing net.ifnames=0 to the kernel will stop systemd renaming the interfaces.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/dmesg.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/dmesg.py b/meta/lib/oeqa/runtime/dmesg.py
index b0e2175..48370fb 100644
--- a/meta/lib/oeqa/runtime/dmesg.py
+++ b/meta/lib/oeqa/runtime/dmesg.py
@@ -9,5 +9,5 @@ class DmesgTest(oeRuntimeTest):
 
     @skipUnlessPassed('test_ssh')
     def test_dmesg(self):
-        (status, output) = self.target.run('dmesg | grep -v mmci-pl18x | grep -i error')
+        (status, output) = self.target.run('dmesg | grep -v mmci-pl18x | grep -v "error changing net interface name" | grep -i error')
         self.assertEqual(status, 1, msg = "Error messages in dmesg log: %s" % output)
-- 
1.8.3.1

[PATCH v2 2/9] lib/oeqa/utils/sshcontrol.py: fix passing command to subprocess

[Stefan Stanacar]

Don't use shlex.split in subprocess call and also prepend
. /etc/profile, because PATH over ssh is always
/usr/bin:/bin which isn't what many tests expect.

Changed in v2:
 We now need to use a separate call for scp command.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/utils/sshcontrol.py | 40 +++++++++++++++++----------------------
 1 file changed, 17 insertions(+), 23 deletions(-)

diff --git a/meta/lib/oeqa/utils/sshcontrol.py b/meta/lib/oeqa/utils/sshcontrol.py
index 85a09a0..6c61caa 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -1,7 +1,6 @@
 import subprocess
 import time
 import os
-import shlex
 
 class SSHControl(object):
 
@@ -18,8 +17,12 @@ class SSHControl(object):
                 f.write("%s\n" % msg)
 
     def _internal_run(self, cmd):
+        # We need this for a proper PATH
+        cmd = ". /etc/profile; " + cmd
+        command = ['ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-l', 'root', self.host, cmd ]
+        self.log("[Running]$ %s" % " ".join(command))
         # ssh hangs without os.setsid
-        proc = subprocess.Popen(shlex.split(cmd), shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, preexec_fn=os.setsid)
+        proc = subprocess.Popen(command, shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, preexec_fn=os.setsid)
         return proc
 
     def run(self, cmd, timeout=None):
@@ -30,15 +33,12 @@ class SSHControl(object):
 
 
 
-        actualcmd = "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -l root %s '%s'" % (self.host, cmd)
         if self.host:
-            sshconn = self._internal_run(actualcmd)
+            sshconn = self._internal_run(cmd)
         else:
             raise Exception("Remote IP hasn't been set: '%s'" % actualcmd)
 
         if timeout == 0:
-            self.log("[SSH run without timeout]$ %s" % actualcmd)
-            self.log("  # %s" % cmd)
             self._out = sshconn.communicate()[0]
             self._ret = sshconn.poll()
         else:
@@ -48,8 +48,6 @@ class SSHControl(object):
                 endtime = time.time() + timeout
             while sshconn.poll() is None and time.time() < endtime:
                 time.sleep(1)
-            self.log("[SSH run with timeout]$ %s" % actualcmd)
-            self.log("  # %s" % cmd)
             # process hasn't returned yet
             if sshconn.poll() is None:
                 self._ret = 255
@@ -70,27 +68,23 @@ class SSHControl(object):
         return (self._ret, self._out)
 
     def _internal_scp(self, cmd):
-        self.log("[SCP]$ %s" % cmd)
-        scpconn = self._internal_run(cmd)
-        try:
-            self._out = scpconn.communicate()[0]
-            self._ret = scpconn.poll()
-            if self._ret != 0:
-                self.log("%s" % self._out)
-                raise Exception("Error copying file")
-        except Exception as e:
-            print("Execution failed: %s :" % cmd)
-            print e
-        self.log("%s" % self._out)
-        return (self._ret, self._out)
+        self.log("[Running SCP]$ %s" % " ".join(cmd))
+        scpconn = subprocess.Popen(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, preexec_fn=os.setsid)
+        out = scpconn.communicate()[0]
+        ret = scpconn.poll()
+        self.log("%s" % out)
+        if ret != 0:
+            # we raise an exception so that tests fail in setUp and setUpClass without a need for an assert
+            raise Exception("Error running %s, output: %s" % ( " ".join(cmd), out))
+        return (ret, out)
 
     def copy_to(self, localpath, remotepath):
-        actualcmd = "scp -o UserKnownHostsFile=/dev/null  -o StrictHostKeyChecking=no %s root@%s:%s" % (localpath, self.host, remotepath)
+        actualcmd = ['scp', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', localpath, 'root@%s:%s' % (self.host, remotepath)]
         return self._internal_scp(actualcmd)
 
 
     def copy_from(self, remotepath, localpath):
-        actualcmd = "scp -o UserKnownHostsFile=/dev/null  -o StrictHostKeyChecking=no root@%s:%s %s" % (self.host, remotepath, localpath)
+        actualcmd = ['scp', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', 'root@%s:%s' % (self.host, remotepath), localpath]
         return self._internal_scp(actualcmd)
 
     def get_status(self):
-- 
1.8.3.1

[PATCH v2 3/9] testimage.bbclass, lib/oeqa/utils/qemurunner.py: make boot timeout configurable and increase it

[Stefan Stanacar]

On a loaded host (such as an AB setup where multiple builds run)
the 200 seconds timeout might not be enough to reach the login prompt.
Also make it configurable so we can set it from local.conf/auto.conf

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/classes/testimage.bbclass    |  6 ++++++
 meta/lib/oeqa/utils/qemurunner.py | 10 +++++-----
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 35c6811..f66f514 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -7,6 +7,8 @@ DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman rpm smart gcc xor
 
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
+TEST_QEMUBOOT_TIMEOUT ?= "500"
+
 python do_testimage() {
     testimage_main(d)
 }
@@ -65,6 +67,10 @@ def testimage_main(d):
     qemu.tmpdir = d.getVar("TMPDIR", True)
     qemu.display = d.getVar("BB_ORIGENV", False).getVar("DISPLAY", True)
     qemu.logfile = os.path.join(testdir, "qemu_boot_log.%s" % d.getVar('DATETIME', True))
+    try:
+        qemu.boottime = int(d.getVar("TEST_QEMUBOOT_TIMEOUT", True))
+    except ValueError:
+        qemu.boottime = 500
 
     bb.note("DISPLAY value: %s" % qemu.display)
     bb.note("rootfs file: %s" %  rootfs)
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 3132b68..ec92988 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -10,7 +10,7 @@ from oeqa.utils.oeqemuconsole import oeQemuConsole
 
 class QemuRunner:
 
-    def __init__(self, machine, rootfs, display = None, tmpdir = None, logfile = None):
+    def __init__(self, machine, rootfs, display = None, tmpdir = None, logfile = None, boottime = 400):
         # Popen object
         self.runqemu = None
 
@@ -25,6 +25,7 @@ class QemuRunner:
         self.display = display
         self.tmpdir = tmpdir
         self.logfile = logfile
+        self.boottime = boottime
 
     def launch(self, qemuparams = None):
 
@@ -49,7 +50,6 @@ class QemuRunner:
         self.runqemu = subprocess.Popen(launch_cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,preexec_fn=os.setpgrp)
 
         bb.note("runqemu started, pid is %s" % self.runqemu.pid)
-        # wait at most 30 seconds until qemu pid appears
         bb.note("waiting at most 60 seconds for qemu pid")
         endtime = time.time() + 60
         while not self.is_alive() and time.time() < endtime:
@@ -59,8 +59,8 @@ class QemuRunner:
             bb.note("qemu started - qemu procces pid is %s" % self.qemupid)
 
             console = oeQemuConsole(self.streampath, self.logfile)
-            bb.note("Waiting at most 200 seconds for login banner")
-            (match, text) = console.read_all_timeout("login:", 200)
+            bb.note("Waiting at most %d seconds for login banner" % self.boottime )
+            (match, text) = console.read_all_timeout("login:", self.boottime)
 
             if match:
                 bb.note("Reached login banner")
@@ -80,7 +80,7 @@ class QemuRunner:
                     return False
             else:
                 console.close()
-                bb.note("Target didn't reached login boot in 120 seconds")
+                bb.note("Target didn't reached login boot in %d seconds" % self.boottime)
                 lines = "\n".join(text.splitlines()[-5:])
                 bb.note("Last 5 lines of text:\n%s" % lines)
                 bb.note("Check full boot log: %s" % self.logfile)
-- 
1.8.3.1

[PATCH v2 4/9] lib/oeqa/runtime/multilib.py: use readelf instead of file

[Stefan Stanacar]

file was pulled by rpm in sato images, readelf comes with binutils
so it's in ipk/deb too.
Also the test was looking at connmand instead of connman-applet
(which is what the testcase says and the AB nightly-multilib target installs)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/multilib.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/runtime/multilib.py b/meta/lib/oeqa/runtime/multilib.py
index 397d075..2d952aa 100644
--- a/meta/lib/oeqa/runtime/multilib.py
+++ b/meta/lib/oeqa/runtime/multilib.py
@@ -1,5 +1,6 @@
 import unittest
 from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
 
 def setUpModule():
     multilibs = oeRuntimeTest.tc.d.getVar("MULTILIBS", True) or ""
@@ -7,8 +8,10 @@ def setUpModule():
         skipModule("this isn't a multilib:lib32 image")
 
 
-class MultilibFileTest(oeRuntimeTest):
+class MultilibTest(oeRuntimeTest):
 
+    @skipUnlessPassed('test_ssh')
     def test_file_connman(self):
-        (status, output) = self.target.run('file -L /usr/sbin/connmand | grep "ELF 32-bit LSB executable"')
-        self.assertEqual(status, 0, msg="status and output : %s and %s" % (status,output))
+        self.assertTrue(oeRuntimeTest.hasPackage('connman-gnome'), msg="This test assumes connman-gnome is installed")
+        (status, output) = self.target.run("readelf -h /usr/bin/connman-applet | sed -n '3p' | awk '{print $2}'")
+        self.assertEqual(output, "ELF32", msg="connman-applet isn't an ELF32 binary. readelf says: %s" % self.target.run("readelf -h /usr/bin/connman-applet")[1])
-- 
1.8.3.1

[PATCH v2 5/9] lib/oeqa/oetest.py: provide a ps command for all tests

[Stefan Stanacar]

Many tests will use 'ps' but we need to know if it's busybox
or standard ps.
Drop the existing check from the connman test.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/oetest.py          | 5 ++---
 meta/lib/oeqa/runtime/connman.py | 5 -----
 2 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 24548e9..5777ff8 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -9,7 +9,8 @@ def runTests(tc):
 
     # set the context object passed from the test class
     setattr(oeRuntimeTest, "tc", tc)
-
+    # set ps command to use
+    setattr(oeRuntimeTest, "pscmd", "ps -ef" if oeRuntimeTest.hasPackage("procps") else "ps")
     # prepare test suite, loader and runner
     suite = unittest.TestSuite()
     testloader = unittest.TestLoader()
@@ -30,7 +31,6 @@ class oeRuntimeTest(unittest.TestCase):
     testFailures = []
     testSkipped = []
     testErrors = []
-    pscmd = "ps"
 
     def __init__(self, methodName='runTest'):
         self.target = oeRuntimeTest.tc.target
@@ -74,7 +74,6 @@ class oeRuntimeTest(unittest.TestCase):
 
 
 
-
 def getmodule(pos=2):
     # stack returns a list of tuples containg frame information
     # First element of the list the is current frame, caller is 1
diff --git a/meta/lib/oeqa/runtime/connman.py b/meta/lib/oeqa/runtime/connman.py
index 835e135..5ef96f6 100644
--- a/meta/lib/oeqa/runtime/connman.py
+++ b/meta/lib/oeqa/runtime/connman.py
@@ -17,11 +17,6 @@ class ConnmanTest(oeRuntimeTest):
 
     @skipUnlessPassed('test_connmand_help')
     def test_connmand_running(self):
-        status = self.target.run('ls -l `which ps` | grep busybox')[0]
-        if status == 0:
-            oeRuntimeTest.pscmd = 'ps'
-        else:
-            oeRuntimeTest.pscmd = 'ps -ef'
         (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep [c]onnmand')
         self.assertEqual(status, 0, msg="no connmand process, ps output: %s" % self.target.run(oeRuntimeTest.pscmd)[1])
 
-- 
1.8.3.1

[PATCH v2 6/9] lib/oeqa/runtime: add tests for syslog and df

[Stefan Stanacar]

From: Alexandru Palalau <alexandru.palalau@intel.com>

Add tests for free space and syslog.

Changed in v2:
    - limit df's output to /
    - syslog: fix restart in case of systemd

Signed-off-by: Alexandru Palalau <alexandru.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/classes/testimage.bbclass  |  4 ++--
 meta/lib/oeqa/runtime/df.py     | 13 +++++++++++++
 meta/lib/oeqa/runtime/syslog.py | 37 +++++++++++++++++++++++++++++++++++++
 3 files changed, 52 insertions(+), 2 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/df.py
 create mode 100644 meta/lib/oeqa/runtime/syslog.py

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index f66f514..22f0a92 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -2,8 +2,8 @@ TEST_LOG_DIR ?= "${WORKDIR}/testimage"
 
 DEFAULT_TEST_SUITES = "ping auto"
 DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
-DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh connman rpm smart xorg dmesg"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman rpm smart gcc xorg dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh connman df rpm smart xorg syslog dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman df rpm smart gcc xorg syslog dmesg"
 
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
diff --git a/meta/lib/oeqa/runtime/df.py b/meta/lib/oeqa/runtime/df.py
new file mode 100644
index 0000000..c911989
--- /dev/null
+++ b/meta/lib/oeqa/runtime/df.py
@@ -0,0 +1,13 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    skipModuleUnless(oeRuntimeTest.tc.target.run('which df')[0] == 0, "No df in image or no connection")
+
+class DfTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_df(self):
+        (status,output) = self.target.run("df / | sed -n '2p' | awk '{print $4}'")
+        self.assertTrue(int(output)>5120, msg="Not enough space on image. Current size is %s" % output)
diff --git a/meta/lib/oeqa/runtime/syslog.py b/meta/lib/oeqa/runtime/syslog.py
new file mode 100644
index 0000000..cec4f97
--- /dev/null
+++ b/meta/lib/oeqa/runtime/syslog.py
@@ -0,0 +1,37 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("syslog"):
+        skipModule("No syslog package in image")
+
+class SyslogTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_syslog_help(self):
+        (status,output) = self.target.run('/sbin/syslogd --help')
+        self.assertEqual(status, 1, msg="status and output: %s and %s" % (status,output))
+
+    @skipUnlessPassed("test_syslog_help")
+    def test_syslogd_running(self):
+        (status,output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i [s]yslogd')
+        self.assertEqual(status, 0, msg="no syslogd process, ps output: %s" % self.target.run(oeRuntimeTest.pscmd)[1])
+
+class SyslogTestConfig(oeRuntimeTest):
+
+    def setUp(self):
+        self.target.run('echo "LOGFILE=/var/log/test" >> /etc/syslog-startup.conf')
+
+    @skipUnlessPassed("test_syslogd_running")
+    def test_syslogd_configurable(self):
+        if "systemd" != oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"):
+            (status,output) = self.target.run('/etc/init.d/syslog restart')
+        else:
+            (status,output) = self.target.run('systemctl restart syslog.service')
+        self.assertEqual(status, 0, msg="Could not restart syslog service. Status and output: %s and %s" % (status,output))
+        (status,output) = self.target.run('logger foobar && grep foobar /var/log/test')
+        self.assertEqual(status, 0, msg="Test log string not found. Output: %s " % output)
+
+    def tearDown(self):
+        self.target.run("sed -i 's#LOGFILE=/var/log/test##' /etc/syslog-startup.conf")
-- 
1.8.3.1

[PATCH v2 7/9] classes/testimage.bbclass: use a copy of rootfs for tests

[Stefan Stanacar]

Make a copy of the rootfs and test that.
We can now drop the snapshot option.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/classes/testimage.bbclass    | 9 ++++++++-
 meta/lib/oeqa/utils/qemurunner.py | 2 +-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 22f0a92..940520f 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -22,6 +22,7 @@ def testimage_main(d):
     import os
     import oeqa.runtime
     import re
+    import shutil
     from oeqa.oetest import runTests
     from oeqa.utils.sshcontrol import SSHControl
     from oeqa.utils.qemurunner import QemuRunner
@@ -61,7 +62,13 @@ def testimage_main(d):
     # and boot each supported fs type
     machine=d.getVar("MACHINE", True)
     #will handle fs type eventually, stick with ext3 for now
-    rootfs=d.getVar("DEPLOY_DIR_IMAGE", True) + '/' + d.getVar("IMAGE_BASENAME",True) + '-' + machine + '.ext3'
+    #make a copy of the original rootfs and use that for tests
+    origrootfs=os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True),  d.getVar("IMAGE_LINK_NAME",True) + '.ext3')
+    rootfs=os.path.join(testdir, d.getVar("IMAGE_LINK_NAME", True) + '-testimage.ext3')
+    try:
+        shutil.copyfile(origrootfs, rootfs)
+    except Exception as e:
+        bb.fatal("Error copying rootfs: %s" % e)
 
     qemu = QemuRunner(machine, rootfs)
     qemu.tmpdir = d.getVar("TMPDIR", True)
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index ec92988..1051b2b 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -18,7 +18,7 @@ class QemuRunner:
         self.rootfs = rootfs
 
         self.streampath = '/tmp/qemuconnection.%s' % os.getpid()
-        self.qemuparams = 'bootparams="console=ttyS0" qemuparams="-snapshot -serial unix:%s,server,nowait"' % self.streampath
+        self.qemuparams = 'bootparams="console=ttyS0" qemuparams="-serial unix:%s,server,nowait"' % self.streampath
         self.qemupid = None
         self.ip = None
 
-- 
1.8.3.1

[PATCH v2 8/9] testimage.bbclass, lib/oeqa: add headers and comments

[Stefan Stanacar]

Adds some comments to testimage.bbclass and the files it calls,
just to give an ideea of what it does.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/classes/testimage.bbclass    | 24 ++++++++++++++++++++++++
 meta/lib/oeqa/oetest.py           |  9 +++++++++
 meta/lib/oeqa/utils/decorators.py |  8 ++++++++
 meta/lib/oeqa/utils/sshcontrol.py |  9 +++++++++
 4 files changed, 50 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 940520f..903849d 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -1,3 +1,27 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+
+# testimage.bbclass enables testing of qemu images using python unittests.
+# Most of the tests are commands run on target image over ssh.
+# To use it add testimage to global inherit and call your target image with -c testimage
+# You can try it out like this:
+# - first build a qemu core-image-sato
+# - add INHERIT += "testimage" in local.conf
+# - then bitbake core-image-sato -c testimage. That will run a standard suite of tests.
+
+# You can set (or append to) TEST_SUITES in local.conf to select the tests
+# which you want to run for your target.
+# The test names are the module names in meta/lib/oeqa/runtime.
+# Each name in TEST_SUITES represents a required test for the image. (no skipping allowed)
+# Appending "auto" means that it will try to run all tests that are suitable for the image (each test decides that on it's own).
+# Note that order in TEST_SUITES is important (it's the order tests run) and it influences tests dependencies.
+
+# TEST_LOG_DIR contains a ssh log (what command is running, output and return codes) and a qemu boot log till login
+# Booting is handled by this class, and it's not a test in itself.
+# TEST_QEMUBOOT_TIMEOUT can be used to set the maximum time in seconds the launch code will wait for the login prompt.
+
 TEST_LOG_DIR ?= "${WORKDIR}/testimage"
 
 DEFAULT_TEST_SUITES = "ping auto"
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 5777ff8..7f6baa4 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -1,3 +1,12 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Main unittest module used by testimage.bbclass
+# This provides the oeRuntimeTest base class which is inherited by all tests in meta/lib/oeqa/runtime.
+
+# It also has some helper functions and it's responsible for actually starting the tests
+
 import os, re, mmap
 import unittest
 import inspect
diff --git a/meta/lib/oeqa/utils/decorators.py b/meta/lib/oeqa/utils/decorators.py
index 21e6b22..adec65e 100644
--- a/meta/lib/oeqa/utils/decorators.py
+++ b/meta/lib/oeqa/utils/decorators.py
@@ -1,3 +1,11 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Some custom decorators that can be used by unittests
+# Most useful is skipUnlessPassed which can be used for
+# creating dependecies between two test methods.
+
 from oeqa.oetest import *
 
 class skipIfFailure(object):
diff --git a/meta/lib/oeqa/utils/sshcontrol.py b/meta/lib/oeqa/utils/sshcontrol.py
index 6c61caa..8f98c45 100644
--- a/meta/lib/oeqa/utils/sshcontrol.py
+++ b/meta/lib/oeqa/utils/sshcontrol.py
@@ -1,3 +1,12 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Provides a class for setting up ssh connections,
+# running commands and copying files to/from a target.
+# It's used by testimage.bbclass and tests in lib/oeqa/runtime.
+
+
 import subprocess
 import time
 import os
-- 
1.8.3.1

[PATCH v2 9/9] lib/oeqa: fix dependecy check

[Stefan Stanacar]

Adds missing skip for smart test and fix the check (which I somehow broke
a while ago).

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/smart.py    | 1 +
 meta/lib/oeqa/utils/decorators.py | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime/smart.py
index 8cfacd4..0b03a30 100644
--- a/meta/lib/oeqa/runtime/smart.py
+++ b/meta/lib/oeqa/runtime/smart.py
@@ -10,6 +10,7 @@ def setUpModule():
 
 class SmartHelpTest(oeRuntimeTest):
 
+    @skipUnlessPassed('test_ssh')
     def test_smart_help(self):
         status = self.target.run('smart --help')[0]
         self.assertEqual(status, 0)
diff --git a/meta/lib/oeqa/utils/decorators.py b/meta/lib/oeqa/utils/decorators.py
index adec65e..e0ca6fd 100644
--- a/meta/lib/oeqa/utils/decorators.py
+++ b/meta/lib/oeqa/utils/decorators.py
@@ -41,7 +41,9 @@ class skipUnlessPassed(object):
 
     def __call__(self,f):
         def wrapped_f(*args):
-            if self.testcase in (oeRuntimeTest.testSkipped, oeRuntimeTest.testFailures, oeRuntimeTest.testErrors):
+            if self.testcase in oeRuntimeTest.testSkipped or \
+                    self.testcase in  oeRuntimeTest.testFailures or \
+                    self.testcase in oeRuntimeTest.testErrors:
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
             f(*args)
         wrapped_f.__name__ = f.__name__
-- 
1.8.3.1