[PATCH v2 00/19] Some new runtime tests or fixes

[PATCH v2 00/19] Some new runtime tests or fixes

[Stefan Stanacar]

Hello,

This set includes new runtime tests and required changes for them.

Not all of them will get run by default, for example all the project
build tests on target (iptables, cvs and sudoku) need to be enabled manually.
Same applies to tests which require special options (skeleton, logrotate, pam).
A new patch for AutoBuilder is needed to add a new target for these.
The new smart tests are the most important change here, (and those will get run by
default on all targets on AB).


Cheers,
Stefan


Changed in v2:
  - squashed two commits and reorderd the series


The following changes since commit 501e1a321d15abd712e3a6ee6ccdb8e21c3d4ea4:

  bitbake: prserv/serv: Fix pid file removal (2013-08-23 17:43:40 +0100)

are available in the git repository at:

  git://mirror.rb.intel.com/git.yoctoproject.org/poky-contrib stefans/stage-oeqa2

for you to fetch changes up to 79006debe0d4fde2e9b14995c68d9e41b812855b:

  classess/testimage: change default test suites (2013-08-23 20:59:10 +0300)

----------------------------------------------------------------
Alexandru Georgescu (1):
      lib/oeqa/runtime: add basic test for x32 images

Alexandru Palalau (4):
      lib/oeqa/runtime: add new logrotate test
      lib/oeqa/runtime: add new skeletoninit test
      lib/oeqa/runtime: add new PAM support test
      lib/oeqa/runtime: add new scp test

Cornel Stoicescu (2):
      lib/oeqa/runtime: add test for perl
      lib/oeqa/runtime: add a test for ldd

Mihai Lindner (4):
      lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target
      lib/oeqa/utils: qemurunner: save host IP address
      lib/oeqa/utils: decorators: return the decorated method
      lib/oeqa/runtime: smart: add new smart tests

Mihai Prica (4):
      lib/oeqa/runtime: add adjust date and time test
      lib/oeqa/runtime: add vncserver for target test
      lib/oeqa/utils: targetbuild: Add helper class for building packages on target
      lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target

Stefan Stanacar (4):
      lib/oeqa/runtime: rpm: add install and erase tests
      lib/oeqa: make it possible to restart the target
      lib/oeqa: change behaviour for unskippable tests
      classess/testimage: change default test suites

 meta/classes/testimage.bbclass         |   4 +-
 meta/lib/oeqa/oetest.py                |  11 +++-
 meta/lib/oeqa/runtime/buildcvs.py      |  32 +++++++++++
 meta/lib/oeqa/runtime/buildiptables.py |  32 +++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   |  29 ++++++++++
 meta/lib/oeqa/runtime/date.py          |  27 +++++++++
 meta/lib/oeqa/runtime/files/test.pl    |   2 +
 meta/lib/oeqa/runtime/ldd.py           |  19 ++++++
 meta/lib/oeqa/runtime/logrotate.py     |  27 +++++++++
 meta/lib/oeqa/runtime/pam.py           |  24 ++++++++
 meta/lib/oeqa/runtime/perl.py          |  28 +++++++++
 meta/lib/oeqa/runtime/rpm.py           |  30 +++++++++-
 meta/lib/oeqa/runtime/scp.py           |  22 +++++++
 meta/lib/oeqa/runtime/smart.py         | 102 +++++++++++++++++++++++++++++----
 meta/lib/oeqa/runtime/vnc.py           |  22 +++++++
 meta/lib/oeqa/runtime/x32lib.py        |  17 ++++++
 meta/lib/oeqa/skeletoninit.py          |  28 +++++++++
 meta/lib/oeqa/utils/decorators.py      |   6 +-
 meta/lib/oeqa/utils/httpserver.py      |  32 +++++++++++
 meta/lib/oeqa/utils/qemurunner.py      |  39 ++++++++-----
 meta/lib/oeqa/utils/targetbuild.py     |  63 ++++++++++++++++++++
 21 files changed, 561 insertions(+), 35 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
 create mode 100644 meta/lib/oeqa/runtime/date.py
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/ldd.py
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py
 create mode 100644 meta/lib/oeqa/runtime/pam.py
 create mode 100644 meta/lib/oeqa/runtime/perl.py
 create mode 100644 meta/lib/oeqa/runtime/scp.py
 create mode 100644 meta/lib/oeqa/runtime/vnc.py
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py
 create mode 100644 meta/lib/oeqa/skeletoninit.py
 create mode 100644 meta/lib/oeqa/utils/httpserver.py
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py

Alexandru Georgescu (1):
  lib/oeqa/runtime: add basic test for x32 images

Alexandru Palalau (4):
  lib/oeqa/runtime: add new logrotate test
  lib/oeqa/runtime: add new skeletoninit test
  lib/oeqa/runtime: add new PAM support test
  lib/oeqa/runtime: add new scp test

Cornel Stoicescu (2):
  lib/oeqa/runtime: add test for perl
  lib/oeqa/runtime: add a test for ldd

Mihai Lindner (4):
  lib/oeqa/utils: new file: httpserver.py useful for serving files over
    HTTP to the target
  lib/oeqa/utils: qemurunner: save host IP address
  lib/oeqa/utils: decorators: return the decorated method
  lib/oeqa/runtime: smart: add new smart tests

Mihai Prica (4):
  lib/oeqa/runtime: add adjust date and time test
  lib/oeqa/runtime: add vncserver for target test
  lib/oeqa/utils: targetbuild: Add helper class for building packages on
    target
  lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on
    target

Stefan Stanacar (4):
  lib/oeqa/runtime: rpm: add install and erase tests
  lib/oeqa: make it possible to restart the target
  lib/oeqa: change behaviour for unskippable tests
  classess/testimage: change default test suites

 meta/classes/testimage.bbclass         |   4 +-
 meta/lib/oeqa/oetest.py                |  11 +++-
 meta/lib/oeqa/runtime/buildcvs.py      |  32 +++++++++++
 meta/lib/oeqa/runtime/buildiptables.py |  32 +++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   |  29 ++++++++++
 meta/lib/oeqa/runtime/date.py          |  27 +++++++++
 meta/lib/oeqa/runtime/files/test.pl    |   2 +
 meta/lib/oeqa/runtime/ldd.py           |  19 ++++++
 meta/lib/oeqa/runtime/logrotate.py     |  27 +++++++++
 meta/lib/oeqa/runtime/pam.py           |  24 ++++++++
 meta/lib/oeqa/runtime/perl.py          |  28 +++++++++
 meta/lib/oeqa/runtime/rpm.py           |  30 +++++++++-
 meta/lib/oeqa/runtime/scp.py           |  22 +++++++
 meta/lib/oeqa/runtime/smart.py         | 102 +++++++++++++++++++++++++++++----
 meta/lib/oeqa/runtime/vnc.py           |  22 +++++++
 meta/lib/oeqa/runtime/x32lib.py        |  17 ++++++
 meta/lib/oeqa/skeletoninit.py          |  28 +++++++++
 meta/lib/oeqa/utils/decorators.py      |   6 +-
 meta/lib/oeqa/utils/httpserver.py      |  32 +++++++++++
 meta/lib/oeqa/utils/qemurunner.py      |  39 ++++++++-----
 meta/lib/oeqa/utils/targetbuild.py     |  63 ++++++++++++++++++++
 21 files changed, 561 insertions(+), 35 deletions(-)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py
 create mode 100644 meta/lib/oeqa/runtime/date.py
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/ldd.py
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py
 create mode 100644 meta/lib/oeqa/runtime/pam.py
 create mode 100644 meta/lib/oeqa/runtime/perl.py
 create mode 100644 meta/lib/oeqa/runtime/scp.py
 create mode 100644 meta/lib/oeqa/runtime/vnc.py
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py
 create mode 100644 meta/lib/oeqa/skeletoninit.py
 create mode 100644 meta/lib/oeqa/utils/httpserver.py
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py

-- 
1.8.3.1

[PATCH v2 01/19] lib/oeqa/runtime: add basic test for x32 images

[Stefan Stanacar]

From: Alexandru Georgescu <alexandru.c.georgescu@intel.com>

Checks that an x86-64-x32 image has the right binaries.

Signed-off-by: Alexandru Georgescu <alexandru.c.georgescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/x32lib.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/x32lib.py

diff --git a/meta/lib/oeqa/runtime/x32lib.py b/meta/lib/oeqa/runtime/x32lib.py
new file mode 100644
index 0000000..6bad201
--- /dev/null
+++ b/meta/lib/oeqa/runtime/x32lib.py
@@ -0,0 +1,17 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+        #check if DEFAULTTUNE is set and it's value is: x86-64-x32
+        defaulttune = oeRuntimeTest.tc.d.getVar("DEFAULTTUNE", True)
+        if "x86-64-x32" not in defaulttune:
+            skipModule("DEFAULTTUNE is not set to x86-64-x32")
+
+class X32libTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_x32_file(self):
+        status1 = self.target.run("readelf -h /bin/ls | grep Class | grep ELF32")[0]
+        status2 = self.target.run("readelf -h /bin/ls | grep Machine | grep X86-64")[0]
+        self.assertTrue(status1 == 0 and status2 == 0, msg="/bin/ls isn't an X86-64 ELF32 binary. readelf says: %s" % self.target.run("readelf -h /bin/ls")[1])
-- 
1.8.3.1

[PATCH v2 02/19] lib/oeqa/runtime: add test for perl

[Stefan Stanacar]

From: Cornel Stoicescu <corneliux.stoicescu@intel.com>

This test runs a perl script on the target and checks the output.

Signed-off-by: Cornel Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/files/test.pl |  2 ++
 meta/lib/oeqa/runtime/perl.py       | 28 ++++++++++++++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/files/test.pl
 create mode 100644 meta/lib/oeqa/runtime/perl.py

diff --git a/meta/lib/oeqa/runtime/files/test.pl b/meta/lib/oeqa/runtime/files/test.pl
new file mode 100644
index 0000000..689c8f1
--- /dev/null
+++ b/meta/lib/oeqa/runtime/files/test.pl
@@ -0,0 +1,2 @@
+$a = 9.01e+21 - 9.01e+21 + 0.01;
+print ("the value of a is ", $a, "\n");
diff --git a/meta/lib/oeqa/runtime/perl.py b/meta/lib/oeqa/runtime/perl.py
new file mode 100644
index 0000000..c9bb684
--- /dev/null
+++ b/meta/lib/oeqa/runtime/perl.py
@@ -0,0 +1,28 @@
+import unittest
+import os
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("perl"):
+        skipModule("No perl package in the image")
+
+
+class PerlTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        oeRuntimeTest.tc.target.copy_to(os.path.join(oeRuntimeTest.tc.filesdir, "test.pl"), "/tmp/test.pl")
+
+    def test_perl_exists(self):
+        (status, output) = self.target.run('which perl')
+        self.assertEqual(status, 0, msg="Perl binary not in PATH or not on target.")
+
+    def test_perl_works(self):
+        (status, output) = self.target.run('perl /tmp/test.pl')
+        self.assertEqual(status, 0, msg="Exit status was not 0. Output: %s" % output)
+        self.assertEqual(output, "the value of a is 0.01", msg="Incorrect output: %s" % output)
+
+    @classmethod
+    def tearDownClass(self):
+        oeRuntimeTest.tc.target.run("rm /tmp/test.pl")
-- 
1.8.3.1

[PATCH v2 03/19] lib/oeqa/runtime: add a test for ldd

[Stefan Stanacar]

From: Cornel Stoicescu <corneliux.stoicescu@intel.com>

This test checks that at least one path in RTLDLIST exists.

Signed-off-by: Cornel Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/ldd.py | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/ldd.py

diff --git a/meta/lib/oeqa/runtime/ldd.py b/meta/lib/oeqa/runtime/ldd.py
new file mode 100644
index 0000000..577c5e2
--- /dev/null
+++ b/meta/lib/oeqa/runtime/ldd.py
@@ -0,0 +1,19 @@
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class LddTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_ldd_exists(self):
+        (status, output) = self.target.run('which ldd')
+        self.assertEqual(status, 0, msg = "ldd does not exist in PATH !")
+
+    @skipUnlessPassed('test_ldd_exists')
+    def test_ldd(self):
+        (status, output) = self.target.run('for i in $(which ldd | xargs cat | grep "^RTLDLIST"|cut -d\'=\' -f2|tr -d \'"\'); do test -f $i && echo $i && break; done')
+        self.assertEqual(status, 0, msg = "ldd path not correct or RTLDLIST files don't exist. ")
-- 
1.8.3.1

[PATCH v2 04/19] lib/oeqa/runtime: add new logrotate test

[Stefan Stanacar]

From: Alexandru Palalau <alexandrux.palalau@intel.com>

New logrotate test which verifies the log directory change in logrotate.conf.
Needs an image with logrotate installed.

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/logrotate.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/logrotate.py

diff --git a/meta/lib/oeqa/runtime/logrotate.py b/meta/lib/oeqa/runtime/logrotate.py
new file mode 100644
index 0000000..80489a3
--- /dev/null
+++ b/meta/lib/oeqa/runtime/logrotate.py
@@ -0,0 +1,27 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=289 testcase
+# Note that the image under test must have logrotate installed
+
+import unittest
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("logrotate"):
+        skipModule("No logrotate package in image")
+
+
+class LogrotateTest(oeRuntimeTest):
+
+    @skipUnlessPassed("test_ssh")
+    def test_1_logrotate_setup(self):
+        (status, output) = self.target.run('mkdir /home/root/logrotate_dir')
+        self.assertEqual(status, 0, msg = "Could not create logrotate_dir. Output: %s" % output)
+        (status, output) = self.target.run("sed -i 's#wtmp {#wtmp {\\n    olddir /home/root/logrotate_dir#' /etc/logrotate.conf")
+        self.assertEqual(status, 0, msg = "Could not write to logrotate.conf file. Status and output: %s and %s)" % (status, output))
+
+    @skipUnlessPassed("test_1_logrotate_setup")
+    def test_2_logrotate(self):
+        (status, output) = self.target.run('logrotate -f /etc/logrotate.conf')
+        self.assertEqual(status, 0, msg = "logrotate service could not be reloaded. Status and output: %s and %s" % (status, output))
+        output = self.target.run('ls -la /home/root/logrotate_dir/ | wc -l')[1]
+        self.assertTrue(int(output)>=3, msg = "new logfile could not be created. List of files within log directory: %s" %(self.target.run('ls -la /home/root/logrotate_dir')[1]))
-- 
1.8.3.1

[PATCH v2 05/19] lib/oeqa/runtime: add new skeletoninit test

[Stefan Stanacar]

From: Alexandru Palalau <alexandrux.palalau@intel.com>

New test which verifies the usage of skeleton init script available with meta-skeleton layer

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/skeletoninit.py | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 meta/lib/oeqa/skeletoninit.py

diff --git a/meta/lib/oeqa/skeletoninit.py b/meta/lib/oeqa/skeletoninit.py
new file mode 100644
index 0000000..557e715
--- /dev/null
+++ b/meta/lib/oeqa/skeletoninit.py
@@ -0,0 +1,28 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=284 testcase
+# Note that the image under test must have meta-skeleton layer in bblayers and IMAGE_INSTALL_append = " service" in local.conf
+
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasPackage("service"):
+        skipModule("No service package in image")
+
+
+class SkeletonBasicTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image")
+    def test_skeleton_availability(self):
+        (status, output) = self.target.run('ls /etc/init.d/skeleton')
+        self.assertEqual(status, 0, msg = "skeleton init script not found. Output:\n%s " % output)
+        (status, output) =  self.target.run('ls /usr/sbin/skeleton-test')
+        self.assertEqual(status, 0, msg = "skeleton-test not found. Output:\n%s" % output)
+
+    @skipUnlessPassed('test_skeleton_availability')
+    @unittest.skipIf("systemd" == oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager"), "Not appropiate for systemd image")
+    def test_skeleton_script(self):
+        output1 = self.target.run("/etc/init.d/skeleton start")[1]
+        (status, output2) = self.target.run(oeRuntimeTest.pscmd + ' | grep [s]keleton-test')
+        self.assertEqual(status, 0, msg = "Skeleton script could not be started:\n%s\n%s" % (output1, output2))
-- 
1.8.3.1

[PATCH v2 06/19] lib/oeqa/runtime: add new PAM support test

[Stefan Stanacar]

From: Alexandru Palalau <alexandrux.palalau@intel.com>

New test which verifies some usual commands functionality with PAM support

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/pam.py | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/pam.py

diff --git a/meta/lib/oeqa/runtime/pam.py b/meta/lib/oeqa/runtime/pam.py
new file mode 100644
index 0000000..52e1eb8
--- /dev/null
+++ b/meta/lib/oeqa/runtime/pam.py
@@ -0,0 +1,24 @@
+# This test should cover https://bugzilla.yoctoproject.org/tr_show_case.cgi?case_id=287 testcase
+# Note that the image under test must have "pam" in DISTRO_FEATURES
+
+import unittest
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("pam"):
+        skipModule("target doesn't have 'pam' in DISTRO_FEATURES")
+
+
+class PamBasicTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_pam(self):
+        (status, output) = self.target.run('login --help')
+        self.assertEqual(status, 1, msg = "login command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('passwd --help')
+        self.assertEqual(status, 6, msg = "passwd command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('su --help')
+        self.assertEqual(status, 2, msg = "su command does not work as expected. Status and output:%s and %s" %(status, output))
+        (status, output) = self.target.run('useradd --help')
+        self.assertEqual(status, 2, msg = "useradd command does not work as expected. Status and output:%s and %s" %(status, output))
-- 
1.8.3.1

[PATCH v2 07/19] lib/oeqa/runtime: add new scp test

[Stefan Stanacar]

From: Alexandru Palalau <alexandrux.palalau@intel.com>

Copies a 5MB to target using scp, more of an
network test than a scp one.

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/scp.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/scp.py

diff --git a/meta/lib/oeqa/runtime/scp.py b/meta/lib/oeqa/runtime/scp.py
new file mode 100644
index 0000000..b914802
--- /dev/null
+++ b/meta/lib/oeqa/runtime/scp.py
@@ -0,0 +1,22 @@
+import subprocess
+import unittest
+import os
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+
+def setUpModule():
+    if not (oeRuntimeTest.hasPackage("dropbear") or oeRuntimeTest.hasPackage("openssh-sshd")):
+        skipModule("No ssh package in image")
+
+
+class ScpTest(oeRuntimeTest):
+
+    def setUp(self):
+         subprocess.check_call("dd if=/dev/zero of=%s bs=512k count=10" % os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), shell=True)
+
+    @skipUnlessPassed('test_ssh')
+    def test_scp(self):
+        (status, output) = self.target.copy_to(os.path.join(oeRuntimeTest.tc.d.getVar("TEST_LOG_DIR", True), 'test_scp_file'), '/tmp/test_scp_file')
+        self.assertEqual(status, 0, msg = "File could not be copied. Output: %s" % output)
+        (status, output) = self.target.run("ls -la /tmp/test_scp_file")
+        self.assertEqual(status, 0, msg = "SCP test failed")
-- 
1.8.3.1

[PATCH v2 08/19] lib/oeqa/runtime: add adjust date and time test

[Stefan Stanacar]

From: Mihai Prica <mihai.prica@intel.com>

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/date.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/date.py

diff --git a/meta/lib/oeqa/runtime/date.py b/meta/lib/oeqa/runtime/date.py
new file mode 100644
index 0000000..d6c04cb
--- /dev/null
+++ b/meta/lib/oeqa/runtime/date.py
@@ -0,0 +1,27 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+import re
+
+def setUpModule():
+    skipModuleUnless(oeRuntimeTest.tc.target.run('which date')[0] == 0, "No date in image or no connection")
+
+class DateTest(oeRuntimeTest):
+
+    def setUp(self):
+        (status, output) = self.target.run('date +"%Y-%m-%d %T"')
+        self.assertEqual(status, 0, msg="Failed to get initial date, output: %s" % output)
+        self.oldDate = output
+
+    @skipUnlessPassed("test_ssh")
+    def test_date(self):
+        sampleDate = '"2016-08-09 10:00:00"'
+        (status, output) = self.target.run("date -s %s" % sampleDate)
+        self.assertEqual(status, 0, msg="Date set failed, output: %s" % output)
+
+        (status, output) = self.target.run("date -R")
+        p = re.match('Tue, 09 Aug 2016 10:00:.. \+0000', output)
+        self.assertTrue(p, msg="The date was not set correctly, output: %s" % output)
+
+    def tearDown(self):
+        (status, output) = self.target.run('date -s "%s"' % self.oldDate)
+        self.assertEqual(status, 0, msg="Failed to reset date, output: %s" % output)
-- 
1.8.3.1

[PATCH v2 09/19] lib/oeqa/runtime: add vncserver for target test

[Stefan Stanacar]

From: Mihai Prica <mihai.prica@intel.com>

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/vnc.py | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/vnc.py

diff --git a/meta/lib/oeqa/runtime/vnc.py b/meta/lib/oeqa/runtime/vnc.py
new file mode 100644
index 0000000..9476184
--- /dev/null
+++ b/meta/lib/oeqa/runtime/vnc.py
@@ -0,0 +1,22 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+import re
+
+def setUpModule():
+    skipModuleUnless(oeRuntimeTest.tc.target.run('which x11vnc')[0] == 0, "No x11vnc in image")
+
+class VNCTest(oeRuntimeTest):
+
+    @skipUnlessPassed('test_ssh')
+    def test_vnc(self):
+        (status, output) = self.target.run('x11vnc -display :0.0 -bg -q')
+        self.assertEqual(status, 0, msg="x11vnc server failed to start: %s" % output)
+        port = re.search('PORT=[0-9]*', output)
+        self.assertTrue(port, msg="Listening port not specified in command output: %s" %output)
+
+        (status, output) = self.target.run(oeRuntimeTest.pscmd + ' | grep -i [x]11vnc')
+        self.assertEqual(status, 0, msg="x11vnc process not running")
+
+        vncport = port.group(0).split('=')[1]
+        (status, output) = self.target.run('netstat -atun | grep :%s | grep LISTEN' % vncport)
+        self.assertEqual(status, 0, msg="x11vnc server not running on port %s" % vncport)
-- 
1.8.3.1

[PATCH v2 10/19] lib/oeqa/runtime: rpm: add install and erase tests

[Stefan Stanacar]

Copies to target rpm-doc file from deploy_dir
and tries to install and then remove that package.
rpm-doc was chosen because it's small, it only adds
a few files to target, and it's almost always found in
deploy_dir for images with package-management/rpm.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/rpm.py | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/runtime/rpm.py b/meta/lib/oeqa/runtime/rpm.py
index 57101b0..154cad5 100644
--- a/meta/lib/oeqa/runtime/rpm.py
+++ b/meta/lib/oeqa/runtime/rpm.py
@@ -1,6 +1,8 @@
 import unittest
+import os
 from oeqa.oetest import oeRuntimeTest, skipModule
 from oeqa.utils.decorators import *
+import oe.packagedata
 
 def setUpModule():
     if not oeRuntimeTest.hasFeature("package-management"):
@@ -9,17 +11,39 @@ def setUpModule():
             skipModule("rpm module skipped: target doesn't have rpm as primary package manager")
 
 
-class RpmHelpTest(oeRuntimeTest):
+class RpmBasicTest(oeRuntimeTest):
 
     @skipUnlessPassed('test_ssh')
     def test_rpm_help(self):
         (status, output) = self.target.run('rpm --help')
         self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
 
-class RpmQueryTest(oeRuntimeTest):
-
     @skipUnlessPassed('test_rpm_help')
     def test_rpm_query(self):
         (status, output) = self.target.run('rpm -q rpm')
         self.assertEqual(status, 0, msg="status and output: %s and %s" % (status,output))
 
+class RpmInstallRemoveTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        deploydir = os.path.join(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True), "rpm", oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
+        pkgdata = oe.packagedata.read_subpkgdata("rpm-doc", oeRuntimeTest.tc.d)
+        # pick rpm-doc as a test file to get installed, because it's small and it will always be built for standard targets
+        testrpmfile = "rpm-doc-%s-%s.%s.rpm" % (pkgdata["PKGV"], pkgdata["PKGR"], oeRuntimeTest.tc.d.getVar('TUNE_PKGARCH', True))
+        oeRuntimeTest.tc.target.copy_to(os.path.join(deploydir,testrpmfile), "/tmp/rpm-doc.rpm")
+
+    @skipUnlessPassed('test_rpm_help')
+    def test_rpm_install(self):
+        (status, output) = self.target.run('rpm -ivh /tmp/rpm-doc.rpm')
+        self.assertEqual(status, 0, msg="Failed to install rpm-doc package: %s" % output)
+
+    @skipUnlessPassed('test_rpm_install')
+    def test_rpm_remove(self):
+        (status,output) = self.target.run('rpm -e rpm-doc')
+        self.assertEqual(status, 0, msg="Failed to remove rpm-doc package: %s" % output)
+
+    @classmethod
+    def tearDownClass(self):
+        oeRuntimeTest.tc.target.run('rm -f /tmp/rpm-doc.rpm')
+
-- 
1.8.3.1

[PATCH v2 11/19] lib/oeqa/utils: new file: httpserver.py useful for serving files over HTTP to the target

[Stefan Stanacar]

From: Mihai Lindner <mihaix.lindner@linux.intel.com>

It can be used by smart repo/channel tests to serve deploy_dir.

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/utils/httpserver.py | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/httpserver.py

diff --git a/meta/lib/oeqa/utils/httpserver.py b/meta/lib/oeqa/utils/httpserver.py
new file mode 100644
index 0000000..d4b6154
--- /dev/null
+++ b/meta/lib/oeqa/utils/httpserver.py
@@ -0,0 +1,32 @@
+import SimpleHTTPServer
+import multiprocessing
+import os
+
+class HTTPServer(SimpleHTTPServer.BaseHTTPServer.HTTPServer):
+
+    def server_start(self, root_dir):
+        os.chdir(root_dir)
+        self.serve_forever()
+
+class HTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
+
+    def log_message(self, format_str, *args):
+        pass
+
+class HTTPService(object):
+
+    def __init__(self, root_dir):
+        self.root_dir = root_dir
+        self.port = 0
+
+    def start(self):
+        self.server = HTTPServer(('', self.port), HTTPRequestHandler)
+        if self.port == 0:
+            self.port = self.server.server_port
+        self.process = multiprocessing.Process(target=self.server.server_start, args=[self.root_dir])
+        self.process.start()
+
+    def stop(self):
+        self.server.server_close()
+        self.process.terminate()
+        self.process.join()
-- 
1.8.3.1

[PATCH v2 12/19] lib/oeqa/utils: qemurunner: save host IP address

[Stefan Stanacar]

From: Mihai Lindner <mihaix.lindner@linux.intel.com>

Save host IP address to host_ip.
Read /proc/PID/cmdline on host to look for IPs of target and host;
instead of running 'ps'.
Also removed some extra empty lines from file.

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/utils/qemurunner.py | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 20bb1e5..9ae618f 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -54,7 +54,6 @@ class QemuRunner:
 
     def launch(self, qemuparams = None):
 
-
         if self.display:
             os.environ["DISPLAY"] = self.display
         else:
@@ -84,12 +83,11 @@ class QemuRunner:
 
         if self.is_alive():
             bb.note("qemu started - qemu procces pid is %s" % self.qemupid)
-            pscmd = 'ps -p %s -fww | grep -o "192\.168\.7\.[0-9]*::" | awk -F":" \'{print $1}\'' % self.qemupid
-            self.ip = subprocess.Popen(pscmd,shell=True,stdout=subprocess.PIPE).communicate()[0].strip()
+            cmdline = open('/proc/%s/cmdline' % self.qemupid).read()
+            self.ip, _, self.host_ip = cmdline.split('ip=')[1].split(' ')[0].split(':')[0:3]
             if not re.search("^((?:[0-9]{1,3}\.){3}[0-9]{1,3})$", self.ip):
                 bb.note("Couldn't get ip from qemu process arguments, I got '%s'" % self.ip)
-                bb.note("Here is the ps output:\n%s" % \
-                        subprocess.Popen("ps -p %s -fww" % self.qemupid,shell=True,stdout=subprocess.PIPE).communicate()[0])
+                bb.note("Here is the ps output:\n%s" % cmdline)
                 self.kill()
                 return False
             bb.note("IP found: %s" % self.ip)
@@ -122,7 +120,6 @@ class QemuRunner:
                             sock.close()
                             stopread = True
 
-
             if not reachedlogin:
                 bb.note("Target didn't reached login boot in %d seconds" % self.boottime)
                 lines = "\n".join(self.bootlog.splitlines()[-5:])
@@ -139,7 +136,6 @@ class QemuRunner:
 
         return self.is_alive()
 
-
     def kill(self):
         if self.server_socket:
             self.server_socket.close()
@@ -207,4 +203,3 @@ class QemuRunner:
             basecmd = os.path.basename(basecmd)
             if "qemu-system" in basecmd and "-serial tcp" in commands[p]:
                 return [int(p),commands[p]]
-
-- 
1.8.3.1

[PATCH v2 13/19] lib/oeqa/utils: decorators: return the decorated method

[Stefan Stanacar]

From: Mihai Lindner <mihaix.lindner@linux.intel.com>

Decorators should return whatever the decorated methods return.

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/utils/decorators.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/utils/decorators.py b/meta/lib/oeqa/utils/decorators.py
index e0ca6fd..33fed5a 100644
--- a/meta/lib/oeqa/utils/decorators.py
+++ b/meta/lib/oeqa/utils/decorators.py
@@ -17,7 +17,7 @@ class skipIfFailure(object):
         def wrapped_f(*args):
             if self.testcase in (oeRuntimeTest.testFailures or oeRuntimeTest.testErrors):
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
 
@@ -30,7 +30,7 @@ class skipIfSkipped(object):
         def wrapped_f(*args):
             if self.testcase in oeRuntimeTest.testSkipped:
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
 
@@ -45,6 +45,6 @@ class skipUnlessPassed(object):
                     self.testcase in  oeRuntimeTest.testFailures or \
                     self.testcase in oeRuntimeTest.testErrors:
                 raise unittest.SkipTest("Testcase dependency not met: %s" % self.testcase)
-            f(*args)
+            return f(*args)
         wrapped_f.__name__ = f.__name__
         return wrapped_f
-- 
1.8.3.1

[PATCH v2 14/19] lib/oeqa/runtime: smart: add new smart tests

[Stefan Stanacar]

From: Mihai Lindner <mihaix.lindner@linux.intel.com>

Add class to be inherited by smart tests, along with more basic tests and tests
using a rpm repository.

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/smart.py | 102 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 91 insertions(+), 11 deletions(-)

diff --git a/meta/lib/oeqa/runtime/smart.py b/meta/lib/oeqa/runtime/smart.py
index 0b03a30..4ea2699 100644
--- a/meta/lib/oeqa/runtime/smart.py
+++ b/meta/lib/oeqa/runtime/smart.py
@@ -1,6 +1,7 @@
 import unittest
 from oeqa.oetest import oeRuntimeTest
 from oeqa.utils.decorators import *
+from oeqa.utils.httpserver import HTTPService
 
 def setUpModule():
     if not oeRuntimeTest.hasFeature("package-management"):
@@ -8,21 +9,100 @@ def setUpModule():
     if not oeRuntimeTest.hasPackage("smart"):
         skipModule("Image doesn't have smart installed")
 
-class SmartHelpTest(oeRuntimeTest):
+class SmartTest(oeRuntimeTest):
+
+    longMessage = True
+
+    @skipUnlessPassed('test_smart_help')
+    def smart(self, command, expected = 0):
+        command = 'smart %s' % command
+        status, output = self.target.run(command)
+        message = os.linesep.join([command, output])
+        self.assertEqual(status, expected, message)
+        return output
+
+class SmartBasicTest(SmartTest):
 
     @skipUnlessPassed('test_ssh')
     def test_smart_help(self):
-        status = self.target.run('smart --help')[0]
-        self.assertEqual(status, 0)
+        self.smart('--help')
 
-class SmartQueryTest(oeRuntimeTest):
+    def test_smart_version(self):
+        self.smart('--version')
+
+    def test_smart_info(self):
+        self.smart('info python-smartpm')
 
-    @skipUnlessPassed('test_smart_help')
     def test_smart_query(self):
-        (status, output) = self.target.run('smart query rpm')
-        self.assertEqual(status, 0, msg="smart query failed, output: %s" % output)
+        self.smart('query python-smartpm')
 
-    @skipUnlessPassed('test_smart_query')
-    def test_smart_info(self):
-        (status, output) = self.target.run('smart info rpm')
-        self.assertEqual(status, 0, msg="smart info rpm failed, output: %s" % output)
+    def test_smart_search(self):
+        self.smart('search python-smartpm')
+
+    def test_smart_stats(self):
+        self.smart('stats')
+
+class SmartRepoTest(SmartTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.repo_server = HTTPService(oeRuntimeTest.tc.d.getVar('DEPLOY_DIR', True))
+        self.repo_server.start()
+
+    @classmethod
+    def tearDownClass(self):
+        self.repo_server.stop()
+
+    def test_smart_channel(self):
+        self.smart('channel', 1)
+
+    def test_smart_channel_add(self):
+        image_pkgtype = self.tc.d.getVar('IMAGE_PKGTYPE', True)
+        deploy_url = 'http://%s:%s/%s' %(self.tc.qemu.host_ip, self.repo_server.port, image_pkgtype)
+        for arch in os.listdir('%s/%s' % (self.repo_server.root_dir, image_pkgtype)):
+            self.smart('channel -y --add {a} type=rpm-md baseurl={u}/{a}'.format(a=arch, u=deploy_url))
+        self.smart('update')
+
+    def test_smart_channel_help(self):
+        self.smart('channel --help')
+
+    def test_smart_channel_list(self):
+        self.smart('channel --list')
+
+    def test_smart_channel_show(self):
+        self.smart('channel --show')
+
+    def test_smart_channel_rpmsys(self):
+        self.smart('channel --show rpmsys')
+        self.smart('channel --disable rpmsys')
+        self.smart('channel --enable rpmsys')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install(self):
+        self.smart('remove -y psplash-default')
+        self.smart('install -y psplash-default')
+
+    @skipUnlessPassed('test_smart_install')
+    def test_smart_install_dependency(self):
+        self.smart('remove -y psplash')
+        self.smart('install -y psplash-default')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install_from_disk(self):
+        self.smart('remove -y psplash-default')
+        self.smart('download psplash-default')
+        self.smart('install -y ./psplash-default*')
+
+    @skipUnlessPassed('test_smart_channel_add')
+    def test_smart_install_from_http(self):
+        url = 'http://'
+        output = self.smart('download --urls psplash-default')
+        for line in output.splitlines():
+            if line.startswith(url):
+                url = line
+        self.smart('remove -y psplash-default')
+        self.smart('install -y %s' % url)
+
+    @skipUnlessPassed('test_smart_install')
+    def test_smart_reinstall(self):
+        self.smart('reinstall -y psplash-default')
-- 
1.8.3.1

[PATCH v2 15/19] lib/oeqa: make it possible to restart the target

[Stefan Stanacar]

Tweak QemuRunner so we can actually restart the
qemu target in a test (if we want more memory for example).
Also add a restart method to the base test class so that tests
can use it.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/oetest.py           |  7 +++++++
 meta/lib/oeqa/utils/qemurunner.py | 28 +++++++++++++++++++++-------
 2 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 7f6baa4..4a406e7 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -81,6 +81,13 @@ class oeRuntimeTest(unittest.TestCase):
         else:
             return False
 
+    @classmethod
+    def restartTarget(self,params=None):
+
+        if oeRuntimeTest.tc.qemu.restart(params):
+            oeRuntimeTest.tc.target.host = oeRuntimeTest.tc.qemu.ip
+        else:
+            raise Exception("Restarting target failed")
 
 
 def getmodule(pos=2):
diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py
index 9ae618f..6ee5b85 100644
--- a/meta/lib/oeqa/utils/qemurunner.py
+++ b/meta/lib/oeqa/utils/qemurunner.py
@@ -32,6 +32,10 @@ class QemuRunner:
         self.boottime = boottime
         self.runqemutime = runqemutime
 
+        self.create_socket()
+
+    def create_socket(self):
+
         self.bootlog = ''
         self.qemusock = None
 
@@ -137,21 +141,31 @@ class QemuRunner:
         return self.is_alive()
 
     def kill(self):
+
+        if self.runqemu:
+            bb.note("Sending SIGTERM to runqemu")
+            os.kill(-self.runqemu.pid,signal.SIGTERM)
+            endtime = time.time() + self.runqemutime
+            while self.runqemu.poll() is None and time.time() < endtime:
+                time.sleep(1)
+            if self.runqemu.poll() is None:
+                bb.note("Sending SIGKILL to runqemu")
+                os.kill(-self.runqemu.pid,signal.SIGKILL)
+            self.runqemu = None
         if self.server_socket:
             self.server_socket.close()
             self.server_socket = None
-        if self.runqemu.pid:
-            os.kill(-self.runqemu.pid,signal.SIGTERM)
-            os.kill(-self.runqemu.pid,signal.SIGKILL)
-            self.runqemu.pid = None
         self.qemupid = None
         self.ip = None
 
     def restart(self, qemuparams = None):
-        if self.is_alive():
+        bb.note("Restarting qemu process")
+        if self.runqemu.poll() is None:
             self.kill()
-        bb.note("Qemu Restart required...")
-        return self.launch(qemuparams)
+        self.create_socket()
+        if self.launch(qemuparams):
+            return True
+        return False
 
     def is_alive(self):
         qemu_child = self.find_child(str(self.runqemu.pid))
-- 
1.8.3.1

[PATCH v2 16/19] lib/oeqa/utils: targetbuild: Add helper class for building packages on target

[Stefan Stanacar]

From: Mihai Prica <mihai.prica@intel.com>

This class can be used for test cases that configure
and build packages on target.

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/utils/targetbuild.py | 63 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)
 create mode 100644 meta/lib/oeqa/utils/targetbuild.py

diff --git a/meta/lib/oeqa/utils/targetbuild.py b/meta/lib/oeqa/utils/targetbuild.py
new file mode 100644
index 0000000..9b2cf53
--- /dev/null
+++ b/meta/lib/oeqa/utils/targetbuild.py
@@ -0,0 +1,63 @@
+# Copyright (C) 2013 Intel Corporation
+#
+# Released under the MIT license (see COPYING.MIT)
+
+# Provides a class for automating build tests for projects
+
+from oeqa.oetest import oeRuntimeTest
+import bb.fetch2
+import bb.data
+import os
+import re
+
+
+class TargetBuildProject():
+
+    def __init__(self, target, uri, foldername=None):
+        self.target = target
+        self.uri = uri
+        self.targetdir = "/home/root/"
+
+        self.localdata = bb.data.createCopy(oeRuntimeTest.tc.d)
+        bb.data.update_data(self.localdata)
+
+        if not foldername:
+            self.archive = os.path.basename(uri)
+            self.fname = re.sub(r'.tar.bz2|tar.gz$', '', self.archive)
+        else:
+            self.fname = foldername
+
+    def download_archive(self):
+
+        try:
+            self.localdata.delVar("BB_STRICT_CHECKSUM")
+            fetcher = bb.fetch2.Fetch([self.uri], self.localdata)
+            fetcher.download()
+            self.localarchive = fetcher.localpath(self.uri)
+        except bb.fetch2.BBFetchException:
+            raise Exception("Failed to download archive: %s" % self.uri)
+
+        (status, output) = self.target.copy_to(self.localarchive, self.targetdir)
+        if status != 0:
+            raise Exception("Failed to copy archive to target, output: %s" % output)
+
+        (status, output) = self.target.run('tar xf %s%s -C %s' % (self.targetdir, self.archive, self.targetdir))
+        if status != 0:
+            raise Exception("Failed to extract archive, output: %s" % output)
+
+        #Change targetdir to project folder
+        self.targetdir = self.targetdir + self.fname
+
+    # The timeout parameter of target.run is set to 0 to make the ssh command
+    # run with no timeout.
+    def run_configure(self):
+        return self.target.run('cd %s; ./configure' % self.targetdir, 0)[0]
+
+    def run_make(self):
+        return self.target.run('cd %s; make' % self.targetdir, 0)[0]
+
+    def run_install(self):
+        return self.target.run('cd %s; make install' % self.targetdir, 0)[0]
+
+    def clean(self):
+        self.target.run('rm -rf %s' % self.targetdir)
-- 
1.8.3.1

[PATCH v2 17/19] lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target

[Stefan Stanacar]

From: Mihai Prica <mihai.prica@intel.com>

Downloads iptables/cvs/sudoku-savant sources and builds them on target.

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/runtime/buildcvs.py      | 32 ++++++++++++++++++++++++++++++++
 meta/lib/oeqa/runtime/buildiptables.py | 32 ++++++++++++++++++++++++++++++++
 meta/lib/oeqa/runtime/buildsudoku.py   | 29 +++++++++++++++++++++++++++++
 3 files changed, 93 insertions(+)
 create mode 100644 meta/lib/oeqa/runtime/buildcvs.py
 create mode 100644 meta/lib/oeqa/runtime/buildiptables.py
 create mode 100644 meta/lib/oeqa/runtime/buildsudoku.py

diff --git a/meta/lib/oeqa/runtime/buildcvs.py b/meta/lib/oeqa/runtime/buildcvs.py
new file mode 100644
index 0000000..f024dfa
--- /dev/null
+++ b/meta/lib/oeqa/runtime/buildcvs.py
@@ -0,0 +1,32 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class BuildCvsTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.restartTarget("-m 512")
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://ftp.gnu.org/non-gnu/cvs/source/feature/1.12.13/cvs-1.12.13.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_cvs(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+        self.assertEqual(self.project.run_install(), 0,
+                        msg="Running make install failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
+        self.restartTarget()
diff --git a/meta/lib/oeqa/runtime/buildiptables.py b/meta/lib/oeqa/runtime/buildiptables.py
new file mode 100644
index 0000000..88ece3b
--- /dev/null
+++ b/meta/lib/oeqa/runtime/buildiptables.py
@@ -0,0 +1,32 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class BuildIptablesTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.restartTarget("-m 512")
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://netfilter.org/projects/iptables/files/iptables-1.4.13.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_iptables(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+        self.assertEqual(self.project.run_install(), 0,
+                        msg="Running make install failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
+        self.restartTarget()
diff --git a/meta/lib/oeqa/runtime/buildsudoku.py b/meta/lib/oeqa/runtime/buildsudoku.py
new file mode 100644
index 0000000..0a7306d
--- /dev/null
+++ b/meta/lib/oeqa/runtime/buildsudoku.py
@@ -0,0 +1,29 @@
+from oeqa.oetest import oeRuntimeTest
+from oeqa.utils.decorators import *
+from oeqa.utils.targetbuild import TargetBuildProject
+
+def setUpModule():
+    if not oeRuntimeTest.hasFeature("tools-sdk"):
+        skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
+
+class SudokuTest(oeRuntimeTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.restartTarget("-m 512")
+        self.project = TargetBuildProject(oeRuntimeTest.tc.target,
+                        "http://downloads.sourceforge.net/project/sudoku-savant/sudoku-savant/sudoku-savant-1.3/sudoku-savant-1.3.tar.bz2")
+        self.project.download_archive()
+
+    @skipUnlessPassed("test_ssh")
+    def test_sudoku(self):
+        self.assertEqual(self.project.run_configure(), 0,
+                        msg="Running configure failed")
+
+        self.assertEqual(self.project.run_make(), 0,
+                        msg="Running make failed")
+
+    @classmethod
+    def tearDownClass(self):
+        self.project.clean()
+        self.restartTarget()
-- 
1.8.3.1

[PATCH v2 18/19] lib/oeqa: change behaviour for unskippable tests

[Stefan Stanacar]

When a test module wants to be skipped because it doesn't
apply to the image but it was nevertheless a required
test (one in TEST_SUITES), we issued an warning that it
was a required test and went on with running the module.
Usually all tests in the module failed (e.g gcc tests on a non-sdk image),
but this allowed us to know that something went wrong with the image
(some package/feature didn't make it).

However, instead of just issuing an warning and running the tests
it's better to throw an exception. The traceback will tell us what's wrong,
and we don't run every single test method.
Output will look like this:
--snip--
| NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
| NOTE: Found 5 tests
| test_ping (oeqa.runtime.ping.PingTest) ... ok
| test_ssh (oeqa.runtime.ssh.SshTest) ... ok
| ERROR
|
| ======================================================================
| ERROR: setUpModule (oeqa.runtime.gcc)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
|     skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
|     "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
| Exception:
| Test gcc wants to be skipped.
| Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
| Test was required in TEST_SUITES, so either the condition for skipping is wrong
| or the image really doesn't have the requred feature/package when it should.
|
| ----------------------------------------------------------------------
| Ran 2 tests in 1.036s
|
| FAILED (errors=1)
| NOTE: Sending SIGTERM to runqemu
--snip--

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/lib/oeqa/oetest.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 4a406e7..c9dc5dc 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -103,7 +103,9 @@ def skipModule(reason, pos=2):
     if modname not in oeRuntimeTest.tc.testsrequired:
         raise unittest.SkipTest("%s: %s" % (modname, reason))
     else:
-        bb.warn("Test %s is required, not skipping" % modname)
+        raise Exception("\nTest %s wants to be skipped.\nReason is: %s" \
+                "\nTest was required in TEST_SUITES, so either the condition for skipping is wrong" \
+                "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
 
 def skipModuleIf(cond, reason):
 
-- 
1.8.3.1

[PATCH v2 19/19] classess/testimage: change default test suites

[Stefan Stanacar]

Some new tests were added, safe to have them in the defaults
for sato-sdk and sato. Not all of the new tests are here though,
either because they aren't applicable to default images or take too long.
(like build iptables/cvs/sudoky ones, they can be enabled
in local.conf and a special target on AB setups.). Also reorder them a bit.

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
---
 meta/classes/testimage.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 86121e4..2bebbd8 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -26,8 +26,8 @@ TEST_LOG_DIR ?= "${WORKDIR}/testimage"
 
 DEFAULT_TEST_SUITES = "ping auto"
 DEFAULT_TEST_SUITES_pn-core-image-minimal = "ping"
-DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh connman df rpm smart xorg syslog dmesg"
-DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh connman df rpm smart gcc xorg syslog dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
+DEFAULT_TEST_SUITES_pn-core-image-sato-sdk = "ping ssh df connman syslog xorg scp vnc date perl ldd gcc rpm smart dmesg"
 
 TEST_SUITES ?= "${DEFAULT_TEST_SUITES}"
 
-- 
1.8.3.1