[PATCH 0/6] various oeqa fixes

[PATCH 0/6] various oeqa fixes

[André Draszik]

Hi,

These patches:
* allow hardware controlled targets work with testimage again
  In particular, I have a custom target hardware controller
  modelled after the ssh/qemu controllers and it works now
* fix a few issues

Cheers,
Andre'

PS: BTW, not related to this series, but it appears as if
meta-yocto-bsp's hardware controllers have suffered from
bitrot and can not work. Am I missing something?

[PATCH 1/6] testimage.bbclass: support hardware-controlled targets

[André Draszik]

From: André Draszik <andre.draszik@jci.com>

Since the introduction of the new runtime framework for target
testing in commit 2aa5a4954d76
("testimage.bbclass: Migrate class to use new runtime framework")
commit 3857e5c91da6 in poky.git, target controllers have no
access to the global datastore 'd' anymore.

This makes it impossible for a specific OEQA (hardware)
controller to access documented properties like
TEST_POWERCONTROL_CMD, TEST_SERIALCONTROL_CMD, etc,
meaning it's impossible for those controllers to actually
control the hardware.

To solve this, simply add those documented variables into
the target_kwargs[].

Signed-off-by: André Draszik <andre.draszik@jci.com>
---
 meta/classes/testimage.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 525c5a6173..befda6b72c 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -262,6 +262,12 @@ def testimage_main(d):
     # It would be better to find these modules using instrospection.
     target_kwargs['target_modules_path'] = d.getVar('BBPATH')
 
+    # hardware controlled targets might need further access
+    target_kwargs['powercontrol_cmd'] = d.getVar("TEST_POWERCONTROL_CMD") or None
+    target_kwargs['powercontrol_extra_args'] = d.getVar("TEST_POWERCONTROL_EXTRA_ARGS") or ""
+    target_kwargs['serialcontrol_cmd'] = d.getVar("TEST_SERIALCONTROL_CMD") or None
+    target_kwargs['serialcontrol_extra_args'] = d.getVar("TEST_SERIALCONTROL_EXTRA_ARGS") or ""
+
     # runtime use network for download projects for build
     export_proxies(d)
 
-- 
2.23.0.rc1

[PATCH 2/6] testimage.bbclass: enable ssh agent forwarding

[André Draszik]

Some targets might use ssh to do their power- or serial-
control. In that case, ssh might need access to the
ssh agent, or otherwise won't work.

So export it into the environment.

Note that the (old) oeqa/controllers/masterimage.py
tries to do that as well by exporting all of BB_ORIGENV
into the test environment. Here in testimage.bbclass we
are a bit more strict and only pass the ssh related
environment variables.

Signed-off-by: André Draszik <git@andred.net>
---
 meta/classes/testimage.bbclass | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index befda6b72c..844ed87944 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -268,6 +268,18 @@ def testimage_main(d):
     target_kwargs['serialcontrol_cmd'] = d.getVar("TEST_SERIALCONTROL_CMD") or None
     target_kwargs['serialcontrol_extra_args'] = d.getVar("TEST_SERIALCONTROL_EXTRA_ARGS") or ""
 
+    def export_ssh_agent(d):
+        import os
+
+        variables = ['SSH_AGENT_PID', 'SSH_AUTH_SOCK']
+        for v in variables:
+            if v not in os.environ.keys():
+                val = d.getVar(v)
+                if val is not None:
+                    os.environ[v] = val
+
+    export_ssh_agent(d)
+
     # runtime use network for download projects for build
     export_proxies(d)
 
-- 
2.23.0.rc1

[PATCH 3/6] oeqa/runtime/df: don't fail on long device names

[André Draszik]

When device names are long (more than 20 characters), the
df test will fail with an exception:
    self.assertTrue(int(output)>5120, msg=msg)
    ValueError: invalid literal for int() with base 10: ''
at least when busybox is in use.

The reason is that busybox breaks the line in that case:
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02
                           1998672     87024   1790408   5% /
and the code tries to extract the fourth field from the
second line, which is empty of course.

df can be told not to break lines, though, using the -P
flag, which turns on the POSIX output format, and is
supported by busybox df and coreutils df:
    Filesystem           1024-blocks    Used Available Capacity Mounted on
    /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02   1998672     87024   1790408   5% /

Signed-off-by: André Draszik <git@andred.net>
---
 meta/lib/oeqa/runtime/cases/df.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/df.py b/meta/lib/oeqa/runtime/cases/df.py
index d8d79f32ea..89fd0fb901 100644
--- a/meta/lib/oeqa/runtime/cases/df.py
+++ b/meta/lib/oeqa/runtime/cases/df.py
@@ -11,7 +11,7 @@ class DfTest(OERuntimeTestCase):
     @OETestDepends(['ssh.SSHTest.test_ssh'])
     @OEHasPackage(['coreutils', 'busybox'])
     def test_df(self):
-        cmd = "df / | sed -n '2p' | awk '{print $4}'"
+        cmd = "df -P / | sed -n '2p' | awk '{print $4}'"
         (status,output) = self.target.run(cmd)
         msg = 'Not enough space on image. Current size is %s' % output
         self.assertTrue(int(output)>5120, msg=msg)
-- 
2.23.0.rc1

[PATCH 4/6] oeqa/core/decorator: add skipIfFeature

[André Draszik]

skipIfFeature will skip a test if a given DIST_FEATURE
or IMAGE_FEATURE is enabled.

Signed-off-by: André Draszik <git@andred.net>
---
 meta/lib/oeqa/core/decorator/data.py | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py
index babc9789d6..12d462f202 100644
--- a/meta/lib/oeqa/core/decorator/data.py
+++ b/meta/lib/oeqa/core/decorator/data.py
@@ -113,3 +113,21 @@ class skipIfNotFeature(OETestDecorator):
         self.logger.debug(msg)
         if not has_feature(self.case.td, self.value):
             self.case.skipTest(self.msg)
+
+@registerDecorator
+class skipIfFeature(OETestDecorator):
+    """
+        Skip test based on DISTRO_FEATURES.
+
+        value must not be in distro features or it will skip the test
+        with msg as the reason.
+    """
+
+    attrs = ('value', 'msg')
+
+    def setUpDecorator(self):
+        msg = ('Checking if %s is not in DISTRO_FEATURES '
+               'or IMAGE_FEATURES' % (self.value))
+        self.logger.debug(msg)
+        if has_feature(self.case.td, self.value):
+            self.case.skipTest(self.msg)
-- 
2.23.0.rc1

[PATCH 5/6] oeqa/runtime/opkg: skip install on read-only-rootfs

[André Draszik]

Images can have package management enabled, but be
generally running as read-only. In this case, the
test fails at the moment with various errors due to
that.

Use the new @skipIfFeature decorator to also skip
this test in that case.

Signed-off-by: André Draszik <git@andred.net>
---
 meta/lib/oeqa/runtime/cases/opkg.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/opkg.py b/meta/lib/oeqa/runtime/cases/opkg.py
index bb8b6d99d2..750706161b 100644
--- a/meta/lib/oeqa/runtime/cases/opkg.py
+++ b/meta/lib/oeqa/runtime/cases/opkg.py
@@ -5,7 +5,7 @@
 import os
 from oeqa.utils.httpserver import HTTPService
 from oeqa.runtime.case import OERuntimeTestCase
-from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature
+from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature, skipIfFeature
 from oeqa.runtime.decorator.package import OEHasPackage
 
 class OpkgTest(OERuntimeTestCase):
@@ -45,6 +45,8 @@ class OpkgRepoTest(OpkgTest):
                       'Test requires package-management to be in IMAGE_FEATURES')
     @skipIfNotDataVar('IMAGE_PKGTYPE', 'ipk',
                       'IPK is not the primary package manager')
+    @skipIfFeature('read-only-rootfs',
+                   'Test does not work with read-only-rootfs in IMAGE_FEATURES')
     @OEHasPackage(['opkg'])
     def test_opkg_install_from_repo(self):
         self.setup_source_config_for_package_install()
-- 
2.23.0.rc1

[PATCH 6/6] oeqa/runtime/systemd: skip unit enable/disable on read-only-rootfs

[André Draszik]

This doesn't work on read-only-rootfs:
    AssertionError: 1 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl disable avahi-daemon.service
    Failed to disable unit: File /etc/systemd/system/multi-user.target.wants/avahi-daemon.service: Read-only file system

Skip the test in that case to avoid this.

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: André Draszik <git@andred.net>
---
 meta/lib/oeqa/runtime/cases/systemd.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/systemd.py b/meta/lib/oeqa/runtime/cases/systemd.py
index c11fa49b07..e33c8a41a7 100644
--- a/meta/lib/oeqa/runtime/cases/systemd.py
+++ b/meta/lib/oeqa/runtime/cases/systemd.py
@@ -9,7 +9,7 @@ from oeqa.runtime.case import OERuntimeTestCase
 from oeqa.core.decorator.depends import OETestDepends
 from oeqa.core.decorator.data import skipIfDataVar, skipIfNotDataVar
 from oeqa.runtime.decorator.package import OEHasPackage
-from oeqa.core.decorator.data import skipIfNotFeature
+from oeqa.core.decorator.data import skipIfNotFeature, skipIfFeature
 
 class SystemdTest(OERuntimeTestCase):
 
@@ -114,6 +114,8 @@ class SystemdServiceTests(SystemdTest):
         self.systemctl('is-active', 'avahi-daemon.service', verbose=True)
 
     @OETestDepends(['systemd.SystemdServiceTests.test_systemd_status'])
+    @skipIfFeature('read-only-rootfs',
+                   'Test does not work with read-only-rootfs in IMAGE_FEATURES')
     def test_systemd_disable_enable(self):
         self.systemctl('disable', 'avahi-daemon.service')
         self.systemctl('is-enabled', 'avahi-daemon.service', expected=1)
-- 
2.23.0.rc1

[PATCH v2] oeqa/runtime/systemd: skip unit enable/disable on read-only-rootfs

[André Draszik]

This doesn't work on read-only-rootfs:
    AssertionError: 1 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl disable avahi-daemon.service
    Failed to disable unit: File /etc/systemd/system/multi-user.target.wants/avahi-daemon.service: Read-only file system

This patch does two things:
1) Decorate the existing test to be skipped if the rootfs is
   read-only
2) add a new test to be executed only if the rootfs is
   read-only. This new test remounts the rootfs read-write
   before continuing to execute the existing test, making
   sure to clean up correctly after itself (remount r/o
   again).

Signed-off-by: André Draszik <git@andred.net>

---
v2: We can do better by adding this 2nd test described above,
rather than just doing 1) described above
---
 meta/lib/oeqa/runtime/cases/systemd.py | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/runtime/cases/systemd.py b/meta/lib/oeqa/runtime/cases/systemd.py
index c11fa49b07..7c44abe8ed 100644
--- a/meta/lib/oeqa/runtime/cases/systemd.py
+++ b/meta/lib/oeqa/runtime/cases/systemd.py
@@ -9,7 +9,7 @@ from oeqa.runtime.case import OERuntimeTestCase
 from oeqa.core.decorator.depends import OETestDepends
 from oeqa.core.decorator.data import skipIfDataVar, skipIfNotDataVar
 from oeqa.runtime.decorator.package import OEHasPackage
-from oeqa.core.decorator.data import skipIfNotFeature
+from oeqa.core.decorator.data import skipIfNotFeature, skipIfFeature
 
 class SystemdTest(OERuntimeTestCase):
 
@@ -114,12 +114,26 @@ class SystemdServiceTests(SystemdTest):
         self.systemctl('is-active', 'avahi-daemon.service', verbose=True)
 
     @OETestDepends(['systemd.SystemdServiceTests.test_systemd_status'])
+    @skipIfFeature('read-only-rootfs',
+                   'Test is only meant to run without read-only-rootfs in IMAGE_FEATURES')
     def test_systemd_disable_enable(self):
         self.systemctl('disable', 'avahi-daemon.service')
         self.systemctl('is-enabled', 'avahi-daemon.service', expected=1)
         self.systemctl('enable', 'avahi-daemon.service')
         self.systemctl('is-enabled', 'avahi-daemon.service')
 
+    @OETestDepends(['systemd.SystemdServiceTests.test_systemd_status'])
+    @skipIfNotFeature('read-only-rootfs',
+                      'Test is only meant to run with read-only-rootfs in IMAGE_FEATURES')
+    def test_systemd_disable_enable_ro(self):
+        status = self.target.run('mount -orw,remount /')[0]
+        self.assertTrue(status == 0, msg='Remounting / as r/w failed')
+        try:
+            self.test_systemd_disable_enable()
+        finally:
+            status = self.target.run('mount -oro,remount /')[0]
+            self.assertTrue(status == 0, msg='Remounting / as r/o failed')
+
 class SystemdJournalTests(SystemdTest):
 
     @OETestDepends(['systemd.SystemdBasicTests.test_systemd_basic'])
-- 
2.23.0.rc1