* [PATCH] testimage: Run commands in target and host when test fails
@ 2015-08-17 12:41 mariano.lopez
2015-08-18 8:33 ` Paul Eggleton
0 siblings, 1 reply; 4+ messages in thread
From: mariano.lopez @ 2015-08-17 12:41 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
This patch modify three files altought two of them
are minimal modifications:
testimage.bbclass:
Create new vars for easy modification of the dump
directory and commands to be run on host and target
when a test fails
TESTIMAGE_DUMP_DIR: Directory to save the dumps
TESTIMAGE_DUMP_TRG: Commands to run on target
TESTIMAGE_DUMP_HST: Commands to run on host
targetcontrol.py:
Extract vars from the datastore for later use
oetest.py:
- Allow to use the vars defined in testimage class
- Now able to run commands in the host and dump the
results
- Fix an issue with the condition where to run the
dump commands (Before it run the commands every
test after a failure, now it runs the commands only
in tests that failed)
- Fix the output to stdout
[YOCTO #8118]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta/classes/testimage.bbclass | 9 +++++++++
meta/lib/oeqa/oetest.py | 42 +++++++++++++++++++++++++++---------------
meta/lib/oeqa/targetcontrol.py | 3 +++
3 files changed, 39 insertions(+), 15 deletions(-)
diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
index 140babe..1580a4e 100644
--- a/meta/classes/testimage.bbclass
+++ b/meta/classes/testimage.bbclass
@@ -56,6 +56,15 @@ TESTIMAGEDEPENDS_qemuall = "qemu-native:do_populate_sysroot qemu-helper-native:d
TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
TESTIMAGELOCK_qemuall = ""
+TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
+
+python () {
+ target_cmds = ["top -bn1", "ps", "free", "df", "_ping", "dmesg", "netstat -an", "ip address", "_logs"]
+ host_cmds = ["top -bn1", "ps", "free", "df", "memstat", "dmesg", "netstat -an"]
+ d.setVar("TESTIMAGE_DUMP_TRG", target_cmds)
+ d.setVar("TESTIMAGE_DUMP_HST", host_cmds)
+}
+
python do_testimage() {
testimage_main(d)
}
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index dfed3de..fa8f127 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -12,8 +12,10 @@ import unittest
import inspect
import subprocess
import datetime
+import commands
import bb
from oeqa.utils.decorators import LogResults
+from sys import exc_info, exc_clear
def loadTests(tc, type="runtime"):
if type == "runtime":
@@ -120,35 +122,45 @@ class oeRuntimeTest(oeTest):
def tearDown(self):
# If a test fails or there is an exception
- if (self._resultForDoCleanups.failures or
- self._resultForDoCleanups.errors):
- self.dump_target_logs()
-
- def dump_target_logs(self):
- commands = ["top -bn1", "ps", "free", "df", "_ping", "dmesg", "netstat -a", "ifconfig -a", "_logs"]
- dump_dir = "/tmp/oe-saved-tests"
+ if not exc_info() == (None, None, None):
+ exc_clear()
+ dump_dir = self.create_dump_dir()
+ print ("%s dump data from host and target "
+ "stored in %s" % (self._testMethodName, dump_dir))
+ self.dump_host_logs(dump_dir)
+ self.dump_target_logs(dump_dir)
+
+ def create_dump_dir(self):
dump_sub_dir = ("%s_%s" % (
datetime.datetime.now().strftime('%Y%m%d%H%M'),
self._testMethodName))
- dump_dir = os.path.join(dump_dir, dump_sub_dir)
+ dump_dir = os.path.join(self.target.dump_dir, dump_sub_dir)
os.makedirs(dump_dir)
- bb.warn("%s failed: getting data from target and "
- "saving into %s" % (self._testMethodName, dump_dir))
- for command in commands:
+ return dump_dir
+
+ def dump_host_logs(self, dump_dir):
+ for cmd in self.target.dump_hst:
+ output = commands.getoutput(cmd)
+ filename = "host_%s" % cmd.split()[0]
+ with open(os.path.join(dump_dir, filename), 'w') as f:
+ f.write(output)
+
+ def dump_target_logs(self, dump_dir):
+ for cmd in self.target.dump_trg:
# This will ping the host from target
- if command == "_ping":
+ if cmd == "_ping":
comm = "ping -c3 %s" % self.target.server_ip
# This will get all the logs from /var/log/
- elif command == "_logs":
+ elif cmd == "_logs":
comm = 'find /var/log/ -type f 2>/dev/null '
comm = '%s-exec echo "%s" \\; ' % (comm, '='*20)
comm = '%s-exec echo {} \\; ' % comm
comm = '%s-exec echo "%s" \\; ' % (comm, '='*20)
comm = '%s-exec cat {} \\; -exec echo "" \\;' % comm
else:
- comm = command
+ comm = cmd
(status, output) = self.target.run_serial(comm)
- filename = command.split()[0]
+ filename = "target_%s" % cmd.split()[0]
with open(os.path.join(dump_dir, filename), 'w') as f:
f.write(output)
diff --git a/meta/lib/oeqa/targetcontrol.py b/meta/lib/oeqa/targetcontrol.py
index c76887b..7589ea4 100644
--- a/meta/lib/oeqa/targetcontrol.py
+++ b/meta/lib/oeqa/targetcontrol.py
@@ -123,6 +123,9 @@ class QemuTarget(BaseTarget):
self.origrootfs = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("IMAGE_LINK_NAME", True) + '.' + self.image_fstype)
self.rootfs = os.path.join(self.testdir, d.getVar("IMAGE_LINK_NAME", True) + '-testimage.' + self.image_fstype)
self.kernel = os.path.join(d.getVar("DEPLOY_DIR_IMAGE", True), d.getVar("KERNEL_IMAGETYPE", False) + '-' + d.getVar('MACHINE', False) + '.bin')
+ self.dump_trg = d.getVar("TESTIMAGE_DUMP_TRG", False)
+ self.dump_hst = d.getVar("TESTIMAGE_DUMP_HST", False)
+ self.dump_dir = d.getVar("TESTIMAGE_DUMP_DIR", False)
# Log QemuRunner log output to a file
import oe.path
--
1.8.4.5
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] testimage: Run commands in target and host when test fails
2015-08-17 12:41 [PATCH] testimage: Run commands in target and host when test fails mariano.lopez
@ 2015-08-18 8:33 ` Paul Eggleton
2015-08-18 15:18 ` Mariano Lopez
0 siblings, 1 reply; 4+ messages in thread
From: Paul Eggleton @ 2015-08-18 8:33 UTC (permalink / raw)
To: mariano.lopez; +Cc: openembedded-core
Hi Mariano,
On Monday 17 August 2015 12:41:42 mariano.lopez@linux.intel.com wrote:
> From: Mariano Lopez <mariano.lopez@linux.intel.com>
>
> This patch modify three files altought two of them
> are minimal modifications:
>
> testimage.bbclass:
> Create new vars for easy modification of the dump
> directory and commands to be run on host and target
> when a test fails
> TESTIMAGE_DUMP_DIR: Directory to save the dumps
> TESTIMAGE_DUMP_TRG: Commands to run on target
> TESTIMAGE_DUMP_HST: Commands to run on host
Can we please use full names (i.e. "...HOST" and "...TARGET")? We're not short
of space here ;)
> targetcontrol.py:
> Extract vars from the datastore for later use
>
> oetest.py:
> - Allow to use the vars defined in testimage class
> - Now able to run commands in the host and dump the
> results
> - Fix an issue with the condition where to run the
> dump commands (Before it run the commands every
> test after a failure, now it runs the commands only
> in tests that failed)
> - Fix the output to stdout
>
> [YOCTO #8118]
>
> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
> ---
> meta/classes/testimage.bbclass | 9 +++++++++
> meta/lib/oeqa/oetest.py | 42
> +++++++++++++++++++++++++++--------------- meta/lib/oeqa/targetcontrol.py |
> 3 +++
> 3 files changed, 39 insertions(+), 15 deletions(-)
>
> diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
> index 140babe..1580a4e 100644
> --- a/meta/classes/testimage.bbclass
> +++ b/meta/classes/testimage.bbclass
> @@ -56,6 +56,15 @@ TESTIMAGEDEPENDS_qemuall =
> "qemu-native:do_populate_sysroot qemu-helper-native:d TESTIMAGELOCK =
> "${TMPDIR}/testimage.lock"
> TESTIMAGELOCK_qemuall = ""
>
> +TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
> +
> +python () {
> + target_cmds = ["top -bn1", "ps", "free", "df", "_ping", "dmesg",
> "netstat -an", "ip address", "_logs"] + host_cmds = ["top -bn1", "ps",
> "free", "df", "memstat", "dmesg", "netstat -an"] +
> d.setVar("TESTIMAGE_DUMP_TRG", target_cmds)
> + d.setVar("TESTIMAGE_DUMP_HST", host_cmds)
> +}
This is isn't how I would recommend doing this. What I suggested earlier when
we talked about this was:
testimage_dump_target() {
top -bn1
ps
free
df
...
}
Then you'd just d.getVar('testimage_dump_target', True), and treat the
contents as a complete shell script to be run - i.e. write it to a file with
#!/bin/sh as a prefix and run it. As a bonus, the value can be overridden or
appended to from elsewhere. (This is similar to how we deal with
pkg_postinst_<packagename> for postinstall scripts.)
> + self.dump_trg = d.getVar("TESTIMAGE_DUMP_TRG", False)
> + self.dump_hst = d.getVar("TESTIMAGE_DUMP_HST", False)
> + self.dump_dir = d.getVar("TESTIMAGE_DUMP_DIR", False)
As a general note - for this kind of thing, you need to be specifying True not
False for the expansion parameter - the value may well contain references or
other expressions (such as ${TMPDIR}) that need expanding. In fact, it's
relatively uncommon for False to be what you want - most of the time the value
should be expanded if you're about to use it for something.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] testimage: Run commands in target and host when test fails
2015-08-18 8:33 ` Paul Eggleton
@ 2015-08-18 15:18 ` Mariano Lopez
2015-08-18 15:23 ` Paul Eggleton
0 siblings, 1 reply; 4+ messages in thread
From: Mariano Lopez @ 2015-08-18 15:18 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
Hi Paul,
On 08/18/2015 03:33 AM, Paul Eggleton wrote:
> Hi Mariano,
>
> On Monday 17 August 2015 12:41:42 mariano.lopez@linux.intel.com wrote:
>> From: Mariano Lopez <mariano.lopez@linux.intel.com>
>>
>> This patch modify three files altought two of them
>> are minimal modifications:
>>
>> testimage.bbclass:
>> Create new vars for easy modification of the dump
>> directory and commands to be run on host and target
>> when a test fails
>> TESTIMAGE_DUMP_DIR: Directory to save the dumps
>> TESTIMAGE_DUMP_TRG: Commands to run on target
>> TESTIMAGE_DUMP_HST: Commands to run on host
> Can we please use full names (i.e. "...HOST" and "...TARGET")? We're not short
> of space here ;)
Sure, no problem
>
>> targetcontrol.py:
>> Extract vars from the datastore for later use
>>
>> oetest.py:
>> - Allow to use the vars defined in testimage class
>> - Now able to run commands in the host and dump the
>> results
>> - Fix an issue with the condition where to run the
>> dump commands (Before it run the commands every
>> test after a failure, now it runs the commands only
>> in tests that failed)
>> - Fix the output to stdout
>>
>> [YOCTO #8118]
>>
>> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
>> ---
>> meta/classes/testimage.bbclass | 9 +++++++++
>> meta/lib/oeqa/oetest.py | 42
>> +++++++++++++++++++++++++++--------------- meta/lib/oeqa/targetcontrol.py |
>> 3 +++
>> 3 files changed, 39 insertions(+), 15 deletions(-)
>>
>> diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass
>> index 140babe..1580a4e 100644
>> --- a/meta/classes/testimage.bbclass
>> +++ b/meta/classes/testimage.bbclass
>> @@ -56,6 +56,15 @@ TESTIMAGEDEPENDS_qemuall =
>> "qemu-native:do_populate_sysroot qemu-helper-native:d TESTIMAGELOCK =
>> "${TMPDIR}/testimage.lock"
>> TESTIMAGELOCK_qemuall = ""
>>
>> +TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
>> +
>> +python () {
>> + target_cmds = ["top -bn1", "ps", "free", "df", "_ping", "dmesg",
>> "netstat -an", "ip address", "_logs"] + host_cmds = ["top -bn1", "ps",
>> "free", "df", "memstat", "dmesg", "netstat -an"] +
>> d.setVar("TESTIMAGE_DUMP_TRG", target_cmds)
>> + d.setVar("TESTIMAGE_DUMP_HST", host_cmds)
>> +}
> This is isn't how I would recommend doing this. What I suggested earlier when
> we talked about this was:
>
> testimage_dump_target() {
> top -bn1
> ps
> free
> df
> ...
> }
>
> Then you'd just d.getVar('testimage_dump_target', True), and treat the
> contents as a complete shell script to be run - i.e. write it to a file with
> #!/bin/sh as a prefix and run it. As a bonus, the value can be overridden or
> appended to from elsewhere. (This is similar to how we deal with
> pkg_postinst_<packagename> for postinstall scripts.)
Implement this for the host wouldn't be a problem, it gets
interesting for the target.These commands would be run from
the serial console, when the network fails, so it would be
difficult to run as a shell script. And also would be difficult
to install the scripts on the image because testimage would
run after do_rootfs takes place.
What would be the best approach for the target case?
>> + self.dump_trg = d.getVar("TESTIMAGE_DUMP_TRG", False)
>> + self.dump_hst = d.getVar("TESTIMAGE_DUMP_HST", False)
>> + self.dump_dir = d.getVar("TESTIMAGE_DUMP_DIR", False)
> As a general note - for this kind of thing, you need to be specifying True not
> False for the expansion parameter - the value may well contain references or
> other expressions (such as ${TMPDIR}) that need expanding. In fact, it's
> relatively uncommon for False to be what you want - most of the time the value
> should be expanded if you're about to use it for something.
You are right, I will change that.
>
> Cheers,
> Paul
>
--
Mariano Lopez
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] testimage: Run commands in target and host when test fails
2015-08-18 15:18 ` Mariano Lopez
@ 2015-08-18 15:23 ` Paul Eggleton
0 siblings, 0 replies; 4+ messages in thread
From: Paul Eggleton @ 2015-08-18 15:23 UTC (permalink / raw)
To: Mariano Lopez; +Cc: openembedded-core
On Tuesday 18 August 2015 10:18:11 Mariano Lopez wrote:
> On 08/18/2015 03:33 AM, Paul Eggleton wrote:
> > On Monday 17 August 2015 12:41:42 mariano.lopez@linux.intel.com wrote:
> >> From: Mariano Lopez <mariano.lopez@linux.intel.com>
> >>
> >> This patch modify three files altought two of them
> >> are minimal modifications:
> >>
> >> testimage.bbclass:
> >> Create new vars for easy modification of the dump
> >>
> >> directory and commands to be run on host and target
> >> when a test fails
> >>
> >> TESTIMAGE_DUMP_DIR: Directory to save the dumps
> >> TESTIMAGE_DUMP_TRG: Commands to run on target
> >> TESTIMAGE_DUMP_HST: Commands to run on host
> >
> > Can we please use full names (i.e. "...HOST" and "...TARGET")? We're not
> > short of space here ;)
>
> Sure, no problem
>
> >> targetcontrol.py:
> >> Extract vars from the datastore for later use
> >>
> >> oetest.py:
> >> - Allow to use the vars defined in testimage class
> >> - Now able to run commands in the host and dump the
> >>
> >> results
> >>
> >> - Fix an issue with the condition where to run the
> >>
> >> dump commands (Before it run the commands every
> >> test after a failure, now it runs the commands only
> >> in tests that failed)
> >>
> >> - Fix the output to stdout
> >>
> >> [YOCTO #8118]
> >>
> >> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
> >> ---
> >>
> >> meta/classes/testimage.bbclass | 9 +++++++++
> >> meta/lib/oeqa/oetest.py | 42
> >>
> >> +++++++++++++++++++++++++++--------------- meta/lib/oeqa/targetcontrol.py
> >> |
> >>
> >> 3 +++
> >> 3 files changed, 39 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/meta/classes/testimage.bbclass
> >> b/meta/classes/testimage.bbclass index 140babe..1580a4e 100644
> >> --- a/meta/classes/testimage.bbclass
> >> +++ b/meta/classes/testimage.bbclass
> >> @@ -56,6 +56,15 @@ TESTIMAGEDEPENDS_qemuall =
> >> "qemu-native:do_populate_sysroot qemu-helper-native:d TESTIMAGELOCK =
> >> "${TMPDIR}/testimage.lock"
> >>
> >> TESTIMAGELOCK_qemuall = ""
> >>
> >> +TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
> >> +
> >> +python () {
> >> + target_cmds = ["top -bn1", "ps", "free", "df", "_ping", "dmesg",
> >> "netstat -an", "ip address", "_logs"] + host_cmds = ["top -bn1", "ps",
> >> "free", "df", "memstat", "dmesg", "netstat -an"] +
> >> d.setVar("TESTIMAGE_DUMP_TRG", target_cmds)
> >> + d.setVar("TESTIMAGE_DUMP_HST", host_cmds)
> >> +}
> >
> > This is isn't how I would recommend doing this. What I suggested earlier
> > when we talked about this was:
> >
> > testimage_dump_target() {
> >
> > top -bn1
> > ps
> > free
> > df
> > ...
> >
> > }
> >
> > Then you'd just d.getVar('testimage_dump_target', True), and treat the
> > contents as a complete shell script to be run - i.e. write it to a file
> > with #!/bin/sh as a prefix and run it. As a bonus, the value can be
> > overridden or appended to from elsewhere. (This is similar to how we deal
> > with
> > pkg_postinst_<packagename> for postinstall scripts.)
>
> Implement this for the host wouldn't be a problem, it gets
> interesting for the target.These commands would be run from
> the serial console, when the network fails, so it would be
> difficult to run as a shell script. And also would be difficult
> to install the scripts on the image because testimage would
> run after do_rootfs takes place.
>
> What would be the best approach for the target case?
Alright, in that case just split it on newline and send each command one by
one.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-08-18 15:23 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-08-17 12:41 [PATCH] testimage: Run commands in target and host when test fails mariano.lopez
2015-08-18 8:33 ` Paul Eggleton
2015-08-18 15:18 ` Mariano Lopez
2015-08-18 15:23 ` Paul Eggleton
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.