* [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
[not found] <cover.1465824701.git.mariano.lopez@linux.intel.com>
@ 2016-06-13 13:32 ` mariano.lopez
2016-06-14 9:26 ` Burton, Ross
2016-07-25 22:38 ` Burton, Ross
0 siblings, 2 replies; 9+ messages in thread
From: mariano.lopez @ 2016-06-13 13:32 UTC (permalink / raw)
To: openembedded-core
From: Mariano Lopez <mariano.lopez@linux.intel.com>
There are some features in testimage/testexport that are not tested;
this might lead to break some of these features without notice.
This adds a new test in order to test two features of testimage:
- Import test from other layers.
- Install/Unistall in the DUT without a package manager.
[YOCTO #9764]
[YOCTO #9766]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
---
meta-selftest/lib/oeqa/runtime/selftest.json | 6 +++++
meta-selftest/lib/oeqa/runtime/selftest.py | 31 ++++++++++++++++++++++
.../selftest/{testexport.py => runtime-test.py} | 19 +++++++++++++
3 files changed, 56 insertions(+)
create mode 100644 meta-selftest/lib/oeqa/runtime/selftest.json
create mode 100644 meta-selftest/lib/oeqa/runtime/selftest.py
rename meta/lib/oeqa/selftest/{testexport.py => runtime-test.py} (76%)
diff --git a/meta-selftest/lib/oeqa/runtime/selftest.json b/meta-selftest/lib/oeqa/runtime/selftest.json
new file mode 100644
index 0000000..e5ae46e
--- /dev/null
+++ b/meta-selftest/lib/oeqa/runtime/selftest.json
@@ -0,0 +1,6 @@
+{
+ "test_install_package": {
+ "pkg": "socat",
+ "rm": true
+ }
+}
diff --git a/meta-selftest/lib/oeqa/runtime/selftest.py b/meta-selftest/lib/oeqa/runtime/selftest.py
new file mode 100644
index 0000000..b9c9b23
--- /dev/null
+++ b/meta-selftest/lib/oeqa/runtime/selftest.py
@@ -0,0 +1,31 @@
+from oeqa.oetest import oeRuntimeTest, skipModule
+from oeqa.utils.decorators import *
+
+class Selftest(oeRuntimeTest):
+
+ @skipUnlessPassed("test_ssh")
+ def test_install_package(self):
+ """
+ Summary: Check basic package installation functionality.
+ Expected: 1. Before the test socat must be installed using scp.
+ 2. After the test socat must be unistalled using ssh.
+ This can't be checked in this test.
+ Product: oe-core
+ Author: Mariano Lopez <mariano.lopez@intel.com>
+ """
+
+ (status, output) = self.target.run("socat -V")
+ self.assertEqual(status, 0, msg="socat is not installed")
+
+ @skipUnlessPassed("test_install_package")
+ def test_verify_unistall(self):
+ """
+ Summary: Check basic package installation functionality.
+ Expected: 1. test_install_package must unistall socat.
+ This test is just to verify that.
+ Product: oe-core
+ Author: Mariano Lopez <mariano.lopez@intel.com>
+ """
+
+ (status, output) = self.target.run("socat -V")
+ self.assertNotEqual(status, 0, msg="socat is still installed")
diff --git a/meta/lib/oeqa/selftest/testexport.py b/meta/lib/oeqa/selftest/runtime-test.py
similarity index 76%
rename from meta/lib/oeqa/selftest/testexport.py
rename to meta/lib/oeqa/selftest/runtime-test.py
index 5823b13..1e1d3b0 100644
--- a/meta/lib/oeqa/selftest/testexport.py
+++ b/meta/lib/oeqa/selftest/runtime-test.py
@@ -42,3 +42,22 @@ class TestExport(oeSelfTest):
# Verify ping test was succesful
failure = True if 'FAIL' in result.output else False
self.assertNotEqual(True, failure, 'ping test failed')
+
+
+class TestImage(oeSelfTest):
+
+ def test_testimage_install(self):
+ """
+ Summary: Check install packages functionality for testimage/testexport.
+ Expected: 1. Import tests from a directory other than meta.
+ 2. Check install/unistall of socat.
+ Product: oe-core
+ Author: Mariano Lopez <mariano.lopez@intel.com>
+ """
+
+ features = 'INHERIT += "testexport"\n'
+ features = 'TEST_SUITES = "ping ssh selftest"\n'
+
+ # Build core-image-sato and testimage
+ bitbake('core-image-sato socat')
+ bitbake('-c testimage core-image-sato')
--
2.6.6
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-13 13:32 ` [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers mariano.lopez
@ 2016-06-14 9:26 ` Burton, Ross
2016-06-14 22:08 ` Mariano Lopez
2016-07-25 22:38 ` Burton, Ross
1 sibling, 1 reply; 9+ messages in thread
From: Burton, Ross @ 2016-06-14 9:26 UTC (permalink / raw)
To: Mariano Lopez; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 454 bytes --]
On 13 June 2016 at 14:32, <mariano.lopez@linux.intel.com> wrote:
> + features = 'INHERIT += "testexport"\n'
> + features = 'TEST_SUITES = "ping ssh selftest"\n'
> +
> + # Build core-image-sato and testimage
> + bitbake('core-image-sato socat')
> + bitbake('-c testimage core-image-sato')
>
If we're just running ping/ssh/selftest, can the image be something a lot
smaller such as core-image-base?
Ross
[-- Attachment #2: Type: text/html, Size: 952 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-14 9:26 ` Burton, Ross
@ 2016-06-14 22:08 ` Mariano Lopez
2016-06-15 20:47 ` Khem Raj
0 siblings, 1 reply; 9+ messages in thread
From: Mariano Lopez @ 2016-06-14 22:08 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 802 bytes --]
On 06/14/2016 04:26 AM, Burton, Ross wrote:
>
> On 13 June 2016 at 14:32, <mariano.lopez@linux.intel.com
> <mailto:mariano.lopez@linux.intel.com>> wrote:
>
> + features = 'INHERIT += "testexport"\n'
> + features = 'TEST_SUITES = "ping ssh selftest"\n'
> +
> + # Build core-image-sato and testimage
> + bitbake('core-image-sato socat')
> + bitbake('-c testimage core-image-sato')
>
>
> If we're just running ping/ssh/selftest, can the image be something a
> lot smaller such as core-image-base?
>
> Ross
Unfortunately, base or minimal don't have ssh installed, and ssh it is
required for the runtime tests.
I could add the ssh to DISTRO_FEATURES but personally I think that is
not clean. What do you think?
Mariano
[-- Attachment #2: Type: text/html, Size: 2004 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-14 22:08 ` Mariano Lopez
@ 2016-06-15 20:47 ` Khem Raj
2016-06-15 21:01 ` Mariano Lopez
0 siblings, 1 reply; 9+ messages in thread
From: Khem Raj @ 2016-06-15 20:47 UTC (permalink / raw)
To: Mariano Lopez; +Cc: OE-core
On Tue, Jun 14, 2016 at 3:08 PM, Mariano Lopez
<mariano.lopez@linux.intel.com> wrote:
>
>
> On 06/14/2016 04:26 AM, Burton, Ross wrote:
>
>
> On 13 June 2016 at 14:32, <mariano.lopez@linux.intel.com> wrote:
>>
>> + features = 'INHERIT += "testexport"\n'
>> + features = 'TEST_SUITES = "ping ssh selftest"\n'
>> +
>> + # Build core-image-sato and testimage
>> + bitbake('core-image-sato socat')
>> + bitbake('-c testimage core-image-sato')
>
>
> If we're just running ping/ssh/selftest, can the image be something a lot
> smaller such as core-image-base?
>
> Ross
>
>
> Unfortunately, base or minimal don't have ssh installed, and ssh it is
> required for the runtime tests.
> I could add the ssh to DISTRO_FEATURES but personally I think that is not
> clean. What do you think?
Can ssh be added to images automatically if testing is enabled ?
>
> Mariano
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-15 20:47 ` Khem Raj
@ 2016-06-15 21:01 ` Mariano Lopez
2016-06-15 21:05 ` Khem Raj
0 siblings, 1 reply; 9+ messages in thread
From: Mariano Lopez @ 2016-06-15 21:01 UTC (permalink / raw)
To: Khem Raj; +Cc: OE-core
On 06/15/2016 03:47 PM, Khem Raj wrote:
> On Tue, Jun 14, 2016 at 3:08 PM, Mariano Lopez
> <mariano.lopez@linux.intel.com> wrote:
>> If we're just running ping/ssh/selftest, can the image be something a lot
>> smaller such as core-image-base?
>>
>> Ross
>>
>>
>> Unfortunately, base or minimal don't have ssh installed, and ssh it is
>> required for the runtime tests.
>> I could add the ssh to DISTRO_FEATURES but personally I think that is not
>> clean. What do you think?
> Can ssh be added to images automatically if testing is enabled ?
IMO that is not a good practice, because the image being tested is
polluted with packages not meant to be there.
Mariano
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-15 21:01 ` Mariano Lopez
@ 2016-06-15 21:05 ` Khem Raj
2016-06-15 21:16 ` Mariano Lopez
0 siblings, 1 reply; 9+ messages in thread
From: Khem Raj @ 2016-06-15 21:05 UTC (permalink / raw)
To: Mariano Lopez; +Cc: OE-core
On Wed, Jun 15, 2016 at 2:01 PM, Mariano Lopez
<mariano.lopez@linux.intel.com> wrote:
>
>
> On 06/15/2016 03:47 PM, Khem Raj wrote:
>>
>> On Tue, Jun 14, 2016 at 3:08 PM, Mariano Lopez
>> <mariano.lopez@linux.intel.com> wrote:
>>>
>>> If we're just running ping/ssh/selftest, can the image be something a lot
>>> smaller such as core-image-base?
>>>
>>> Ross
>>>
>>>
>>> Unfortunately, base or minimal don't have ssh installed, and ssh it is
>>> required for the runtime tests.
>>> I could add the ssh to DISTRO_FEATURES but personally I think that is not
>>> clean. What do you think?
>>
>> Can ssh be added to images automatically if testing is enabled ?
>
>
> IMO that is not a good practice, because the image being tested is polluted
> with packages not meant to be there.
I thought you mentioned that ssh is required for testing, I dont see
that as polluting.
and its only added to image with have ptest turned on.
>
> Mariano
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-15 21:05 ` Khem Raj
@ 2016-06-15 21:16 ` Mariano Lopez
0 siblings, 0 replies; 9+ messages in thread
From: Mariano Lopez @ 2016-06-15 21:16 UTC (permalink / raw)
To: Khem Raj; +Cc: OE-core
On 06/15/2016 04:05 PM, Khem Raj wrote:
> On Wed, Jun 15, 2016 at 2:01 PM, Mariano Lopez
> <mariano.lopez@linux.intel.com> wrote:
>>
>> On 06/15/2016 03:47 PM, Khem Raj wrote:
>>> On Tue, Jun 14, 2016 at 3:08 PM, Mariano Lopez
>>> <mariano.lopez@linux.intel.com> wrote:
>>>> If we're just running ping/ssh/selftest, can the image be something a lot
>>>> smaller such as core-image-base?
>>>>
>>>> Ross
>>>>
>>>>
>>>> Unfortunately, base or minimal don't have ssh installed, and ssh it is
>>>> required for the runtime tests.
>>>> I could add the ssh to DISTRO_FEATURES but personally I think that is not
>>>> clean. What do you think?
>>> Can ssh be added to images automatically if testing is enabled ?
>>
>> IMO that is not a good practice, because the image being tested is polluted
>> with packages not meant to be there.
> I thought you mentioned that ssh is required for testing, I dont see
> that as polluting.
> and its only added to image with have ptest turned on.
This is for testimage/testexport and there is a TEST_SUITE specific for
minimal image.
If the resource consumption seems to be the problem; adding ssh to
minimal or base would trigger the build of a new rootfs/image; in the
other hand is very likely core-image-sato is already built when running
this test.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-06-13 13:32 ` [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers mariano.lopez
2016-06-14 9:26 ` Burton, Ross
@ 2016-07-25 22:38 ` Burton, Ross
2016-07-26 20:24 ` Mariano Lopez
1 sibling, 1 reply; 9+ messages in thread
From: Burton, Ross @ 2016-07-25 22:38 UTC (permalink / raw)
To: Mariano Lopez; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 720 bytes --]
On 13 June 2016 at 14:32, <mariano.lopez@linux.intel.com> wrote:
> + features = 'INHERIT += "testexport"\n'
>
You mean testimage here as that class isn't inherited by default.
> + features = 'TEST_SUITES = "ping ssh selftest"\n'
>
This overwrites the first assignment to features, and neither is actually
written using self.write_config().
> + # Build core-image-sato and testimage
> + bitbake('core-image-sato socat')
> + bitbake('-c testimage core-image-sato')
>
I personally believe that core-image-sato is massive overkill for selftest
and core-image-full-cmdline is just as comprehensive (has ssh, for example)
without building all of Sato.
Ross
[-- Attachment #2: Type: text/html, Size: 1612 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers
2016-07-25 22:38 ` Burton, Ross
@ 2016-07-26 20:24 ` Mariano Lopez
0 siblings, 0 replies; 9+ messages in thread
From: Mariano Lopez @ 2016-07-26 20:24 UTC (permalink / raw)
To: Burton, Ross; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]
On 07/25/2016 05:38 PM, Burton, Ross wrote:
>
> On 13 June 2016 at 14:32, <mariano.lopez@linux.intel.com
> <mailto:mariano.lopez@linux.intel.com>> wrote:
>
> + features = 'INHERIT += "testexport"\n'
>
>
> You mean testimage here as that class isn't inherited by default.
>
> + features = 'TEST_SUITES = "ping ssh selftest"\n'
>
>
> This overwrites the first assignment to features, and neither is
> actually written using self.write_config().
>
> + # Build core-image-sato and testimage
> + bitbake('core-image-sato socat')
> + bitbake('-c testimage core-image-sato')
>
>
> I personally believe that core-image-sato is massive overkill for
> selftest and core-image-full-cmdline is just as comprehensive (has
> ssh, for example) without building all of Sato.
I don't know what happened when I made this commit and submitted the
patch, this patch is just wrong. I swear I tested it locally, perhaps I
messed up the branches or something like that. I'll work on a v2 of this.
Mariano
[-- Attachment #2: Type: text/html, Size: 2855 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-07-26 20:24 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <cover.1465824701.git.mariano.lopez@linux.intel.com>
2016-06-13 13:32 ` [PATCH 1/1] selftest/runtime-test.py: Add test for import test from other layers mariano.lopez
2016-06-14 9:26 ` Burton, Ross
2016-06-14 22:08 ` Mariano Lopez
2016-06-15 20:47 ` Khem Raj
2016-06-15 21:01 ` Mariano Lopez
2016-06-15 21:05 ` Khem Raj
2016-06-15 21:16 ` Mariano Lopez
2016-07-25 22:38 ` Burton, Ross
2016-07-26 20:24 ` Mariano Lopez
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.