Executing ptest using image test ?

Executing ptest using image test ?

[Koehler, Yannick]

[-- Attachment #1: Type: text/plain, Size: 376 bytes --]

I have some ptest which I would like to run quickly inside qemux86.  I see that the TEST_SUITES would allow me to do that, and that there is a _ptest.py script which seems to interface with ptest-runner.  Yet, I am unclear how it connects together if it does.


If anyone know how this can be done, can you point me on some doc related to this?


--

Yannick Koehler

[-- Attachment #2: Type: text/html, Size: 798 bytes --]

Re: Executing ptest using image test ?

[Alexander Kanavin]

On 11/24/2017 04:07 AM, Koehler, Yannick wrote:
> I have some ptest which I would like to run quickly inside qemux86.  I 
> see that the TEST_SUITES would allow me to do that, and that there is a 
> _ptest.py script which seems to interface with ptest-runner.  Yet, I am 
> unclear how it connects together if it does.
> 
> 
> If anyone know how this can be done, can you point me on some doc 
> related to this?

Here:
http://www.yoctoproject.org/docs/2.4/dev-manual/dev-manual.html#testing-packages-with-ptest


Alex

Re: Executing ptest using image test ?

[Koehler, Yannick]

Hi Alexander,

  I had pretty much done all that is documented under 4.21 of this documentation.  I can execute my test within the image using "ptest_runner" and I see it PASS/FAIL according to my test's content.  So that's great, but the problem with this is still that I need to manually invoke qemu and initiate the ptest-runner script.  I would like that to occurs from within the testimage using Image test as describe under https://wiki.yoctoproject.org/wiki/Image_tests.  Yet it seems that to get image test support I need to write additional python test scripts which I feel is excessive and since those scripts are not store within the recipe folder, I feel those test are more MACHINE testing than package testing.

  I would like to use 1 python test to execute and report the result of one or more ptest, in the same automatic way image test does.  I cannot see or connect the dot between ptest and image test by reading and re-reading section 4.21.

--
Yannick Koehler

-----Message d'origine-----
De : Alexander Kanavin [mailto:alexander.kanavin@linux.intel.com] 
Envoyé : 24 novembre 2017 07:48
À : Koehler, Yannick <yannick.koehler@hpe.com>; yocto@yoctoproject.org
Objet : Re: [yocto] Executing ptest using image test ?

On 11/24/2017 04:07 AM, Koehler, Yannick wrote:
> I have some ptest which I would like to run quickly inside qemux86.  I 
> see that the TEST_SUITES would allow me to do that, and that there is 
> a _ptest.py script which seems to interface with ptest-runner.  Yet, I 
> am unclear how it connects together if it does.
> 
> 
> If anyone know how this can be done, can you point me on some doc 
> related to this?

Here:
http://www.yoctoproject.org/docs/2.4/dev-manual/dev-manual.html#testing-packages-with-ptest


Alex

Re: Executing ptest using image test ?

[Alexander Kanavin]

On 11/24/2017 03:59 PM, Koehler, Yannick wrote:

> I had pretty much done all that is documented under 4.21 of this
> documentation.  I can execute my test within the image using
> "ptest_runner" and I see it PASS/FAIL according to my test's content.
> So that's great, but the problem with this is still that I need to
> manually invoke qemu and initiate the ptest-runner script.  I would
> like that to occurs from within the testimage using Image test as
> describe under https://wiki.yoctoproject.org/wiki/Image_tests.  Yet
> it seems that to get image test support I need to write additional
> python test scripts which I feel is excessive and since those scripts
> are not store within the recipe folder, I feel those test are more
> MACHINE testing than package testing.
> 
> I would like to use 1 python test to execute and report the result of
> one or more ptest, in the same automatic way image test does.  I
> cannot see or connect the dot between ptest and image test by reading
> and re-reading section 4.21.

Right, I see. The unattended runtime execution of ptests should be 
performed by meta/lib/oeqa/runtime/cases/ptest.py, I don't at the moment 
remember how the pieces fit together exactly so that it runs, but I 
think if you follow what it says [1] and run bitbake -c testimage 
<image> then it should work.

[1]
     @skipIfNotFeature('ptest', 'Test requires ptest to be in 
DISTRO_FEATURES')
     @skipIfNotFeature('ptest-pkgs', 'Test requires ptest-pkgs to be in 
IMAGE_FEATURES')


Alex

Re: Executing ptest using image test ?

[Koehler, Yannick]

Ha!

I was not using -c testimage, and I also think the DISTRO_FEATURE may be incorrectly set in my case, I will try that, thanks for the pointer.

--
Yannick Koehler

-----Message d'origine-----
De : Alexander Kanavin [mailto:alexander.kanavin@linux.intel.com] 
Envoyé : 24 novembre 2017 09:20
À : Koehler, Yannick <yannick.koehler@hpe.com>; yocto@yoctoproject.org
Objet : Re: [yocto] Executing ptest using image test ?

On 11/24/2017 03:59 PM, Koehler, Yannick wrote:

> I had pretty much done all that is documented under 4.21 of this 
> documentation.  I can execute my test within the image using 
> "ptest_runner" and I see it PASS/FAIL according to my test's content.
> So that's great, but the problem with this is still that I need to 
> manually invoke qemu and initiate the ptest-runner script.  I would 
> like that to occurs from within the testimage using Image test as 
> describe under https://wiki.yoctoproject.org/wiki/Image_tests.  Yet it 
> seems that to get image test support I need to write additional python 
> test scripts which I feel is excessive and since those scripts are not 
> store within the recipe folder, I feel those test are more MACHINE 
> testing than package testing.
> 
> I would like to use 1 python test to execute and report the result of 
> one or more ptest, in the same automatic way image test does.  I 
> cannot see or connect the dot between ptest and image test by reading 
> and re-reading section 4.21.

Right, I see. The unattended runtime execution of ptests should be performed by meta/lib/oeqa/runtime/cases/ptest.py, I don't at the moment remember how the pieces fit together exactly so that it runs, but I think if you follow what it says [1] and run bitbake -c testimage <image> then it should work.

[1]
     @skipIfNotFeature('ptest', 'Test requires ptest to be in 
DISTRO_FEATURES')
     @skipIfNotFeature('ptest-pkgs', 'Test requires ptest-pkgs to be in 
IMAGE_FEATURES')


Alex

Re: Executing ptest using image test ?

[Alexander Kanavin]

On 11/24/2017 04:30 PM, Koehler, Yannick wrote:
> I was not using -c testimage, and I also think the DISTRO_FEATURE may be incorrectly set in my case, I will try that, thanks for the pointer.

Do read the comments in testimage.bbclass, as they explain how to use 
it. Particularly, seems like ptests is actually not added to the list of 
runtime tests to run by default, so you need to do that in your 
local.conf (or in the image recipe). A lot of hoops to jump through, I 
know, but at least it's cleanly structured.

Alex