oeqa: test cases at the end of the test suite

oeqa: test cases at the end of the test suite

[Konrad Weihmann]

Hi all,

I have a few oeqa test cases, which always should run last in a test 
suite (log and file collectors for instance).
Due to the ordering of the test case discovery I had to name all those 
tests like "zzz_<name>" to move them to the end of the computed list.

Unfortunately this is very error prone, as the OETestDepends tag does 
have major influence on the ordering (just a single misplaced tag in any 
test case can make it past those "zzz" cases).
Also it makes things hard to read IMO.

That brings me to my question, is there any way to ensure that test 
cases are run at the end of the complete list of test suites?

If not does anyone have an idea how to create such a feature (maybe a 
new decorator or something like that)?

Regards
Konrad

Re: [yocto] oeqa: test cases at the end of the test suite

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1285 bytes --]

I'd say improving the actual test ordering function is the way to go, yes.
Not sure how the ordering 'hint' should be like, but maybe setting an
integer priority via a decorator would work:
@OETestPriority(1) ---> runs first, no guarantees about relative order of
other tests with priority 1.
@OETestPriority(99) ---> runs last, ditto.
Default priority: 50 or similar.

Alex

On Mon, 4 Jan 2021 at 11:51, Konrad Weihmann <kweihmann@outlook.com> wrote:

> Hi all,
>
> I have a few oeqa test cases, which always should run last in a test
> suite (log and file collectors for instance).
> Due to the ordering of the test case discovery I had to name all those
> tests like "zzz_<name>" to move them to the end of the computed list.
>
> Unfortunately this is very error prone, as the OETestDepends tag does
> have major influence on the ordering (just a single misplaced tag in any
> test case can make it past those "zzz" cases).
> Also it makes things hard to read IMO.
>
> That brings me to my question, is there any way to ensure that test
> cases are run at the end of the complete list of test suites?
>
> If not does anyone have an idea how to create such a feature (maybe a
> new decorator or something like that)?
>
> Regards
> Konrad
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 1746 bytes --]

Re: [yocto] oeqa: test cases at the end of the test suite

[Ross Burton]

On Mon, 4 Jan 2021 at 10:50, Konrad Weihmann <kweihmann@outlook.com> wrote:
> I have a few oeqa test cases, which always should run last in a test
> suite (log and file collectors for instance).

Those are not test cases then, surely.  If you want to run actions
after test cases to collect log files then I'd implement that as a
decorator on the test cases.

Ross