[PATCH v2 1/3] oeqa: add OETestPriority decorator

[PATCH v2 1/3] oeqa: add OETestPriority decorator

[Konrad Weihmann]

This modifies an implicit default priority.
Can be used to move the test case either to the front
or to the end of the test suite.
Value can be between 0 and 99, implicit default is 50.
Lower values move are run at the beginning of the list,
higher values at the end.
Within a single priority ordering remains unchanged.

It enables use cases such as shutdown tests, which
should always run at the end of the overall test suite,
no matter what test suites/cases are set in the current
test suite list.

This avoids also using test suite naming hacks, such as
usage of zzz_foo_test.FooClass.test_foo to move it to
the end of the test suite list.

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
---
 meta/lib/oeqa/core/decorator/priority.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 meta/lib/oeqa/core/decorator/priority.py

diff --git a/meta/lib/oeqa/core/decorator/priority.py b/meta/lib/oeqa/core/decorator/priority.py
new file mode 100644
index 0000000000..bca4675db0
--- /dev/null
+++ b/meta/lib/oeqa/core/decorator/priority.py
@@ -0,0 +1,17 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+from . import OETestDecorator, registerDecorator
+from oeqa.core.exception import OEQADependency
+
+@registerDecorator
+class OETestPriority(OETestDecorator):
+    attrs = ('priority',)
+
+    def bind(self, registry, case):
+        if not isinstance(self.priority, int) or self.priority < 0 or self.priority > 99:
+            raise OEQADependency("OETestPriority must be an integer value between 0-99")
+        self.case = case
+        self.logger = case.tc.logger
+        self.case.decorators.append(self)        
-- 
2.25.1

Re: [OE-core] [PATCH v2 1/3] oeqa: add OETestPriority decorator

[Richard Purdie]

On Tue, 2021-01-26 at 19:01 +0100, Konrad Weihmann wrote:
> This modifies an implicit default priority.
> Can be used to move the test case either to the front
> or to the end of the test suite.
> Value can be between 0 and 99, implicit default is 50.
> Lower values move are run at the beginning of the list,
> higher values at the end.
> Within a single priority ordering remains unchanged.
> 
> It enables use cases such as shutdown tests, which
> should always run at the end of the overall test suite,
> no matter what test suites/cases are set in the current
> test suite list.
> 
> This avoids also using test suite naming hacks, such as
> usage of zzz_foo_test.FooClass.test_foo to move it to
> the end of the test suite list.
> 
> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> ---
>  meta/lib/oeqa/core/decorator/priority.py | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>  create mode 100644 meta/lib/oeqa/core/decorator/priority.py

I'm still not convinced this is a great idea as tests should be
independent and if we need pre/post test hooks, we should have those
separately. Its also complex when it comes to parallelism, when should
the tests run in that context.

I did try testing these but it blows up too:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1786

The more I think about it, the more I'm convinced its a bad idea.

Cheers,

Richard