[PATCH] libinput: less parallism to increase chances the test suite works

[PATCH] libinput: less parallism to increase chances the test suite works

[Ross Burton]

Discussion with the libinput maintainer has confirmed that many of the
libinput test cases need to run on a relatively unloaded system,
because input handling is by definition time-sensitive and any
event mismatches are considered a failure to avoid broken code hiding.

We can't expect libinput to run on an otherside unloaded machine
because the autobuilder ptests execute in a qemu at the same time
as builds can be happening.  We can however reduce the amount of
parallellism happening in the libinput test suite to give it a better
chance of succeeding.

This may not be enough, but it's worth a go.

[ YOCTO #14164 ]

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/recipes-graphics/wayland/libinput/run-ptest | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-graphics/wayland/libinput/run-ptest b/meta/recipes-graphics/wayland/libinput/run-ptest
index 5a84c568b9..d11e6eb25b 100644
--- a/meta/recipes-graphics/wayland/libinput/run-ptest
+++ b/meta/recipes-graphics/wayland/libinput/run-ptest
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-/usr/libexec/libinput/libinput-test-suite
+/usr/libexec/libinput/libinput-test-suite -j1
 if [ $? -eq 0 ]; then
   echo 'PASS: libinput-test-suite'
 else
-- 
2.25.1

Re: [OE-core] [PATCH] libinput: less parallism to increase chances the test suite works

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

I have to note that x86 ptest is not affected and runs fine, so this may
not have been fully researched. Could it be specific to arm?

Alex

On Wed 3. Mar 2021 at 22.33, Ross Burton <ross@burtonini.com> wrote:

> Discussion with the libinput maintainer has confirmed that many of the
> libinput test cases need to run on a relatively unloaded system,
> because input handling is by definition time-sensitive and any
> event mismatches are considered a failure to avoid broken code hiding.
>
> We can't expect libinput to run on an otherside unloaded machine
> because the autobuilder ptests execute in a qemu at the same time
> as builds can be happening.  We can however reduce the amount of
> parallellism happening in the libinput test suite to give it a better
> chance of succeeding.
>
> This may not be enough, but it's worth a go.
>
> [ YOCTO #14164 ]
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/recipes-graphics/wayland/libinput/run-ptest | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/recipes-graphics/wayland/libinput/run-ptest
> b/meta/recipes-graphics/wayland/libinput/run-ptest
> index 5a84c568b9..d11e6eb25b 100644
> --- a/meta/recipes-graphics/wayland/libinput/run-ptest
> +++ b/meta/recipes-graphics/wayland/libinput/run-ptest
> @@ -1,6 +1,6 @@
>  #!/bin/sh
>
> -/usr/libexec/libinput/libinput-test-suite
> +/usr/libexec/libinput/libinput-test-suite -j1
>  if [ $? -eq 0 ]; then
>    echo 'PASS: libinput-test-suite'
>  else
> --
> 2.25.1
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 2127 bytes --]

Re: [OE-core] [PATCH] libinput: less parallism to increase chances the test suite works

[Richard Purdie]

On Wed, 2021-03-03 at 22:56 +0100, Alexander Kanavin wrote:
> I have to note that x86 ptest is not affected and runs fine, so this 
> may not have been fully researched. Could it be specific to arm?

We've seen this on both x86 and arm, I think the arm worker is more 
consistently loaded and hence we see it more frequently there.

I'm willing to test this approach, see if it reduces the number of
failures we see...

Cheers,

Richard

Re: [OE-core] [PATCH] libinput: less parallism to increase chances the test suite works

[Ross Burton]

On Wed, 3 Mar 2021 at 22:00, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> > I have to note that x86 ptest is not affected and runs fine, so this
> > may not have been fully researched. Could it be specific to arm?
>
> We've seen this on both x86 and arm, I think the arm worker is more
> consistently loaded and hence we see it more frequently there.

Out of all the failures in 2021, 25% were on x86. Considering the x86
build machines are large Xeons and the Arm works are
previous-generation eMAGs, that isn't bad.

> I'm willing to test this approach, see if it reduces the number of
> failures we see...

It didn't reduce it a lot.

I retract this and have sent a brute force approach of not running the
test suite. It's ran in upstream CI on every commit and isn't really
an integration test suite anyway, so we're not losing much.

Ross