Re: Re: [PATCH 5.10 000/578] 5.10.80-rc2 review

Re: Re: [PATCH 5.10 000/578] 5.10.80-rc2 review

[Tim Lewis]

> No regressions on arm64, arm, x86_64, and i386.

I got
proc-uptime-001: proc-uptime-001.c:39: main: Assertion `i1 >= i0' failed.

I don't see proc-uptime-001 on
https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/skipfile-lkft.yaml

my proc-uptime-001 history
5.10.80-rc2-dirty:not ok 10 selftests: proc: proc-uptime-001 # exit=134
5.10.80-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.79-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.78-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.77-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.76-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.75-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.74-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.73-rc3-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.73-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.72-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.71-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.71-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.70-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.70-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.69-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.69-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.68-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.67-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.65-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.63-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.62-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.60-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.59-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.58-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.57-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.56-rc1-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.54-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.53-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.52-rc2-dirty:ok 10 selftests: proc: proc-uptime-001
5.10.51-rc1-dirty:ok 10 selftests: proc: proc-uptime-001

Re: Re: [PATCH 5.10 000/578] 5.10.80-rc2 review

[Naresh Kamboju]

+ Kernel Selftest
+ Anders

Hi Tim,

Thanks for your email.

On Wed, 17 Nov 2021 at 20:07, Tim Lewis <elatllat@gmail.com> wrote:
>
> > No regressions on arm64, arm, x86_64, and i386.
>
> I got
> proc-uptime-001: proc-uptime-001.c:39: main: Assertion `i1 >= i0' failed.

It is a known intermittent failure due to test running more than expected time
and runner script killed it.

I have noticed intermittent failures on slow devices.

You can see the history of the test case on Linux next here
intermittently failing.
I do compare between the stable-rc branches, Linux mainline and next.
https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20210924/testrun/5897899/suite/kselftest-proc/test/proc.proc-uptime-001/history/


> I don't see proc-uptime-001 on
> https://github.com/Linaro/test-definitions/blob/master/automated/linux/kselftest/skipfile-lkft.yaml

We will add this as known intermittent failure.
It would be great if we report this to the test author and ask them to
review the test case for
the reason for long run time on slow devices.

>
> my proc-uptime-001 history

In general when a test fails,
Please re-run the test independently for 10 times or more on the same
kernel / device before we report it as regression.

> 5.10.80-rc2-dirty:not ok 10 selftests: proc: proc-uptime-001 # exit=134

exit=134 which means Aborted.
When the test runs more than X time (45 sec i guess) the script will
be killed by the runner script.

> 5.10.80-rc1-dirty:ok 10 selftests: proc: proc-uptime-001

This test log details gives more insight that the test was timeout and Aborted.
Test output log:
--------------------
# selftests: proc: proc-uptime-001
[   43.200262] audit: type=1701 audit(1618432600.255:6):
auid=4294967295 uid=0 gid=0 ses=4294967295 pid=11758
comm=\"proc-uptime-001\"
exe=\"/opt/kselftest_intree/proc/proc-uptime-001\" sig=6 res=1
# proc-uptime-001: proc-uptime-001.c:39: main: Assertion `i1 >= i0' failed.
# /usr/bin/tim[   43.224097] audit: type=1701 audit(1618432600.259:7):
auid=4294967295 uid=0 gid=0 ses=4294967295 pid=11756 comm=\"timeout\"
exe=\"/usr/bin/timeout.coreutils\" sig=6 res=1
eout: the monitored command dumped core
# ./kselftest/runner.sh: line 33: 11756 Aborted
/usr/bin/timeout --foreground \"$kselftest_timeout\" \"$1\"
not ok 11 selftests: proc: proc-uptime-001 # exit=134

However, It is good to find that system running slowly.

- Naresh

Re: Re: [PATCH 5.10 000/578] 5.10.80-rc2 review

[Tim Lewis]

On Thu, Nov 18, 2021 at 5:41 AM Naresh Kamboju
<naresh.kamboju@linaro.org> wrote:
...
> I have noticed intermittent failures on slow devices.
...
> When the test runs more than X time (45 sec i guess) the script will
> be killed by the runner script.

In my test environment proc-uptime-001 seems to be passing ~40% (N=10)
of the time,
and taking approximately 0.60 seconds  (~11 seconds / 18 tests).

kselftest is not timing individual targets (maybe it should?),
so I don't have a timing history but it used to pass 100% (N=60) of the time.

> We will add this as known intermittent failure.

Thanks, I'll remove it from my tests.

Data for the numbers above:

for X in $(seq 1 10) ; do echo $X && time make TARGETS="proc"
kselftest | grep -P "ok.*proc-uptime-001" ; done
1
ok 10 selftests: proc: proc-uptime-001

real    0m10.605s
user    0m3.427s
sys    0m7.239s
2
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m10.808s
user    0m3.237s
sys    0m6.614s
3
ok 10 selftests: proc: proc-uptime-001

real    0m10.577s
user    0m3.377s
sys    0m7.269s
4
ok 10 selftests: proc: proc-uptime-001

real    0m12.424s
user    0m3.215s
sys    0m7.402s
5
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m11.101s
user    0m3.257s
sys    0m6.883s
6
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m10.797s
user    0m3.199s
sys    0m6.671s
7
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m12.817s
user    0m3.308s
sys    0m7.177s
8
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m10.816s
user    0m3.201s
sys    0m6.663s
9
not ok 10 selftests: proc: proc-uptime-001 # exit=134

real    0m10.832s
user    0m3.145s
sys    0m6.721s
10
ok 10 selftests: proc: proc-uptime-001

real    0m10.664s
user    0m3.337s
sys    0m7.375s