* next-20190510 kselftest results
@ 2019-05-11 10:14 ` ci_notify
0 siblings, 0 replies; 10+ messages in thread
From: ci_notify @ 2019-05-11 10:14 UTC (permalink / raw)
Summary
------------------------------------------------------------------------
kernel: 5.1.0
git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
git branch: master
git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
git describe: next-20190510
Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
Regressions (compared to build next-20190509)
------------------------------------------------------------------------
No regressions
Fixes (compared to build next-20190509)
------------------------------------------------------------------------
No fixes
In total:
------------------------------------------------------------------------
Ran 12 total tests in the following environments and test suites.
pass 11
fail 1
xfail 0
skip 0
Environments
--------------
- dragonboard-410c - arm64
- hi6220-hikey - arm64
- i386
- juno-r2 - arm64
- qemu_arm
- qemu_arm64
- qemu_i386
- qemu_x86_64
- x15 - arm
- x86_64
Test Suites
-----------
* boot-lkft-kselftests-master-519
* boot-lkft-kselftests-vsyscall-mode-native-master-519
* boot-lkft-kselftests-vsyscall-mode-none-master-519
Failures
------------------------------------------------------------------------
qemu_x86_64:
qemu_arm64:
qemu_i386:
x15:
qemu_arm:
x86:
i386:
hi6220-hikey:
juno-r2:
dragonboard-410c:
* boot-lkft-kselftests-master-519/dragonboard-410c
Skips
------------------------------------------------------------------------
No skips
--
Linaro LKFT
https://lkft.linaro.org
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-11 10:14 ` ci_notify
0 siblings, 0 replies; 10+ messages in thread
From: ci_notify @ 2019-05-11 10:14 UTC (permalink / raw)
Summary
------------------------------------------------------------------------
kernel: 5.1.0
git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
git branch: master
git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
git describe: next-20190510
Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
Regressions (compared to build next-20190509)
------------------------------------------------------------------------
No regressions
Fixes (compared to build next-20190509)
------------------------------------------------------------------------
No fixes
In total:
------------------------------------------------------------------------
Ran 12 total tests in the following environments and test suites.
pass 11
fail 1
xfail 0
skip 0
Environments
--------------
- dragonboard-410c - arm64
- hi6220-hikey - arm64
- i386
- juno-r2 - arm64
- qemu_arm
- qemu_arm64
- qemu_i386
- qemu_x86_64
- x15 - arm
- x86_64
Test Suites
-----------
* boot-lkft-kselftests-master-519
* boot-lkft-kselftests-vsyscall-mode-native-master-519
* boot-lkft-kselftests-vsyscall-mode-none-master-519
Failures
------------------------------------------------------------------------
qemu_x86_64:
qemu_arm64:
qemu_i386:
x15:
qemu_arm:
x86:
i386:
hi6220-hikey:
juno-r2:
dragonboard-410c:
* boot-lkft-kselftests-master-519/dragonboard-410c
Skips
------------------------------------------------------------------------
No skips
--
Linaro LKFT
https://lkft.linaro.org
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-11 14:51 ` Kees Cook
0 siblings, 0 replies; 10+ messages in thread
From: keescook @ 2019-05-11 14:51 UTC (permalink / raw)
On Sat, May 11, 2019 at 10:14:57AM +0000, ci_notify at linaro.org wrote:
> Summary
> ------------------------------------------------------------------------
> kernel: 5.1.0
> git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> git branch: master
> git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
> git describe: next-20190510
> Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
>
> Regressions (compared to build next-20190509)
> ------------------------------------------------------------------------
> No regressions
>
> Fixes (compared to build next-20190509)
> ------------------------------------------------------------------------
> No fixes
>
> In total:
> ------------------------------------------------------------------------
> Ran 12 total tests in the following environments and test suites.
> pass 11
> fail 1
> xfail 0
> skip 0
>
> Environments
> --------------
> - dragonboard-410c - arm64
> - hi6220-hikey - arm64
> - i386
> - juno-r2 - arm64
> - qemu_arm
> - qemu_arm64
> - qemu_i386
> - qemu_x86_64
> - x15 - arm
> - x86_64
>
> Test Suites
> -----------
> * boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
$ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun/716064/log
$ grep '^ok ' log | wc -l
60
$ grep '^not ok' log | grep -iv '# SKIP' | wc -l
30
$ grep -i '^not ok.*# SKIP' log | wc -l
18
60 pass, 30 fail, 18 skip
--
Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-11 14:51 ` Kees Cook
0 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2019-05-11 14:51 UTC (permalink / raw)
On Sat, May 11, 2019@10:14:57AM +0000, ci_notify@linaro.org wrote:
> Summary
> ------------------------------------------------------------------------
> kernel: 5.1.0
> git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> git branch: master
> git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
> git describe: next-20190510
> Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
>
> Regressions (compared to build next-20190509)
> ------------------------------------------------------------------------
> No regressions
>
> Fixes (compared to build next-20190509)
> ------------------------------------------------------------------------
> No fixes
>
> In total:
> ------------------------------------------------------------------------
> Ran 12 total tests in the following environments and test suites.
> pass 11
> fail 1
> xfail 0
> skip 0
>
> Environments
> --------------
> - dragonboard-410c - arm64
> - hi6220-hikey - arm64
> - i386
> - juno-r2 - arm64
> - qemu_arm
> - qemu_arm64
> - qemu_i386
> - qemu_x86_64
> - x15 - arm
> - x86_64
>
> Test Suites
> -----------
> * boot-lkft-kselftests-master-519
What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
$ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun/716064/log
$ grep '^ok ' log | wc -l
60
$ grep '^not ok' log | grep -iv '# SKIP' | wc -l
30
$ grep -i '^not ok.*# SKIP' log | wc -l
18
60 pass, 30 fail, 18 skip
--
Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 15:28 ` Anders Roxell
0 siblings, 0 replies; 10+ messages in thread
From: anders.roxell @ 2019-05-14 15:28 UTC (permalink / raw)
On Sat, 11 May 2019 at 16:51, Kees Cook <keescook at chromium.org> wrote:
>
> On Sat, May 11, 2019 at 10:14:57AM +0000, ci_notify at linaro.org wrote:
> > Summary
> > ------------------------------------------------------------------------
> > kernel: 5.1.0
> > git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > git branch: master
> > git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
> > git describe: next-20190510
> > Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
> >
> > Regressions (compared to build next-20190509)
> > ------------------------------------------------------------------------
> > No regressions
> >
> > Fixes (compared to build next-20190509)
> > ------------------------------------------------------------------------
> > No fixes
> >
> > In total:
> > ------------------------------------------------------------------------
> > Ran 12 total tests in the following environments and test suites.
> > pass 11
> > fail 1
> > xfail 0
> > skip 0
> >
> > Environments
> > --------------
> > - dragonboard-410c - arm64
> > - hi6220-hikey - arm64
> > - i386
> > - juno-r2 - arm64
> > - qemu_arm
> > - qemu_arm64
> > - qemu_i386
> > - qemu_x86_64
> > - x15 - arm
> > - x86_64
> >
> > Test Suites
> > -----------
> > * boot-lkft-kselftests-master-519
>
> What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
The boot is passed when the auto-login-action in suite 'lava' is passed [1].
There's is two issues:
1. We had a way that changed the run_kselftest.sh file in order to
skip tests that hanged (that results in no test output from the run)
various boards. Since the new change to kselftest, the patch to
run_kselftest.sh got added that breaks our way to skip the tests and
kernel hangs.
2. The way we parse the output doesn't match the new output format.
We will turn off the "next-* kselftest restults" until we have
resolved this issue.
Cheers,
Anders
>
> $ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun/716064/log
> $ grep '^ok ' log | wc -l
> 60
> $ grep '^not ok' log | grep -iv '# SKIP' | wc -l
> 30
> $ grep -i '^not ok.*# SKIP' log | wc -l
> 18
>
> 60 pass, 30 fail, 18 skip
>
[1] https://lkft.validation.linaro.org/results/testcase/141157606
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 15:28 ` Anders Roxell
0 siblings, 0 replies; 10+ messages in thread
From: Anders Roxell @ 2019-05-14 15:28 UTC (permalink / raw)
On Sat, 11 May 2019@16:51, Kees Cook <keescook@chromium.org> wrote:
>
> On Sat, May 11, 2019@10:14:57AM +0000, ci_notify@linaro.org wrote:
> > Summary
> > ------------------------------------------------------------------------
> > kernel: 5.1.0
> > git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > git branch: master
> > git commit: a802303934b3bd4df6e2fc8bf2e4ebced1c37556
> > git describe: next-20190510
> > Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510
> >
> > Regressions (compared to build next-20190509)
> > ------------------------------------------------------------------------
> > No regressions
> >
> > Fixes (compared to build next-20190509)
> > ------------------------------------------------------------------------
> > No fixes
> >
> > In total:
> > ------------------------------------------------------------------------
> > Ran 12 total tests in the following environments and test suites.
> > pass 11
> > fail 1
> > xfail 0
> > skip 0
> >
> > Environments
> > --------------
> > - dragonboard-410c - arm64
> > - hi6220-hikey - arm64
> > - i386
> > - juno-r2 - arm64
> > - qemu_arm
> > - qemu_arm64
> > - qemu_i386
> > - qemu_x86_64
> > - x15 - arm
> > - x86_64
> >
> > Test Suites
> > -----------
> > * boot-lkft-kselftests-master-519
>
> What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
The boot is passed when the auto-login-action in suite 'lava' is passed [1].
There's is two issues:
1. We had a way that changed the run_kselftest.sh file in order to
skip tests that hanged (that results in no test output from the run)
various boards. Since the new change to kselftest, the patch to
run_kselftest.sh got added that breaks our way to skip the tests and
kernel hangs.
2. The way we parse the output doesn't match the new output format.
We will turn off the "next-* kselftest restults" until we have
resolved this issue.
Cheers,
Anders
>
> $ wget https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20190510/testrun/716064/log
> $ grep '^ok ' log | wc -l
> 60
> $ grep '^not ok' log | grep -iv '# SKIP' | wc -l
> 30
> $ grep -i '^not ok.*# SKIP' log | wc -l
> 18
>
> 60 pass, 30 fail, 18 skip
>
[1] https://lkft.validation.linaro.org/results/testcase/141157606
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 15:46 ` Kees Cook
0 siblings, 0 replies; 10+ messages in thread
From: keescook @ 2019-05-14 15:46 UTC (permalink / raw)
On Tue, May 14, 2019 at 05:28:45PM +0200, Anders Roxell wrote:
> On Sat, 11 May 2019 at 16:51, Kees Cook <keescook at chromium.org> wrote:
> > > Test Suites
> > > -----------
> > > * boot-lkft-kselftests-master-519
> >
> > What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
>
> The boot is passed when the auto-login-action in suite 'lava' is passed [1].
>
> There's is two issues:
> 1. We had a way that changed the run_kselftest.sh file in order to
> skip tests that hanged (that results in no test output from the run)
> various boards. Since the new change to kselftest, the patch to
> run_kselftest.sh got added that breaks our way to skip the tests and
> kernel hangs.
There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now:
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?id=a745f7af3cbd04b9c9c9e803429e1c20775b538d
> 2. The way we parse the output doesn't match the new output format.
>
> We will turn off the "next-* kselftest restults" until we have
> resolved this issue.
I'd prefer to keep the tests running -- they're still valuable logs to
have. I was just curious about the specific reporting. As long as the
issue is known, that's fine. No need to disable it entirely. :)
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 15:46 ` Kees Cook
0 siblings, 0 replies; 10+ messages in thread
From: Kees Cook @ 2019-05-14 15:46 UTC (permalink / raw)
On Tue, May 14, 2019@05:28:45PM +0200, Anders Roxell wrote:
> On Sat, 11 May 2019@16:51, Kees Cook <keescook@chromium.org> wrote:
> > > Test Suites
> > > -----------
> > > * boot-lkft-kselftests-master-519
> >
> > What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
>
> The boot is passed when the auto-login-action in suite 'lava' is passed [1].
>
> There's is two issues:
> 1. We had a way that changed the run_kselftest.sh file in order to
> skip tests that hanged (that results in no test output from the run)
> various boards. Since the new change to kselftest, the patch to
> run_kselftest.sh got added that breaks our way to skip the tests and
> kernel hangs.
There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now:
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?id=a745f7af3cbd04b9c9c9e803429e1c20775b538d
> 2. The way we parse the output doesn't match the new output format.
>
> We will turn off the "next-* kselftest restults" until we have
> resolved this issue.
I'd prefer to keep the tests running -- they're still valuable logs to
have. I was just curious about the specific reporting. As long as the
issue is known, that's fine. No need to disable it entirely. :)
-Kees
--
Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 21:49 ` Anders Roxell
0 siblings, 0 replies; 10+ messages in thread
From: anders.roxell @ 2019-05-14 21:49 UTC (permalink / raw)
On Tue, 14 May 2019 at 17:46, Kees Cook <keescook at chromium.org> wrote:
>
> On Tue, May 14, 2019 at 05:28:45PM +0200, Anders Roxell wrote:
> > On Sat, 11 May 2019 at 16:51, Kees Cook <keescook at chromium.org> wrote:
> > > > Test Suites
> > > > -----------
> > > > * boot-lkft-kselftests-master-519
> > >
> > > What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
> >
> > The boot is passed when the auto-login-action in suite 'lava' is passed [1].
> >
> > There's is two issues:
> > 1. We had a way that changed the run_kselftest.sh file in order to
> > skip tests that hanged (that results in no test output from the run)
> > various boards. Since the new change to kselftest, the patch to
> > run_kselftest.sh got added that breaks our way to skip the tests and
> > kernel hangs.
aha, right that's awesome.
So its only scripts that we have to skip now then.
>
> There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now:
> https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?id=a745f7af3cbd04b9c9c9e803429e1c20775b538d
>
> > 2. The way we parse the output doesn't match the new output format.
> >
> > We will turn off the "next-* kselftest restults" until we have
> > resolved this issue.
>
> I'd prefer to keep the tests running -- they're still valuable logs to
> have. I was just curious about the specific reporting. As long as the
> issue is known, that's fine. No need to disable it entirely. :)
We will still run the tests on next, the plan is just to stop sending
reports until we have the parsing fixed.
Cheers,
Anders
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
* next-20190510 kselftest results
@ 2019-05-14 21:49 ` Anders Roxell
0 siblings, 0 replies; 10+ messages in thread
From: Anders Roxell @ 2019-05-14 21:49 UTC (permalink / raw)
On Tue, 14 May 2019@17:46, Kees Cook <keescook@chromium.org> wrote:
>
> On Tue, May 14, 2019@05:28:45PM +0200, Anders Roxell wrote:
> > On Sat, 11 May 2019@16:51, Kees Cook <keescook@chromium.org> wrote:
> > > > Test Suites
> > > > -----------
> > > > * boot-lkft-kselftests-master-519
> > >
> > > What counts as a "pass" for this? I looked at the x86_64 log, and there are lots of kselftest failures, but the dashboard counts it as a "pass"?
> >
> > The boot is passed when the auto-login-action in suite 'lava' is passed [1].
> >
> > There's is two issues:
> > 1. We had a way that changed the run_kselftest.sh file in order to
> > skip tests that hanged (that results in no test output from the run)
> > various boards. Since the new change to kselftest, the patch to
> > run_kselftest.sh got added that breaks our way to skip the tests and
> > kernel hangs.
aha, right that's awesome.
So its only scripts that we have to skip now then.
>
> There's a fix in -next that adds an "alarm" check now so no tests should ever hang for more than 30 seconds now:
> https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?id=a745f7af3cbd04b9c9c9e803429e1c20775b538d
>
> > 2. The way we parse the output doesn't match the new output format.
> >
> > We will turn off the "next-* kselftest restults" until we have
> > resolved this issue.
>
> I'd prefer to keep the tests running -- they're still valuable logs to
> have. I was just curious about the specific reporting. As long as the
> issue is known, that's fine. No need to disable it entirely. :)
We will still run the tests on next, the plan is just to stop sending
reports until we have the parsing fixed.
Cheers,
Anders
>
> --
> Kees Cook
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2019-05-14 21:49 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-11 10:14 next-20190510 kselftest results ci_notify
2019-05-11 10:14 ` ci_notify
2019-05-11 14:51 ` keescook
2019-05-11 14:51 ` Kees Cook
2019-05-14 15:28 ` anders.roxell
2019-05-14 15:28 ` Anders Roxell
2019-05-14 15:46 ` keescook
2019-05-14 15:46 ` Kees Cook
2019-05-14 21:49 ` anders.roxell
2019-05-14 21:49 ` Anders Roxell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.