Re: [cip-dev] cip/linux-4.19.y-cip baseline: 121 runs, 1 regressions (v4.19.217-cip62) #kernelci

[Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>]

On Tue, Nov 30, 2021 at 5:41 PM Pavel Machek <pavel@denx.de> wrote:
>
> Hi!
>
> > platform | arch | lab           | compiler | defconfig           | regressions
> > ---------+------+---------------+----------+---------------------+------------
> > panda    | arm  | lab-collabora | gcc-10   | omap2plus_defconfig | 1
> >
> >   Details:     https://kernelci.org/test/plan/id/61a587a0ab3b0079bd18f6d7
> >
> >   Results:     5 PASS, 1 FAIL, 0 SKIP
> >   Full config: omap2plus_defconfig
> >   Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
> >   Plain log:   https://storage.kernelci.org//cip/linux-4.19.y-cip/v4.19.217-cip62/arm/omap2plus_defconfig/gcc-10/lab-collabora/baseline-panda.txt
> >   HTML log:
> > https://storage.kernelci.org//cip/linux-4.19.y-cip/v4.19.217-cip62/arm/omap2plus_defconfig/gcc-10/lab-collabora/baseline-panda.html
>
> So this has enough information to tell me that we have a kernel
> problem there, good:
>
> 02:08:17.792502  <6>[   18.847320] omap-mailbox 4a0f4000.mailbox: omap
> mailbox rev 0x400
> 02:08:17.800596  <0>[   18.857543] BUG: spinlock bad magic on CPU#0,
> udevd/110
> 02:08:17.808843  <6>[   18.857788] emif 4c000000.emif: emif_probe:
> device configured with addr = (ptrval) and IRQ26
> 02:08:17.818753  <0>[   18.863311]  lock: emif_lock+0x0/0xffffecfc
> [emif], .magic: dead4ead, .owner: <none>/-1, .owner_cpu: -1
> 02:08:17.825277  <4>[   18.877502] CPU: 0 PID: 110 Comm: udevd Not
> tainted 4.19.217-cip62 #1
> 02:08:17.831655  <4>[   18.888275] Hardware name: Generic OMAP4
> (Flattened Device Tree)
> 02:08:17.839660  <4>[   18.894195] [<c01123bc>] (unwind_backtrace)
> from [<c010cc28>] (show_stack+0x10/0x14)
> 02:08:17.847570  <4>[   18.900482] [<c010cc28>] (show_stack) from
> [<c0949dd4>] (dump_stack+0xe0/0x114)
> 02:08:17.855211  <4>[   18.905761] [<c0949dd4>] (dump_stack) from
> [<c01a9168>] (do_raw_spin_lock+0xbc/0x124)
>
> Unfortunately, not enough information to debug the problem, I'm
> afraid. Questions to start debugging this would be:
>
> a) is the problem deterministic?
>

as the testing board are actually different is unclear.
The board and results can be seen here:
https://linux.kernelci.org/test/plan/id/61a587a0ab3b0079bd18f6d7/

the output log is here:
https://storage.kernelci.org/cip/linux-4.19.y-cip/v4.19.217-cip62/arm/omap2plus_defconfig/gcc-10/lab-collabora/baseline-panda.html
working board example log is here (works for 4.19.216-cip61 but using
a board on baylibre that could be different from the board on
collabora):
https://storage.kernelci.org/cip/linux-4.19.y-cip/v4.19.216-cip61/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-panda.html
looks like there is also a passed test on lab-baylibre for the panda
board with the same kernel:
https://storage.kernelci.org/cip/linux-4.19.y-cip/v4.19.217-cip62/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-panda.html

The job json file for replicate the job on a lava lab with panda board
can be found here:
https://lava.collabora.co.uk/scheduler/job/5113437/definition


> b) is the problem present in v4.19.217 ?
>

here is the diff with 4.19.217 stable
https://www.diffchecker.com/JjmXiP7b

> c) what were the last -stable and -cip kernels that worked?
>

if you click on the link on the details or the link here under:
https://linux.kernelci.org/test/case/id/61a587a0ab3b0079bd18f6dd/
is wrote:
New regression, last pass: v4.19.216-cip61

This is the one that passed:
https://linux.kernelci.org/test/plan/id/618f6407f33d0dd1e633595d/


- About bisecting the kernel.
We are currently in the process of activating CIP to the list of
automated bisected kernel
for getting more information on this and future problems.
The following issue is the request to KernelCI for adding CIP to the
automated bisected kernel list.
https://github.com/kernelci/kernelci-core/issues/922



Thanks,
Alice


--
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================