* next/master baseline: 368 runs, 20 regressions (next-20200825)
@ 2020-08-25 12:52 kernelci.org bot
2020-08-25 17:58 ` Kevin Hilman
0 siblings, 1 reply; 6+ messages in thread
From: kernelci.org bot @ 2020-08-25 12:52 UTC (permalink / raw)
To: linux-next, kernel-build-reports, kernelci-results
next/master baseline: 368 runs, 20 regressions (next-20200825)
Regressions Summary
-------------------
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
at91-sama5d4_xplained | arm | lab-baylibre | gcc-8 | sama5_defconfig | 0/1
bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | exynos_defconfig | 0/1
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...G_ARM_LPAE=y | 0/1
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 0/1
omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
omap4-panda | arm | lab-collabora | gcc-8 | omap2plus_defconfig | 0/1
rk3399-gru-kevin | arm64 | lab-collabora | clang-10 | defconfig | 83/88
rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig+CON...OMIZE_BASE=y | 83/88
rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig | 0/1
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defconfig | 0/1
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defconfig | 0/1
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | sunxi_defconfig | 0/1
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | sunxi_defconfig | 0/1
Details: https://kernelci.org/test/job/next/branch/master/kernel/next-20200825/plan/baseline/
Test: baseline
Tree: next
Branch: master
Describe: next-20200825
URL: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
SHA: 3a00d3dfd4b68b208ecd5405e676d06c8ad6bb63
Test Regressions
----------------
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
at91-sama5d4_xplained | arm | lab-baylibre | gcc-8 | sama5_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d1f560dfd32ca09fb433
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: sama5_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/sama5_defconfig/gcc-8/lab-baylibre/baseline-at91-sama5d4_xplained.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/sama5_defconfig/gcc-8/lab-baylibre/baseline-at91-sama5d4_xplained.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d1f560dfd32ca09fb434
failing since 118 days (last pass: next-20200424, first fail: next-20200428)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
Results: 3 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
* baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
new failure (last pass: next-20200824)
1 lines
2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
2020-08-25 08:59:02.167000 (user:khilman) is already connected
2020-08-25 08:59:18.764000 \0
2020-08-25 08:59:18.765000
2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
2020-08-25 08:59:18.765000
2020-08-25 08:59:18.765000 DRAM: 948 MiB
2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
... (379 line(s) more)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | exynos_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d4700ef960b2c79fb42b
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: exynos_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/exynos_defconfig/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/exynos_defconfig/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d4700ef960b2c79fb42c
failing since 1 day (last pass: next-20200821, first fail: next-20200824)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...G_ARM_LPAE=y | 0/1
Details: https://kernelci.org/test/plan/id/5f44d702b9e92a2ade9fb443
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d702b9e92a2ade9fb444
failing since 31 days (last pass: next-20200723, first fail: next-20200724)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
Details: https://kernelci.org/test/plan/id/5f44d857bc45f69fcf9fb42b
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig+CONFIG_SMP=n
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d857bc45f69fcf9fb42c
failing since 31 days (last pass: next-20200723, first fail: next-20200724)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d9a3aaa45163fe9fb46c
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-exynos5422-odroidxu3.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d9a3aaa45163fe9fb46d
failing since 1 day (last pass: next-20200821, first fail: next-20200824)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
Results: 4 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig+CONFIG_SMP=n
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
failing since 40 days (last pass: next-20200706, first fail: next-20200715)
60 lines
2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
... (49 line(s) more)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
Results: 4 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
failing since 40 days (last pass: next-20200706, first fail: next-20200715)
60 lines
2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
... (49 line(s) more)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
omap4-panda | arm | lab-collabora | gcc-8 | omap2plus_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d34f0609c8cca89fb430
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: omap2plus_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/omap2plus_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/omap2plus_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d34f0609c8cca89fb431
failing since 34 days (last pass: next-20200706, first fail: next-20200721)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
rk3399-gru-kevin | arm64 | lab-collabora | clang-10 | defconfig | 83/88
Details: https://kernelci.org/test/plan/id/5f44d1eb402a9b85ed9fb450
Results: 83 PASS, 5 FAIL, 0 SKIP
Full config: defconfig
Compiler: clang-10 (clang version 10.0.0-++20200412072704+50d7e5d5e7d-1~exp1~20200412053303.133 )
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/clang-10/lab-collabora/baseline-rk3399-gru-kevin.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/clang-10/lab-collabora/baseline-rk3399-gru-kevin.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
* baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d1eb402a9b85ed9fb4a9
failing since 5 days (last pass: next-20200819, first fail: next-20200820)
11 lines
2020-08-25 08:54:57.138000 kern :alert : Mem abort info:
2020-08-25 08:54:57.142000 kern :alert : ESR = 0x96000006
2020-08-25 08:54:57.147000 kern :alert : EC = 0x25: DABT (current EL), IL = 32 bits
2020-08-25 08:54:57.151000 kern :alert : SET = 0, FnV = 0
2020-08-25 08:54:57.154000 kern :alert : EA = 0, S1PTW = 0
2020-08-25 08:54:57.158000 kern :alert : Data abort info:
2020-08-25 08:54:57.162000 kern :alert : ISV = 0, ISS = 0x00000006
2020-08-25 08:54:57.165000 kern :alert : CM = 0, WnR = 0
2020-08-25 08:54:57.172000 kern :alert : user pgtable: 4k pages, 48-bit VAs, pgdp=00000000eb396000
2020-08-25 08:54:57.184000 kern :alert : [0000000000000010] pgd=00000000eb397003, p4d=00000000eb397003, pud=00000000eb398003, pmd=0000000000000000
* baseline.dmesg.emerg: https://kernelci.org/test/case/id/5f44d1eb402a9b85ed9fb4aa
failing since 5 days (last pass: next-20200819, first fail: next-20200820)
2 lines
2020-08-25 08:54:57.205000 kern :emerg : Code: 2a0303f4 721d102a 54000040 b4000089 (f9400928)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig+CON...OMIZE_BASE=y | 83/88
Details: https://kernelci.org/test/plan/id/5f44d0ab32740dbec49fb466
Results: 83 PASS, 5 FAIL, 0 SKIP
Full config: defconfig+CONFIG_RANDOMIZE_BASE=y
Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-8/lab-collabora/baseline-rk3399-gru-kevin.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-8/lab-collabora/baseline-rk3399-gru-kevin.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
* baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d0ab32740dbec49fb4bf
failing since 5 days (last pass: next-20200819, first fail: next-20200820)
11 lines
2020-08-25 08:49:37.327000 kern :alert : Mem abort info:
2020-08-25 08:49:37.330000 kern :alert : ESR = 0x96000006
2020-08-25 08:49:37.336000 kern :alert : EC = 0x25: DABT (current EL), IL = 32 bits
2020-08-25 08:49:37.339000 kern :alert : SET = 0, FnV = 0
2020-08-25 08:49:37.343000 kern :alert : EA = 0, S1PTW = 0
2020-08-25 08:49:37.346000 kern :alert : Data abort info:
2020-08-25 08:49:37.350000 kern :alert : ISV = 0, ISS = 0x00000006
2020-08-25 08:49:37.354000 kern :alert : CM = 0, WnR = 0
2020-08-25 08:49:37.361000 kern :alert : user pgtable: 4k pages, 48-bit VAs, pgdp=00000000ebecf000
2020-08-25 08:49:37.373000 kern :alert : [0000000000000010] pgd=00000000ebf2f003, p4d=00000000ebf2f003, pud=00000000ebf34003, pmd=0000000000000000
* baseline.dmesg.emerg: https://kernelci.org/test/case/id/5f44d0ab32740dbec49fb4c0
failing since 5 days (last pass: next-20200819, first fail: next-20200820)
2 lines
2020-08-25 08:49:37.394000 kern :emerg : Code: 1a9f07e7 f10000df 7a4008e4 54000521 (f94008c8)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d57bf75eb727db9fb436
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: defconfig
Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-collabora/baseline-rk3399-gru-kevin.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-collabora/baseline-rk3399-gru-kevin.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d57bf75eb727db9fb437
failing since 1 day (last pass: next-20200821, first fail: next-20200824)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
Details: https://kernelci.org/test/plan/id/5f44d4f6092cbe05499fb42b
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig+CONFIG_SMP=n
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d4f6092cbe05499fb42c
failing since 38 days (last pass: next-20200715, first fail: next-20200717)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d6c3a9fea799309fb451
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d6c3a9fea799309fb452
failing since 38 days (last pass: next-20200716, first fail: next-20200717)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
Details: https://kernelci.org/test/plan/id/5f44d545f8872253769fb435
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig+CONFIG_SMP=n
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d545f8872253769fb436
failing since 38 days (last pass: next-20200715, first fail: next-20200717)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d6ebb9e92a2ade9fb435
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: multi_v7_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d6ebb9e92a2ade9fb436
failing since 38 days (last pass: next-20200716, first fail: next-20200717)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | sunxi_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d1efe6be1f56ca9fb436
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: sunxi_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/sunxi_defconfig/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/sunxi_defconfig/gcc-8/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d1efe6be1f56ca9fb437
failing since 35 days (last pass: next-20200717, first fail: next-20200720)
platform | arch | lab | compiler | defconfig | results
----------------------+-------+---------------+----------+------------------------------+--------
sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | sunxi_defconfig | 0/1
Details: https://kernelci.org/test/plan/id/5f44d2b2f9eace68909fb43f
Results: 0 PASS, 1 FAIL, 0 SKIP
Full config: sunxi_defconfig
Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/sunxi_defconfig/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/sunxi_defconfig/gcc-8/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
* baseline.login: https://kernelci.org/test/case/id/5f44d2b2f9eace68909fb440
failing since 35 days (last pass: next-20200717, first fail: next-20200720)
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: next/master baseline: 368 runs, 20 regressions (next-20200825)
2020-08-25 12:52 next/master baseline: 368 runs, 20 regressions (next-20200825) kernelci.org bot
@ 2020-08-25 17:58 ` Kevin Hilman
2020-09-01 10:57 ` Guillaume Tucker
[not found] ` <1630A3B4244BEF6E.8371@groups.io>
0 siblings, 2 replies; 6+ messages in thread
From: Kevin Hilman @ 2020-08-25 17:58 UTC (permalink / raw)
To: kernelci; +Cc: kernelci-results
[ -linux-next, +kernelci dev ]
"kernelci.org bot" <bot@kernelci.org> writes:
> next/master baseline: 368 runs, 20 regressions (next-20200825)
>
> Regressions Summary
> -------------------
>
> platform | arch | lab | compiler | defconfig | results
> ----------------------+-------+---------------+----------+------------------------------+--------
> at91-sama5d4_xplained | arm | lab-baylibre | gcc-8 | sama5_defconfig | 0/1
> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
> exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | exynos_defconfig | 0/1
> exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...G_ARM_LPAE=y | 0/1
> exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
> exynos5422-odroidxu3 | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 0/1
> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
> omap4-panda | arm | lab-collabora | gcc-8 | omap2plus_defconfig | 0/1
> rk3399-gru-kevin | arm64 | lab-collabora | clang-10 | defconfig | 83/88
> rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig+CON...OMIZE_BASE=y | 83/88
> rk3399-gru-kevin | arm64 | lab-collabora | gcc-8 | defconfig | 0/1
> sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
> sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | multi_v7_defconfig | 0/1
> sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 0/1
> sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | multi_v7_defconfig | 0/1
> sun7i-a20-cubieboard2 | arm | lab-baylibre | gcc-8 | sunxi_defconfig | 0/1
> sun7i-a20-cubieboard2 | arm | lab-clabbe | gcc-8 | sunxi_defconfig | 0/1
>
> Details: https://kernelci.org/test/job/next/branch/master/kernel/next-20200825/plan/baseline/
>
> Test: baseline
> Tree: next
> Branch: master
> Describe: next-20200825
> URL: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> SHA: 3a00d3dfd4b68b208ecd5405e676d06c8ad6bb63
>
>
> Test Regressions
> ----------------
>
>
> platform | arch | lab | compiler | defconfig | results
> ----------------------+-------+---------------+----------+------------------------------+--------
> at91-sama5d4_xplained | arm | lab-baylibre | gcc-8 | sama5_defconfig | 0/1
>
> Details: https://kernelci.org/test/plan/id/5f44d1f560dfd32ca09fb433
>
> Results: 0 PASS, 1 FAIL, 0 SKIP
> Full config: sama5_defconfig
> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/sama5_defconfig/gcc-8/lab-baylibre/baseline-at91-sama5d4_xplained.txt
> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/sama5_defconfig/gcc-8/lab-baylibre/baseline-at91-sama5d4_xplained.html
> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>
> * baseline.login: https://kernelci.org/test/case/id/5f44d1f560dfd32ca09fb434
> failing since 118 days (last pass: next-20200424, first fail: next-20200428)
>
>
> platform | arch | lab | compiler | defconfig | results
> ----------------------+-------+---------------+----------+------------------------------+--------
> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
>
> Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
>
> Results: 3 PASS, 1 FAIL, 0 SKIP
> Full config: defconfig
> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
>
> * baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
> new failure (last pass: next-20200824)
> 1 lines
>
> 2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
> 2020-08-25 08:59:02.167000 (user:khilman) is already connected
> 2020-08-25 08:59:18.764000 .
> 2020-08-25 08:59:18.765000
> 2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
> 2020-08-25 08:59:18.765000
> 2020-08-25 08:59:18.765000 DRAM: 948 MiB
> 2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
> 2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
> 2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
> ... (379 line(s) more)
Shouldn't we show the last few lines of the log instead of the first
few lines?
> platform | arch | lab | compiler | defconfig | results
> ----------------------+-------+---------------+----------+------------------------------+--------
> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
>
> Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
>
> Results: 4 PASS, 1 FAIL, 0 SKIP
> Full config: multi_v7_defconfig+CONFIG_SMP=n
> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>
> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
> 60 lines
>
> 2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
> 2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
> 2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
> 2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
> 2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
> 2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
> 2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
> 2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
> 2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
> 2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
> ... (49 line(s) more)
... similar here ...
> platform | arch | lab | compiler | defconfig | results
> ----------------------+-------+---------------+----------+------------------------------+--------
> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
>
> Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
>
> Results: 4 PASS, 1 FAIL, 0 SKIP
> Full config: multi_v7_defconfig
> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>
> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
> 60 lines
>
> 2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
> 2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
> 2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
> 2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
> 2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
> 2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
> 2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
> 2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
> 2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
> 2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
> ... (49 line(s) more)
>
... and here.
It's not always true, but I think typically the last part of the log
will be more interesting to spotting an issue.
Kevin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: next/master baseline: 368 runs, 20 regressions (next-20200825)
2020-08-25 17:58 ` Kevin Hilman
@ 2020-09-01 10:57 ` Guillaume Tucker
2020-09-01 13:22 ` Guenter Roeck
[not found] ` <1630A3B4244BEF6E.8371@groups.io>
1 sibling, 1 reply; 6+ messages in thread
From: Guillaume Tucker @ 2020-09-01 10:57 UTC (permalink / raw)
To: khilman; +Cc: kernelci, kernelci-results
On 25/08/2020 18:58, Kevin Hilman wrote:
> [ -linux-next, +kernelci dev ]
>
> "kernelci.org bot" <bot@kernelci.org> writes:
>
>> next/master baseline: 368 runs, 20 regressions (next-20200825)
[...]
>> platform | arch | lab | compiler | defconfig | results
>> ----------------------+-------+---------------+----------+------------------------------+--------
>> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
>>
>> Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
>>
>> Results: 3 PASS, 1 FAIL, 0 SKIP
>> Full config: defconfig
>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
>>
>> * baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
>> new failure (last pass: next-20200824)
>> 1 lines
>>
>> 2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
>> 2020-08-25 08:59:02.167000 (user:khilman) is already connected
>> 2020-08-25 08:59:18.764000 .
>> 2020-08-25 08:59:18.765000
>> 2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
>> 2020-08-25 08:59:18.765000
>> 2020-08-25 08:59:18.765000 DRAM: 948 MiB
>> 2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
>> 2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
>> 2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
>> ... (379 line(s) more)
>
> Shouldn't we show the last few lines of the log instead of the first
> few lines?
In this case, it's the span of log lines that was wrong. The log
lines should have started after login, not at the bootloader
stage. The "critical" error is:
[ 17.957677] hwmon hwmon1: Undervoltage detected!
This is a recurring error and I believe previous runs showed it
correctly, so it's worth investigating why the start line was not
set correctly in this case.
By the way, it only seems to be happening in lab-baylibre and I
wonder if it could have anything to do with an unstable power
supply.
>> platform | arch | lab | compiler | defconfig | results
>> ----------------------+-------+---------------+----------+------------------------------+--------
>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
>>
>> Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
>>
>> Results: 4 PASS, 1 FAIL, 0 SKIP
>> Full config: multi_v7_defconfig+CONFIG_SMP=n
>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>
>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>> 60 lines
>>
>> 2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
>> 2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
>> 2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
>> 2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
>> 2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
>> 2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
>> 2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
>> 2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
>> 2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
>> 2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
>> ... (49 line(s) more)
>
> ... similar here ...
In this case, all the lines are actually the same.
>> platform | arch | lab | compiler | defconfig | results
>> ----------------------+-------+---------------+----------+------------------------------+--------
>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
>>
>> Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
>>
>> Results: 4 PASS, 1 FAIL, 0 SKIP
>> Full config: multi_v7_defconfig
>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>
>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>> 60 lines
>>
>> 2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
>> 2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
>> 2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
>> 2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
>> 2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
>> 2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
>> 2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
>> 2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
>> 2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
>> 2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
>> ... (49 line(s) more)
>>
> ... and here.
>
> It's not always true, but I think typically the last part of the log
> will be more interesting to spotting an issue.
In the examples above, the top part of the log lines was
completely relevant. For things like Oops and other kernel
errors, the issue is often shown first and then there's a stack
dump. Ultimately, each test suite will be doing things
differently. So it's really hard to tell whether the last lines
are more likely to be useful than the first ones. There's also a
link to the details on the frontend where all the lines are
visible:
https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
Maybe we could increase the limit and show up to say, 16 lines
instead of 10 but then when there are 17 or more show the first
and last 3 lines? It's all kind of heuristics at this point. It
might also be something to tune on a per-test-suite basis when we
have kci_email and some YAML configuration.
Guillaume
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: next/master baseline: 368 runs, 20 regressions (next-20200825)
[not found] ` <1630A3B4244BEF6E.8371@groups.io>
@ 2020-09-01 12:41 ` Guillaume Tucker
0 siblings, 0 replies; 6+ messages in thread
From: Guillaume Tucker @ 2020-09-01 12:41 UTC (permalink / raw)
To: khilman; +Cc: kernelci, kernelci-results
On 01/09/2020 11:57, Guillaume Tucker wrote:
> On 25/08/2020 18:58, Kevin Hilman wrote:
>> [ -linux-next, +kernelci dev ]
>>
>> "kernelci.org bot" <bot@kernelci.org> writes:
>>
>>> next/master baseline: 368 runs, 20 regressions (next-20200825)
>
> [...]
>
>>> platform | arch | lab | compiler | defconfig | results
>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
>>>
>>> Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
>>>
>>> Results: 3 PASS, 1 FAIL, 0 SKIP
>>> Full config: defconfig
>>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
>>>
>>> * baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
>>> new failure (last pass: next-20200824)
>>> 1 lines
>>>
>>> 2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
>>> 2020-08-25 08:59:02.167000 (user:khilman) is already connected
>>> 2020-08-25 08:59:18.764000 .
>>> 2020-08-25 08:59:18.765000
>>> 2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
>>> 2020-08-25 08:59:18.765000
>>> 2020-08-25 08:59:18.765000 DRAM: 948 MiB
>>> 2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
>>> 2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
>>> 2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
>>> ... (379 line(s) more)
>>
>> Shouldn't we show the last few lines of the log instead of the first
>> few lines?
>
> In this case, it's the span of log lines that was wrong. The log
> lines should have started after login, not at the bootloader
> stage. The "critical" error is:
>
> [ 17.957677] hwmon hwmon1: Undervoltage detected!
>
> This is a recurring error and I believe previous runs showed it
> correctly, so it's worth investigating why the start line was not
> set correctly in this case.
In fact, the baseline.login test case doesn't appear here:
https://kernelci.org/test/plan/id/5f456c171cabe7cb219fb493/
So that would explain why all the boot log was included in this
test case. I guess that's the known issue of the unreliable LAVA
auto-login step result.
Guillaume
> By the way, it only seems to be happening in lab-baylibre and I
> wonder if it could have anything to do with an unstable power
> supply.
>
>>> platform | arch | lab | compiler | defconfig | results
>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
>>>
>>> Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
>>>
>>> Results: 4 PASS, 1 FAIL, 0 SKIP
>>> Full config: multi_v7_defconfig+CONFIG_SMP=n
>>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>>
>>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
>>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>>> 60 lines
>>>
>>> 2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
>>> 2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
>>> 2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
>>> 2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
>>> 2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
>>> 2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
>>> 2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
>>> 2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
>>> 2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
>>> 2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
>>> ... (49 line(s) more)
>>
>> ... similar here ...
>
> In this case, all the lines are actually the same.
>
>>> platform | arch | lab | compiler | defconfig | results
>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
>>>
>>> Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
>>>
>>> Results: 4 PASS, 1 FAIL, 0 SKIP
>>> Full config: multi_v7_defconfig
>>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>>
>>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>>> 60 lines
>>>
>>> 2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
>>> 2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
>>> 2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
>>> 2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
>>> 2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
>>> 2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
>>> 2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
>>> 2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
>>> 2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
>>> 2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
>>> ... (49 line(s) more)
>>>
>> ... and here.
>>
>> It's not always true, but I think typically the last part of the log
>> will be more interesting to spotting an issue.
>
> In the examples above, the top part of the log lines was
> completely relevant. For things like Oops and other kernel
> errors, the issue is often shown first and then there's a stack
> dump. Ultimately, each test suite will be doing things
> differently. So it's really hard to tell whether the last lines
> are more likely to be useful than the first ones. There's also a
> link to the details on the frontend where all the lines are
> visible:
>
> https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>
> Maybe we could increase the limit and show up to say, 16 lines
> instead of 10 but then when there are 17 or more show the first
> and last 3 lines? It's all kind of heuristics at this point. It
> might also be something to tune on a per-test-suite basis when we
> have kci_email and some YAML configuration.
>
> Guillaume
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: next/master baseline: 368 runs, 20 regressions (next-20200825)
2020-09-01 10:57 ` Guillaume Tucker
@ 2020-09-01 13:22 ` Guenter Roeck
2020-09-01 13:42 ` Guillaume Tucker
0 siblings, 1 reply; 6+ messages in thread
From: Guenter Roeck @ 2020-09-01 13:22 UTC (permalink / raw)
To: kernelci, Guillaume Tucker; +Cc: Kevin Hilman, kernelci-results
On Tue, Sep 1, 2020 at 3:57 AM Guillaume Tucker
<guillaume.tucker@collabora.com> wrote:
>
> On 25/08/2020 18:58, Kevin Hilman wrote:
> > [ -linux-next, +kernelci dev ]
> >
> > "kernelci.org bot" <bot@kernelci.org> writes:
> >
> >> next/master baseline: 368 runs, 20 regressions (next-20200825)
>
> [...]
>
> >> platform | arch | lab | compiler | defconfig | results
> >> ----------------------+-------+---------------+----------+------------------------------+--------
> >> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
> >>
> >> Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
> >>
> >> Results: 3 PASS, 1 FAIL, 0 SKIP
> >> Full config: defconfig
> >> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
> >> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
> >> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
> >> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
> >>
> >> * baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
> >> new failure (last pass: next-20200824)
> >> 1 lines
> >>
> >> 2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
> >> 2020-08-25 08:59:02.167000 (user:khilman) is already connected
> >> 2020-08-25 08:59:18.764000 .
> >> 2020-08-25 08:59:18.765000
> >> 2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
> >> 2020-08-25 08:59:18.765000
> >> 2020-08-25 08:59:18.765000 DRAM: 948 MiB
> >> 2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
> >> 2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
> >> 2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
> >> ... (379 line(s) more)
> >
> > Shouldn't we show the last few lines of the log instead of the first
> > few lines?
>
> In this case, it's the span of log lines that was wrong. The log
> lines should have started after login, not at the bootloader
> stage. The "critical" error is:
>
> [ 17.957677] hwmon hwmon1: Undervoltage detected!
>
That is just a log message, though, that triggers a sysfs_notify call
to in0_lcrit_alarm. By itself it doesn't result in a crash/reboot.
Guenter
> This is a recurring error and I believe previous runs showed it
> correctly, so it's worth investigating why the start line was not
> set correctly in this case.
>
> By the way, it only seems to be happening in lab-baylibre and I
> wonder if it could have anything to do with an unstable power
> supply.
>
> >> platform | arch | lab | compiler | defconfig | results
> >> ----------------------+-------+---------------+----------+------------------------------+--------
> >> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
> >>
> >> Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
> >>
> >> Results: 4 PASS, 1 FAIL, 0 SKIP
> >> Full config: multi_v7_defconfig+CONFIG_SMP=n
> >> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
> >> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
> >> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
> >> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
> >>
> >> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
> >> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
> >> 60 lines
> >>
> >> 2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
> >> 2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
> >> 2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
> >> 2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
> >> 2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
> >> 2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
> >> 2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
> >> 2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
> >> 2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
> >> 2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
> >> ... (49 line(s) more)
> >
> > ... similar here ...
>
> In this case, all the lines are actually the same.
>
> >> platform | arch | lab | compiler | defconfig | results
> >> ----------------------+-------+---------------+----------+------------------------------+--------
> >> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
> >>
> >> Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
> >>
> >> Results: 4 PASS, 1 FAIL, 0 SKIP
> >> Full config: multi_v7_defconfig
> >> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
> >> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
> >> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
> >> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
> >>
> >> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
> >> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
> >> 60 lines
> >>
> >> 2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
> >> 2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
> >> 2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
> >> 2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
> >> 2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
> >> 2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
> >> 2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
> >> 2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
> >> 2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
> >> 2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
> >> ... (49 line(s) more)
> >>
> > ... and here.
> >
> > It's not always true, but I think typically the last part of the log
> > will be more interesting to spotting an issue.
>
> In the examples above, the top part of the log lines was
> completely relevant. For things like Oops and other kernel
> errors, the issue is often shown first and then there's a stack
> dump. Ultimately, each test suite will be doing things
> differently. So it's really hard to tell whether the last lines
> are more likely to be useful than the first ones. There's also a
> link to the details on the frontend where all the lines are
> visible:
>
> https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>
> Maybe we could increase the limit and show up to say, 16 lines
> instead of 10 but then when there are 17 or more show the first
> and last 3 lines? It's all kind of heuristics at this point. It
> might also be something to tune on a per-test-suite basis when we
> have kci_email and some YAML configuration.
>
> Guillaume
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: next/master baseline: 368 runs, 20 regressions (next-20200825)
2020-09-01 13:22 ` Guenter Roeck
@ 2020-09-01 13:42 ` Guillaume Tucker
0 siblings, 0 replies; 6+ messages in thread
From: Guillaume Tucker @ 2020-09-01 13:42 UTC (permalink / raw)
To: Guenter Roeck; +Cc: kernelci, Kevin Hilman, kernelci-results
On 01/09/2020 14:22, Guenter Roeck wrote:
> On Tue, Sep 1, 2020 at 3:57 AM Guillaume Tucker
> <guillaume.tucker@collabora.com> wrote:
>>
>> On 25/08/2020 18:58, Kevin Hilman wrote:
>>> [ -linux-next, +kernelci dev ]
>>>
>>> "kernelci.org bot" <bot@kernelci.org> writes:
>>>
>>>> next/master baseline: 368 runs, 20 regressions (next-20200825)
>>
>> [...]
>>
>>>> platform | arch | lab | compiler | defconfig | results
>>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>>> bcm2837-rpi-3-b | arm64 | lab-baylibre | gcc-8 | defconfig | 3/4
>>>>
>>>> Details: https://kernelci.org/test/plan/id/5f44d354974cc7a9749fb42b
>>>>
>>>> Results: 3 PASS, 1 FAIL, 0 SKIP
>>>> Full config: defconfig
>>>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
>>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.txt
>>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm64/defconfig/gcc-8/lab-baylibre/baseline-bcm2837-rpi-3-b.html
>>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/arm64/baseline/rootfs.cpio.gz
>>>>
>>>> * baseline.dmesg.crit: https://kernelci.org/test/case/id/5f44d354974cc7a9749fb42f
>>>> new failure (last pass: next-20200824)
>>>> 1 lines
>>>>
>>>> 2020-08-25 08:59:02.167000 Connected to bcm2837-rpi-3-b console [channel connected] (~$quit to exit)
>>>> 2020-08-25 08:59:02.167000 (user:khilman) is already connected
>>>> 2020-08-25 08:59:18.764000 .
>>>> 2020-08-25 08:59:18.765000
>>>> 2020-08-25 08:59:18.765000 U-Boot 2018.11 (Dec 04 2018 - 10:54:32 -0800)
>>>> 2020-08-25 08:59:18.765000
>>>> 2020-08-25 08:59:18.765000 DRAM: 948 MiB
>>>> 2020-08-25 08:59:18.780000 RPI 3 Model B (0xa02082)
>>>> 2020-08-25 08:59:18.866000 MMC: mmc@7e202000: 0, sdhci@7e300000: 1
>>>> 2020-08-25 08:59:18.899000 Loading Environment from FAT... *** Warning - bad CRC, using default environment
>>>> ... (379 line(s) more)
>>>
>>> Shouldn't we show the last few lines of the log instead of the first
>>> few lines?
>>
>> In this case, it's the span of log lines that was wrong. The log
>> lines should have started after login, not at the bootloader
>> stage. The "critical" error is:
>>
>> [ 17.957677] hwmon hwmon1: Undervoltage detected!
>>
> That is just a log message, though, that triggers a sysfs_notify call
> to in0_lcrit_alarm. By itself it doesn't result in a crash/reboot.
That's right, the test case is "baseline.dmesg.crit" which looks
for "critical" kernel error messages. Early boot crashes are
normally caught by the "baseline.login" test case.
New kernel errors at the "emerg", "crit" or "alert" levels are
detected as regressions and reported as such. The hwmon issue in
this case is an intermittent one, so it regularly comes back as a
regression.
Best wishes,
Guillaume
>> This is a recurring error and I believe previous runs showed it
>> correctly, so it's worth investigating why the start line was not
>> set correctly in this case.
>>
>> By the way, it only seems to be happening in lab-baylibre and I
>> wonder if it could have anything to do with an unstable power
>> supply.
>>
>>>> platform | arch | lab | compiler | defconfig | results
>>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defc...CONFIG_SMP=n | 4/5
>>>>
>>>> Details: https://kernelci.org/test/plan/id/5f44d4448493d6f73c9fb42b
>>>>
>>>> Results: 4 PASS, 1 FAIL, 0 SKIP
>>>> Full config: multi_v7_defconfig+CONFIG_SMP=n
>>>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.txt
>>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-8/lab-collabora/baseline-omap4-panda.html
>>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>>>
>>>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d4448493d6f73c9fb431
>>>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>>>> 60 lines
>>>>
>>>> 2020-08-25 09:05:03.232000 kern :alert : BUG: Bad page state in process swapper pfn:9c802
>>>> 2020-08-25 09:05:03.238000 kern :alert : BUG: Bad page state in process swapper pfn:9c803
>>>> 2020-08-25 09:05:03.244000 kern :alert : BUG: Bad page state in process swapper pfn:9c804
>>>> 2020-08-25 09:05:03.249000 kern :alert : BUG: Bad page state in process swapper pfn:9c805
>>>> 2020-08-25 09:05:03.255000 kern :alert : BUG: Bad page state in process swapper pfn:9c806
>>>> 2020-08-25 09:05:03.261000 kern :alert : BUG: Bad page state in process swapper pfn:9c807
>>>> 2020-08-25 09:05:03.267000 kern :alert : BUG: Bad page state in process swapper pfn:9c808
>>>> 2020-08-25 09:05:03.273000 kern :alert : BUG: Bad page state in process swapper pfn:9c809
>>>> 2020-08-25 09:05:03.278000 kern :alert : BUG: Bad page state in process swapper pfn:9c80a
>>>> 2020-08-25 09:05:03.284000 kern :alert : BUG: Bad page state in process swapper pfn:9c80b
>>>> ... (49 line(s) more)
>>>
>>> ... similar here ...
>>
>> In this case, all the lines are actually the same.
>>
>>>> platform | arch | lab | compiler | defconfig | results
>>>> ----------------------+-------+---------------+----------+------------------------------+--------
>>>> omap4-panda | arm | lab-collabora | gcc-8 | multi_v7_defconfig | 4/5
>>>>
>>>> Details: https://kernelci.org/test/plan/id/5f44d6605371b404fc9fb42b
>>>>
>>>> Results: 4 PASS, 1 FAIL, 0 SKIP
>>>> Full config: multi_v7_defconfig
>>>> Compiler: gcc-8 (arm-linux-gnueabihf-gcc (Debian 8.3.0-2) 8.3.0)
>>>> Plain log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.txt
>>>> HTML log: https://storage.kernelci.org//next/master/next-20200825/arm/multi_v7_defconfig/gcc-8/lab-collabora/baseline-omap4-panda.html
>>>> Rootfs: http://storage.kernelci.org/images/rootfs/buildroot/kci-2020.05/armel/baseline/rootfs.cpio.gz
>>>>
>>>> * baseline.dmesg.alert: https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>>>> failing since 40 days (last pass: next-20200706, first fail: next-20200715)
>>>> 60 lines
>>>>
>>>> 2020-08-25 09:14:02.674000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c802
>>>> 2020-08-25 09:14:02.680000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c803
>>>> 2020-08-25 09:14:02.686000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c804
>>>> 2020-08-25 09:14:02.692000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c805
>>>> 2020-08-25 09:14:02.698000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c806
>>>> 2020-08-25 09:14:02.704000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c807
>>>> 2020-08-25 09:14:02.710000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c808
>>>> 2020-08-25 09:14:02.716000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c809
>>>> 2020-08-25 09:14:02.722000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80a
>>>> 2020-08-25 09:14:02.728000 kern :alert : BUG: Bad page state in process swapper/0 pfn:9c80b
>>>> ... (49 line(s) more)
>>>>
>>> ... and here.
>>>
>>> It's not always true, but I think typically the last part of the log
>>> will be more interesting to spotting an issue.
>>
>> In the examples above, the top part of the log lines was
>> completely relevant. For things like Oops and other kernel
>> errors, the issue is often shown first and then there's a stack
>> dump. Ultimately, each test suite will be doing things
>> differently. So it's really hard to tell whether the last lines
>> are more likely to be useful than the first ones. There's also a
>> link to the details on the frontend where all the lines are
>> visible:
>>
>> https://kernelci.org/test/case/id/5f44d6605371b404fc9fb431
>>
>> Maybe we could increase the limit and show up to say, 16 lines
>> instead of 10 but then when there are 17 or more show the first
>> and last 3 lines? It's all kind of heuristics at this point. It
>> might also be something to tune on a per-test-suite basis when we
>> have kci_email and some YAML configuration.
>>
>> Guillaume
>>
>>
>>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-01 13:42 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-25 12:52 next/master baseline: 368 runs, 20 regressions (next-20200825) kernelci.org bot
2020-08-25 17:58 ` Kevin Hilman
2020-09-01 10:57 ` Guillaume Tucker
2020-09-01 13:22 ` Guenter Roeck
2020-09-01 13:42 ` Guillaume Tucker
[not found] ` <1630A3B4244BEF6E.8371@groups.io>
2020-09-01 12:41 ` Guillaume Tucker
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.