Subject: next/master baseline: 817 runs, 53 regressions (next-20230317)

From: "kernelci.org bot" <[email-masked]>
Date: Fri, 17 Mar 2023 01:31:23 -0700 (PDT)

next/master baseline: 817 runs, 53 regressions (next-20230317)

Regressions Summary
-------------------

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          
at91-sama5d4_xplained        | arm    | lab-baylibre    | gcc-10   | sama5_defconfig              | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 1          
fsl-ls1088a-rdb              | arm64  | lab-nxp         | gcc-10   | defconfig+debug              | 1          
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...6-chromebook | 1          
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
jh7100-starfi...isionfive-v1 | riscv  | lab-clabbe      | gcc-10   | defconfig+kselftest          | 1          
meson-gxl-s905x-khadas-vim   | arm64  | lab-baylibre    | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          
qemu_arm-vexpress-a9         | arm    | lab-baylibre    | gcc-10   | vexpress_defconfig           | 1          
qemu_arm-vexpress-a9         | arm    | lab-broonie     | gcc-10   | vexpress_defconfig           | 1          
qemu_arm-vexpress-a9         | arm    | lab-collabora   | gcc-10   | vexpress_defconfig           | 1          
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_mips-malta              | mips   | lab-collabora   | gcc-10   | malta_defconfig              | 1          
r8a7743-iwg20d-q7            | arm    | lab-cip         | gcc-10   | shmobile_defconfig           | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+videodec           | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+ima                | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+crypto             | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+debug              | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig                    | 1          
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+CON...OMIZE_BASE=y | 1          
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
rk3399-roc-pc                | arm64  | lab-clabbe      | gcc-10   | defconfig+debug              | 1          
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          
sun8i-h3-libretech-all-h3-cc | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
zynqmp-zcu102                | arm64  | lab-cip         | gcc-10   | defconfig+debug              | 1          

  Details:  https://kernelci.org/test/job/next/branch/master/kernel/next-20230317/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   master
  Describe: next-20230317
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      6f08c1de13a9403341c18b66638a05588b2663ce 

Test Regressions
---------------- 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/6413f4dab414cce1e38c8658

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f4dab414cce1e38c8659
        failing since 180 days (last pass: next-20220915, first fail: next-20220916) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f7aac359e7ae318c8631

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f7aac359e7ae318c8632
        failing since 183 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413ea3c2a48347b498c8630

  Results:     6 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+kselftest
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/6413ea3c2a48347b498c8633
        failing since 6 days (last pass: next-20230309, first fail: next-20230310)
        3 lines

    2023-03-17T04:18:57.229329  kern  :emerg : __common_interrupt: 1.55 No irq handler for vector

    2023-03-17T04:18:57.235288  kern  :emerg : __common_interrupt: 2.55 No irq handler for vector

    2023-03-17T04:18:57.242177  kern  :emerg : __common_interrupt: 3.55 No irq handler for vector

    2023-03-17T04:18:57.249016  <8>[   13.918778] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=3>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
at91-sama5d4_xplained        | arm    | lab-baylibre    | gcc-10   | sama5_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/6413ea3a111a2a66578c8695

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sama5_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/sama5_defconfig/gcc-10/lab-baylibre/baseline-at91-sama5d4_xplained.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/sama5_defconfig/gcc-10/lab-baylibre/baseline-at91-sama5d4_xplained.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ea3a111a2a66578c8696
        failing since 44 days (last pass: next-20230125, first fail: next-20230131) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/6413eb3c6bf1613f648c8630

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413eb3c6bf1613f648c8639
        failing since 57 days (last pass: next-20221219, first fail: next-20230118)

    2023-03-17T04:23:13.493814  <8>[   16.726918] <LAVA_SIGNAL_ENDRUN 0_dmesg 3418663_1.5.2.4.1>
    2023-03-17T04:23:13.601819  / # #
    2023-03-17T04:23:13.705489  export SHELL=/bin/sh
    2023-03-17T04:23:13.706425  #
    2023-03-17T04:23:13.808477  / # export SHELL=/bin/sh. /lava-3418663/environment
    2023-03-17T04:23:13.809348  
    2023-03-17T04:23:13.911458  / # . /lava-3418663/environment/lava-3418663/bin/lava-test-runner /lava-3418663/1
    2023-03-17T04:23:13.912797  
    2023-03-17T04:23:13.917622  / # /lava-3418663/bin/lava-test-runner /lava-3418663/1
    2023-03-17T04:23:13.998628  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ebdf465cde20858c8685

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413ebdf465cde20858c868e
        failing since 58 days (last pass: next-20221219, first fail: next-20230117)

    2023-03-17T04:25:40.231829  + set +x<8>[   15.275928] <LAVA_SIGNAL_ENDRUN 0_dmesg 3418751_1.5.2.4.1>
    2023-03-17T04:25:40.232778  
    2023-03-17T04:25:40.347762  / # #
    2023-03-17T04:25:40.451009  export SHELL=/bin/sh
    2023-03-17T04:25:40.452075  #
    2023-03-17T04:25:40.553865  / # export SHELL=/bin/sh. /lava-3418751/environment
    2023-03-17T04:25:40.554959  
    2023-03-17T04:25:40.555674  / # <3>[   15.551269] Bluetooth: hci0: command 0x0c03 tx timeout
    2023-03-17T04:25:40.657813  . /lava-3418751/environment/lava-3418751/bin/lava-test-runner /lava-3418751/1
    2023-03-17T04:25:40.660112   
    ... (13 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413ef6d6d431991068c866c

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413ef6d6d431991068c8675
        failing since 57 days (last pass: next-20221219, first fail: next-20230118)

    2023-03-17T04:41:02.644605  <8>[   15.085006] <LAVA_SIGNAL_ENDRUN 0_dmesg 3418810_1.5.2.4.1>
    2023-03-17T04:41:02.755524  / # #
    2023-03-17T04:41:02.857777  export SHELL=/bin/sh
    2023-03-17T04:41:02.858634  #
    2023-03-17T04:41:02.960555  / # export SHELL=/bin/sh. /lava-3418810/environment
    2023-03-17T04:41:02.961799  
    2023-03-17T04:41:03.064486  / # . /lava-3418810/environment/lava-3418810/bin/lava-test-runner /lava-3418810/1
    2023-03-17T04:41:03.065641  
    2023-03-17T04:41:03.070301  / # /lava-3418810/bin/lava-test-runner /lava-3418810/1
    2023-03-17T04:41:03.161006  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/6413e90cebc68850fb8c8650

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: davinci_all_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/6413e90cebc68850fb8c8653
        failing since 31 days (last pass: next-20230208, first fail: next-20230213)
        29 lines

    2023-03-17T04:13:49.578281  kern  :alert : Register r9 information: non-paged memory
    2023-03-17T04:13:49.579061  kern  :alert : Register r10 information: NULL pointer
    2023-03-17T04:13:49.579676  kern  :alert : Register r11 information: 1-page vmalloc region starting at 0xbf2de000 allocated at load_module+0x6d4/0x18ec
    2023-03-17T04:13:49.581363  kern  :alert : Register r12 information: non-paged memory
    2023-03-17T04:13:49.685228  kern  :emerg : Internal error: Oops: 17 [#1] PREEMPT ARM
    2023-03-17T04:13:49.685857  kern  :emerg : Process udevd (pid: 73, stack limit = 0x26d774d9)
    2023-03-17T04:13:49.686358  kern  :emerg : Stack: (0xc8909d08 to 0xc890a000)
    2023-03-17T04:13:49.686806  kern  :emerg : 9d00:                   c4e86200 c076b880 00000000 c03ac9f8 00000000 4ac20d7a
    2023-03-17T04:13:49.687550  kern  :emerg : 9d20: 0000005d 4ac20d7a c076b880 bf2de220 00000000 00000000 00000000 00000001
    2023-03-17T04:13:49.687943  kern  :emerg : 9d40: c076b87c bf311408 bf2de220 00000000 00000000 00000000 00000000 bf2e23c8 
    ... (6 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
fsl-ls1088a-rdb              | arm64  | lab-nxp         | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6413f4688fe5e104f48c8675

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-nxp/baseline-fsl-ls1088a-rdb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-nxp/baseline-fsl-ls1088a-rdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f4688fe5e104f48c8676
        failing since 52 days (last pass: next-20230120, first fail: next-20230123) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...6-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee397e3f6414848c862f

  Results:     18 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.bootrr.tpm-chip-is-online: https://kernelci.org/test/case/id/6413ee397e3f6414848c8642
        new failure (last pass: next-20230316)

    2023-03-17T04:35:51.377378  /usr/bin/tpm2_getcap

    2023-03-17T04:36:01.595876  /lava-9660081/1/../bin/lava-test-case

    2023-03-17T04:36:01.602813  <8>[   20.762772] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee8b8b40448a808c864f

  Results:     18 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+amdgpu
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.bootrr.tpm-chip-is-online: https://kernelci.org/test/case/id/6413ee8b8b40448a808c8662
        failing since 1 day (last pass: next-20230315, first fail: next-20230316)

    2023-03-17T04:37:06.303814  /lava-9660212/1/../bin/lava-test-case<8>[   10.793853] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=pass>

    2023-03-17T04:37:06.304185  

    2023-03-17T04:37:06.306904  /usr/bin/tpm2_getcap

    2023-03-17T04:37:16.516719  /lava-9660212/1/../bin/lava-test-case<8>[   21.008579] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f134b0137e436a8c867f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f134b0137e436a8c8680
        failing since 206 days (last pass: next-20220810, first fail: next-20220822) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413ef1bb8661224048c8634

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ef1bb8661224048c8635
        failing since 339 days (last pass: next-20220401, first fail: next-20220411) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
jh7100-starfi...isionfive-v1 | riscv  | lab-clabbe      | gcc-10   | defconfig+kselftest          | 1          

  Details:     https://kernelci.org/test/plan/id/6413ed992e4922b9af8c8675

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+kselftest
  Compiler:    gcc-10 (riscv64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/riscv/defconfig+kselftest/gcc-10/lab-clabbe/baseline-jh7100-starfive-visionfive-v1.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/riscv/defconfig+kselftest/gcc-10/lab-clabbe/baseline-jh7100-starfive-visionfive-v1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/riscv/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ed992e4922b9af8c8676
        failing since 37 days (last pass: next-20230206, first fail: next-20230207) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
meson-gxl-s905x-khadas-vim   | arm64  | lab-baylibre    | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413ef17b8661224048c862f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+CONFIG_CPU_BIG_ENDIAN=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-meson-gxl-s905x-khadas-vim.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-meson-gxl-s905x-khadas-vim.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64be/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ef17b8661224048c8630
        new failure (last pass: next-20230316) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee9d4d0132c99b8c8658

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee9d4d0132c99b8c8659
        failing since 51 days (last pass: next-20230123, first fail: next-20230124) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee9ed860dab2048c8643

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee9ed860dab2048c8644
        failing since 36 days (last pass: next-20230206, first fail: next-20230208) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee9e8b40448a808c8685

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee9e8b40448a808c8686
        failing since 36 days (last pass: next-20230206, first fail: next-20230208) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          

  Details:     https://kernelci.org/test/plan/id/6413f22404755d07de8c86b2

  Results:     177 PASS, 9 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.clk-mt8192-pericfg-probed: https://kernelci.org/test/case/id/6413f22404755d07de8c8768
        failing since 42 days (last pass: next-20230130, first fail: next-20230202)

    2023-03-17T04:52:30.660511  /lava-9660367/1/../bin/lava-test-case

    2023-03-17T04:52:30.666999  <8>[   20.876971] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-infracfg-probed: https://kernelci.org/test/case/id/6413f22404755d07de8c8769
        failing since 42 days (last pass: next-20230130, first fail: next-20230202)

    2023-03-17T04:52:29.641468  /lava-9660367/1/../bin/lava-test-case

    2023-03-17T04:52:29.648140  <8>[   19.857652] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-infracfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-topckgen-probed: https://kernelci.org/test/case/id/6413f22404755d07de8c876a
        failing since 42 days (last pass: next-20230130, first fail: next-20230202)

    2023-03-17T04:52:28.621018  /lava-9660367/1/../bin/lava-test-case

    2023-03-17T04:52:28.627737  <8>[   18.837816] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-topckgen-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee0855b377cb5b8c8680

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: oxnas_v6_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee0855b377cb5b8c8681
        failing since 205 days (last pass: next-20220822, first fail: next-20220823) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64140657065801d5a08c8633

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64140657065801d5a08c8634
        failing since 36 days (last pass: next-20230206, first fail: next-20230208) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-vexpress-a9         | arm    | lab-baylibre    | gcc-10   | vexpress_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ece49a62d585108c8649

  Results:     3 PASS, 2 FAIL, 2 SKIP
  Full config: vexpress_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-baylibre/baseline-qemu_arm-vexpress-a9.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-baylibre/baseline-qemu_arm-vexpress-a9.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413ece49a62d585108c8651
        failing since 29 days (last pass: next-20230213, first fail: next-20230215)

    2023-03-17T04:29:37.271267  /lava-3418857/1/../bin/lava-test-case
    2023-03-17T04:29:37.363000  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-vexpress-a9         | arm    | lab-broonie     | gcc-10   | vexpress_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ed13e815ce8a9c8c86a6

  Results:     3 PASS, 2 FAIL, 2 SKIP
  Full config: vexpress_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-broonie/baseline-qemu_arm-vexpress-a9.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-broonie/baseline-qemu_arm-vexpress-a9.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413ed13e815ce8a9c8c86ae
        failing since 29 days (last pass: next-20230213, first fail: next-20230215)

    2023-03-17T04:31:05.073157  /lava-176031/1/../bin/lava-test-case
    2023-03-17T04:31:05.129053  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-vexpress-a9         | arm    | lab-collabora   | gcc-10   | vexpress_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ebcf465cde20858c863c

  Results:     3 PASS, 2 FAIL, 2 SKIP
  Full config: vexpress_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-collabora/baseline-qemu_arm-vexpress-a9.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/vexpress_defconfig/gcc-10/lab-collabora/baseline-qemu_arm-vexpress-a9.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413ebcf465cde20858c8644
        failing since 29 days (last pass: next-20230213, first fail: next-20230215)

    2023-03-17T04:25:44.526262  /lava-9659946/1/../bin/lava-test-case
    2023-03-17T04:25:44.526507  

    2023-03-17T04:25:44.554600  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>
    2023-03-17T04:25:44.554736  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f1d7f1e74df23d8c8631

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f1d7f1e74df23d8c8632
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f266a0743bc3748c8689

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f266a0743bc3748c868a
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f196251d025c5d8c868f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f196251d025c5d8c8690
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f1d5c7211516c08c86b7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f1d5c7211516c08c86b8
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f26371070b1ad48c8631

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f26371070b1ad48c8632
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f1d3c7211516c08c86b4

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f1d3c7211516c08c86b5
        failing since 182 days (last pass: next-20220907, first fail: next-20220915) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f23b71b5c0aaea8c8635

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f23b71b5c0aaea8c8636
        failing since 182 days (last pass: next-20220907, first fail: next-20220915) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f182f095dbd02e8c862f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f182f095dbd02e8c8630
        failing since 182 days (last pass: next-20220907, first fail: next-20220915) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f1d6c7211516c08c86ba

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f1d6c7211516c08c86bb
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/6413f2651d9208167c8c86f3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f2651d9208167c8c86f4
        failing since 184 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_mips-malta              | mips   | lab-collabora   | gcc-10   | malta_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/6413ed757c097878378c866b

  Results:     4 PASS, 1 FAIL, 2 SKIP
  Full config: malta_defconfig
  Compiler:    gcc-10 (mips-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/mips/malta_defconfig/gcc-10/lab-collabora/baseline-qemu_mips-malta.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/mips/malta_defconfig/gcc-10/lab-collabora/baseline-qemu_mips-malta.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/mipsel/rootfs.cpio.gz 

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/6413ed757c097878378c866f
        new failure (last pass: next-20230316)
        1 lines

    2023-03-17T04:32:42.763757  kern  :alert : CPU 0 Unable to handle kernel paging request at virtual address 29f8580c, epc == 80200a28, ra == 80203354
    2023-03-17T04:32:42.763889  

    2023-03-17T04:32:42.792111  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=1>
    2023-03-17T04:32:42.792230  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a7743-iwg20d-q7            | arm    | lab-cip         | gcc-10   | shmobile_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6413eddf2f1ebfa42d8c8644

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: shmobile_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413eddf2f1ebfa42d8c8645
        failing since 1 day (last pass: next-20230310, first fail: next-20230315) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+videodec           | 1          

  Details:     https://kernelci.org/test/plan/id/6413ed2adcc94598ef8c869b

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+videodec/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+videodec/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413ed2adcc94598ef8c86a3
        failing since 29 days (last pass: next-20230214, first fail: next-20230215)

    2023-03-17T04:31:30.427214  /lava-9660000/1/../bin/lava-test-case

    2023-03-17T04:31:30.427931  <8>[   32.975146] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

    2023-03-17T04:31:30.428350  /lava-9660000/1/../bin/lava-test-case

    2023-03-17T04:31:30.428839  <8>[   32.990953] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=skip>

    2023-03-17T04:31:30.429273  + set +x

    2023-03-17T04:31:30.429699  <8>[   33.004328] <LAVA_SIGNAL_ENDRUN 1_bootrr 9660000_1.5.2.4.5>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+ima                | 1          

  Details:     https://kernelci.org/test/plan/id/6413eda2bdee10ff8a8c8658

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+ima
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+ima/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+ima/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413eda2bdee10ff8a8c8660
        failing since 27 days (last pass: next-20230214, first fail: next-20230217)

    2023-03-17T04:33:16.963060  /lava-9660023/1/../bin/lava-test-case

    2023-03-17T04:33:16.963691  <8>[   36.562877] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

    2023-03-17T04:33:16.964134  /lava-9660023/1/../bin/lava-test-case

    2023-03-17T04:33:16.964607  <8>[   36.578614] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=skip>

    2023-03-17T04:33:16.965027  + set +x

    2023-03-17T04:33:16.965440  <8>[   36.592235] <LAVA_SIGNAL_ENDRUN 1_bootrr 9660023_1.5.2.4.5>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+crypto             | 1          

  Details:     https://kernelci.org/test/plan/id/6413f2a5936ee790098c8660

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+crypto
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+crypto/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+crypto/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413f2a5936ee790098c8668
        failing since 29 days (last pass: next-20230214, first fail: next-20230215)

    2023-03-17T04:54:44.198789  /lava-9660395/1/../bin/lava-test-case

    2023-03-17T04:54:44.199424  <8>[   37.014689] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

    2023-03-17T04:54:44.199808  /lava-9660395/1/../bin/lava-test-case

    2023-03-17T04:54:44.200174  <8>[   37.030243] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=skip>

    2023-03-17T04:54:44.200561  + set +x

    2023-03-17T04:54:44.200926  <8>[   37.043698] <LAVA_SIGNAL_ENDRUN 1_bootrr 9660395_1.5.2.4.5>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6413f380e147ec17148c8637

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413f380e147ec17148c8640
        failing since 29 days (last pass: next-20230214, first fail: next-20230215)

    2023-03-17T04:58:27.991210  /lava-9660409/1/../bin/lava-test-case

    2023-03-17T04:58:28.093143  <8>[   54.685533][  T347] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig                    | 1          

  Details:     https://kernelci.org/test/plan/id/6413f3a9c6a691970d8c87dd

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413f3a9c6a691970d8c87e5
        failing since 29 days (last pass: next-20230214, first fail: next-20230215)

    2023-03-17T04:59:12.917510  /lava-9660435/1/../bin/lava-test-case

    2023-03-17T04:59:12.965560  <8>[   33.392976] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

    2023-03-17T04:59:12.965797  /lava-9660435/1/../bin/lava-test-case

    2023-03-17T04:59:12.965991  <8>[   33.408682] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=skip>

    2023-03-17T04:59:12.966179  + set +x

    2023-03-17T04:59:12.966364  <8>[   33.422177] <LAVA_SIGNAL_ENDRUN 1_bootrr 9660435_1.5.2.4.5>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a77960-ulcb                | arm64  | lab-collabora   | gcc-10   | defconfig+CON...OMIZE_BASE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413f4861dbbbd87778c865f

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+CONFIG_RANDOMIZE_BASE=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-10/lab-collabora/baseline-r8a77960-ulcb.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-10/lab-collabora/baseline-r8a77960-ulcb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6413f4861dbbbd87778c8667
        failing since 29 days (last pass: next-20230214, first fail: next-20230215)

    2023-03-17T05:02:49.968445  /lava-9660468/1/../bin/lava-test-case

    2023-03-17T05:02:50.016125  <8>[   33.961324] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>

    2023-03-17T05:02:50.016380  /lava-9660468/1/../bin/lava-test-case

    2023-03-17T05:02:50.016574  <8>[   33.976842] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=skip>

    2023-03-17T05:02:50.016783  + set +x

    2023-03-17T05:02:50.016971  <8>[   33.990360] <LAVA_SIGNAL_ENDRUN 1_bootrr 9660468_1.5.2.4.5>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413f1631128e547d68c866e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413f1631128e547d68c866f
        failing since 114 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee6cd0155bcfb18c864c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee6cd0155bcfb18c864d
        failing since 114 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413ee9c8b40448a808c8669

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ee9c8b40448a808c866a
        failing since 36 days (last pass: next-20230206, first fail: next-20230208) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3399-roc-pc                | arm64  | lab-clabbe      | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6413f4793dfc9488c48c8641

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-clabbe/baseline-rk3399-roc-pc.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-clabbe/baseline-rk3399-roc-pc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413f4793dfc9488c48c864a
        failing since 70 days (last pass: next-20221130, first fail: next-20230105)

    2023-03-17T05:02:27.308371  + set +x
    2023-03-17T05:02:27.309820  <8>[   59.370885][  T194] <LAVA_SIGNAL_ENDRUN 0_dmesg 414068_1.5.2.4.1>
    2023-03-17T05:02:27.463625  #
    2023-03-17T05:02:27.565745  / # #export SHELL=/bin/sh
    2023-03-17T05:02:27.566273  
    2023-03-17T05:02:27.667472  / # export SHELL=/bin/sh. /lava-414068/environment
    2023-03-17T05:02:27.668024  
    2023-03-17T05:02:27.769389  / # . /lava-414068/environment/lava-414068/bin/lava-test-runner /lava-414068/1
    2023-03-17T05:02:27.770138  
    2023-03-17T05:02:27.778320  / # /lava-414068/bin/lava-test-runner /lava-414068/1 
    ... (43 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/6413eeb1ac459528668c8646

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413eeb1ac459528668c8647
        failing since 36 days (last pass: next-20230206, first fail: next-20230208) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/6413ef2c59d6e2fcce8c8631

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413ef2c59d6e2fcce8c863a
        failing since 19 days (last pass: next-20230224, first fail: next-20230225)

    2023-03-17T04:39:45.374994  / # #
    2023-03-17T04:39:45.481115  export SHELL=/bin/sh
    2023-03-17T04:39:45.482784  #
    2023-03-17T04:39:45.587265  / # export SHELL=/bin/sh. /lava-3418780/environment
    2023-03-17T04:39:45.589193  
    2023-03-17T04:39:45.693014  / # . /lava-3418780/environment/lava-3418780/bin/lava-test-runner /lava-3418780/1
    2023-03-17T04:39:45.695948  
    2023-03-17T04:39:45.712547  / # /lava-3418780/bin/lava-test-runner /lava-3418780/1
    2023-03-17T04:39:45.827180  + export 'TESTRUN_ID=1_bootrr'
    2023-03-17T04:39:45.830419  + cd /lava-3418780/1/tests/1_bootrr 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun8i-h3-libretech-all-h3-cc | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/6413ea9cbbb8f9228f8c867c

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h3-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413ea9cbbb8f9228f8c8685
        failing since 51 days (last pass: next-20230123, first fail: next-20230124)

    2023-03-17T04:20:19.560484  / # #
    2023-03-17T04:20:19.662190  export SHELL=/bin/sh
    2023-03-17T04:20:19.662701  #
    2023-03-17T04:20:19.764096  / # export SHELL=/bin/sh. /lava-3418778/environment
    2023-03-17T04:20:19.764649  
    2023-03-17T04:20:19.866033  / # . /lava-3418778/environment/lava-3418778/bin/lava-test-runner /lava-3418778/1
    2023-03-17T04:20:19.866799  
    2023-03-17T04:20:19.871609  / # /lava-3418778/bin/lava-test-runner /lava-3418778/1
    2023-03-17T04:20:20.019520  + export 'TESTRUN_ID=1_bootrr'
    2023-03-17T04:20:20.020127  + cd /lava-3418778/1/tests/1_bootrr 
    ... (10 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6413ef8440315862288c8630

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6413ef8440315862288c8631
        failing since 113 days (last pass: next-20221121, first fail: next-20221123) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
zynqmp-zcu102                | arm64  | lab-cip         | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6413f30e07956f19ed8c863d

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-cip/baseline-zynqmp-zcu102.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20230317/arm64/defconfig+debug/gcc-10/lab-cip/baseline-zynqmp-zcu102.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6413f30e07956f19ed8c8644
        failing since 70 days (last pass: next-20221207, first fail: next-20230105)

    2023-03-17T04:56:34.449858  + set +x
    2023-03-17T04:56:34.450916  <8>[   39.773947][  T170] <LAVA_SIGNAL_ENDRUN 0_dmesg 878161_1.5.2.4.1>
    2023-03-17T04:56:34.580813  #
    2023-03-17T04:56:34.682971  / # #export SHELL=/bin/sh
    2023-03-17T04:56:34.683538  
    2023-03-17T04:56:34.784953  / # export SHELL=/bin/sh. /lava-878161/environment
    2023-03-17T04:56:34.785429  
    2023-03-17T04:56:34.886843  / # . /lava-878161/environment/lava-878161/bin/lava-test-runner /lava-878161/1
    2023-03-17T04:56:34.887687  
    2023-03-17T04:56:34.892039  / # /lava-878161/bin/lava-test-runner /lava-878161/1 
    ... (18 line(s) more)