Subject: next/master baseline: 409 runs, 44 regressions (next-20231212)

From: "kernelci.org bot" <[email-masked]>
Date: Tue, 12 Dec 2023 05:26:50 -0800 (PST)

next/master baseline: 409 runs, 44 regressions (next-20231212)

Regressions Summary
-------------------

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
at91-sama5d4_xplained        | arm   | lab-baylibre  | gcc-10   | sama5_defconfig              | 1          
at91sam9g20ek                | arm   | lab-broonie   | gcc-10   | at91_dt_defconfig            | 1          
at91sam9g20ek                | arm   | lab-broonie   | gcc-10   | multi_v5_defconfig           | 1          
bcm2836-rpi-2-b              | arm   | lab-collabora | gcc-10   | bcm2835_defconfig            | 1          
bcm2836-rpi-2-b              | arm   | lab-collabora | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
beagle-xm                    | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
beagle-xm                    | arm   | lab-baylibre  | gcc-10   | omap2plus_defconfig          | 1          
beaglebone-black             | arm   | lab-broonie   | gcc-10   | multi_v7_defconfig           | 1          
beaglebone-black             | arm   | lab-cip       | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
jetson-tk1                   | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
kontron-kbox-a-230-ls        | arm64 | lab-kontron   | gcc-10   | defconfig+videodec           | 5          
kontron-kbox-a-230-ls        | arm64 | lab-kontron   | gcc-10   | defconfig                    | 5          
kontron-sl28-var3-ads2       | arm64 | lab-kontron   | gcc-10   | defconfig+videodec           | 2          
kontron-sl28-var3-ads2       | arm64 | lab-kontron   | gcc-10   | defconfig                    | 2          
meson-gxm-khadas-vim2        | arm64 | lab-baylibre  | gcc-10   | defconfig                    | 1          
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defconfig           | 1          
qemu_arm64-virt-gicv3        | arm64 | lab-baylibre  | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv3        | arm64 | lab-collabora | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv3-uefi   | arm64 | lab-baylibre  | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv3-uefi   | arm64 | lab-collabora | gcc-10   | defconfig+debug              | 1          
r8a7743-iwg20d-q7            | arm   | lab-cip       | gcc-10   | shmobile_defconfig           | 1          
rk3288-rock2-square          | arm   | lab-collabora | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3288-veyron-jaq            | arm   | lab-collabora | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | clang-17 | multi_v7_defconfig           | 1          
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | clang-17 | multi_v7_defconfig           | 1          
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defconfig           | 1          
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defconfig           | 1          
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre  | gcc-10   | sunxi_defconfig              | 1          

  Details:  https://kernelci.org/test/job/next/branch/master/kernel/next-20231212/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   master
  Describe: next-20231212
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      abb240f7a2bd14567ab53e602db562bb683391e6 

Test Regressions
---------------- 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
at91-sama5d4_xplained        | arm   | lab-baylibre  | gcc-10   | sama5_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/657824cfbaba6b0eb9e1349a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: sama5_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/sama5_defconfig/gcc-10/lab-baylibre/baseline-at91-sama5d4_xplained.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/sama5_defconfig/gcc-10/lab-baylibre/baseline-at91-sama5d4_xplained.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/657824cfbaba6b0eb9e1349b
        failing since 314 days (last pass: next-20230125, first fail: next-20230131) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
at91sam9g20ek                | arm   | lab-broonie   | gcc-10   | at91_dt_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/65781edcef1087b869e13478

  Results:     52 PASS, 2 FAIL, 1 SKIP
  Full config: at91_dt_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/at91_dt_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/at91_dt_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.atmel-ramc-driver-present: https://kernelci.org/test/case/id/65781eddef1087b869e13498
        failing since 190 days (last pass: next-20230601, first fail: next-20230605)

    2023-12-12T08:50:10.067580  /lava-344165/1/../bin/lava-test-case
    2023-12-12T08:50:10.109385  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=atmel-ramc-driver-present RESULT=fail>   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
at91sam9g20ek                | arm   | lab-broonie   | gcc-10   | multi_v5_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/657825fa0fa81eb8f3e134f3

  Results:     52 PASS, 2 FAIL, 1 SKIP
  Full config: multi_v5_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v5_defconfig/gcc-10/lab-broonie/baseline-at91sam9g20ek.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.atmel-ramc-driver-present: https://kernelci.org/test/case/id/657825fa0fa81eb8f3e13512
        failing since 190 days (last pass: next-20230601, first fail: next-20230605)

    2023-12-12T09:20:48.417245  /lava-344273/1/../bin/lava-test-case
    2023-12-12T09:20:48.460103  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=atmel-ramc-driver-present RESULT=fail>   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
bcm2836-rpi-2-b              | arm   | lab-collabora | gcc-10   | bcm2835_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/65781f0fdd610948e0e1347c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: bcm2835_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65781f0fdd610948e0e1347d
        failing since 190 days (last pass: next-20230601, first fail: next-20230605) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
bcm2836-rpi-2-b              | arm   | lab-collabora | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/657826a4a4cdc27cafe134af

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/657826a4a4cdc27cafe134b0
        failing since 258 days (last pass: next-20230327, first fail: next-20230328) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
beagle-xm                    | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/6578271191268977efe13477

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-beagle-xm.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-beagle-xm.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6578271191268977efe13478
        failing since 256 days (last pass: next-20230330, first fail: next-20230331) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
beagle-xm                    | arm   | lab-baylibre  | gcc-10   | omap2plus_defconfig          | 1          

  Details:     https://kernelci.org/test/plan/id/6578278b583fbc4befe1347f

  Results:     3 PASS, 1 FAIL, 1 SKIP
  Full config: omap2plus_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/omap2plus_defconfig/gcc-10/lab-baylibre/baseline-beagle-xm.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.all-cpus-are-online: https://kernelci.org/test/case/id/6578278b583fbc4befe13485
        new failure (last pass: next-20231211)

    2023-12-12T09:27:31.651846  cat: symbol lookup error: /lib/libc.so.6: undefined symbol: __po<1>[   50.300506] BUG: Bad rss-counter state mm:3adb5ca9 type:MM_FILEPAGES val:-3
    2023-12-12T09:27:31.652495  inter_chk_guard, version 
    2023-12-12T09:27:32.717421  /lava-3876494/1/../bin/lava-test-case
    2023-12-12T09:27:32.746599  <8>[   51.402893] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=fail>   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
beaglebone-black             | arm   | lab-broonie   | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/65782bb0c6543a9778e13475

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-broonie/baseline-beaglebone-black.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65782bb0c6543a9778e13476
        failing since 4 days (last pass: next-20231207, first fail: next-20231208) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
beaglebone-black             | arm   | lab-cip       | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/6578266c1a646319c7e13499

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-cip/baseline-beaglebone-black.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-cip/baseline-beaglebone-black.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6578266c1a646319c7e1349a
        failing since 4 days (last pass: next-20231207, first fail: next-20231208) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
jetson-tk1                   | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/657820a91295fc1279e13653

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/657820a91295fc1279e13654
        failing since 609 days (last pass: next-20220401, first fail: next-20220411) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
kontron-kbox-a-230-ls        | arm64 | lab-kontron   | gcc-10   | defconfig+videodec           | 5          

  Details:     https://kernelci.org/test/plan/id/6578227f02f32e4750e13484

  Results:     90 PASS, 5 FAIL, 1 SKIP
  Full config: defconfig+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578227f02f32e4750e13487
        new failure (last pass: next-20231207)

    2023-12-12T09:05:53.626911  / # #
    2023-12-12T09:05:53.729057  export SHELL=/bin/sh
    2023-12-12T09:05:53.729775  #
    2023-12-12T09:05:53.831247  / # export SHELL=/bin/sh. /lava-404665/environment
    2023-12-12T09:05:53.831975  
    2023-12-12T09:05:53.933421  / # . /lava-404665/environment/lava-404665/bin/lava-test-runner /lava-404665/1
    2023-12-12T09:05:53.934496  
    2023-12-12T09:05:53.953284  / # /lava-404665/bin/lava-test-runner /lava-404665/1
    2023-12-12T09:05:53.997196  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:05:53.997623  + <8>[   20.693508] <LAVA_SIGNAL_STARTRUN 1_bootrr 404665_1.5.2.4.5> 
    ... (15 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc2-probed: https://kernelci.org/test/case/id/6578227f02f32e4750e1348b
        new failure (last pass: next-20231207)

    2023-12-12T09:05:56.106052  /lava-404665/1/../bin/lava-test-case
    2023-12-12T09:05:56.106479  <8>[   22.803888] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc2-probed RESULT=fail>
    2023-12-12T09:05:56.106903  /lava-404665/1/../bin/lava-test-case   

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/6578227f02f32e4750e1348d
        new failure (last pass: next-20231207)

    2023-12-12T09:05:57.166272  /lava-404665/1/../bin/lava-test-case
    2023-12-12T09:05:57.166716  <8>[   23.842477] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>
    2023-12-12T09:05:57.167017  /lava-404665/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/6578227f02f32e4750e13492
        new failure (last pass: next-20231207)

    2023-12-12T09:05:58.241852  /lava-404665/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc1-probed: https://kernelci.org/test/case/id/6578227f02f32e4750e13493
        new failure (last pass: next-20231207)

    2023-12-12T09:05:58.245111  <8>[   24.956495] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-12-12T09:05:59.303946  /lava-404665/1/../bin/lava-test-case
    2023-12-12T09:05:59.304396  <8>[   25.978199] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc1-probed RESULT=fail>
    2023-12-12T09:05:59.304733  /lava-404665/1/../bin/lava-test-case   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
kontron-kbox-a-230-ls        | arm64 | lab-kontron   | gcc-10   | defconfig                    | 5          

  Details:     https://kernelci.org/test/plan/id/65782d50536a775951e134b4

  Results:     90 PASS, 5 FAIL, 1 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782d50536a775951e134b7
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:54.283932  <8>[   20.129039] <LAVA_SIGNAL_ENDRUN 0_dmesg 404698_1.5.2.4.1>
    2023-12-12T09:51:54.388948  / # #
    2023-12-12T09:51:54.491052  export SHELL=/bin/sh
    2023-12-12T09:51:54.491748  #
    2023-12-12T09:51:54.592961  / # export SHELL=/bin/sh. /lava-404698/environment
    2023-12-12T09:51:54.593667  
    2023-12-12T09:51:54.694984  / # . /lava-404698/environment/lava-404698/bin/lava-test-runner /lava-404698/1
    2023-12-12T09:51:54.695944  
    2023-12-12T09:51:54.715126  / # /lava-404698/bin/lava-test-runner /lava-404698/1
    2023-12-12T09:51:54.759007  + export 'TESTRUN_ID=1_bootrr' 
    ... (15 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc2-probed: https://kernelci.org/test/case/id/65782d50536a775951e134bb
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:56.868403  /lava-404698/1/../bin/lava-test-case
    2023-12-12T09:51:56.868607  <8>[   22.695458] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc2-probed RESULT=fail>
    2023-12-12T09:51:56.868814  /lava-404698/1/../bin/lava-test-case   

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/65782d50536a775951e134bd
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:57.926527  /lava-404698/1/../bin/lava-test-case
    2023-12-12T09:51:57.926977  <8>[   23.734250] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>
    2023-12-12T09:51:57.927330  /lava-404698/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/65782d50536a775951e134c2
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:59.002532  /lava-404698/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc1-probed: https://kernelci.org/test/case/id/65782d50536a775951e134c3
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:59.005661  <8>[   24.847434] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-12-12T09:52:00.065076  /lava-404698/1/../bin/lava-test-case
    2023-12-12T09:52:00.065542  <8>[   25.869160] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc1-probed RESULT=fail>
    2023-12-12T09:52:00.065831  /lava-404698/1/../bin/lava-test-case   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
kontron-sl28-var3-ads2       | arm64 | lab-kontron   | gcc-10   | defconfig+videodec           | 2          

  Details:     https://kernelci.org/test/plan/id/6578226b392bdc6129e13480

  Results:     101 PASS, 2 FAIL, 1 SKIP
  Full config: defconfig+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-sl28-var3-ads2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-sl28-var3-ads2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578226b392bdc6129e13483
        new failure (last pass: next-20231207)

    2023-12-12T09:05:35.334460  / # #
    2023-12-12T09:05:35.436734  export SHELL=/bin/sh
    2023-12-12T09:05:35.437476  #
    2023-12-12T09:05:35.539069  / # export SHELL=/bin/sh. /lava-404666/environment
    2023-12-12T09:05:35.539803  
    2023-12-12T09:05:35.641193  / # . /lava-404666/environment/lava-404666/bin/lava-test-runner /lava-404666/1
    2023-12-12T09:05:35.642342  
    2023-12-12T09:05:35.661520  / # /lava-404666/bin/lava-test-runner /lava-404666/1
    2023-12-12T09:05:35.715173  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:05:35.715337  + <8>[   21.162763] <LAVA_SIGNAL_STARTRUN 1_bootrr 404666_1.5.2.4.5> 
    ... (10 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/6578226b392bdc6129e13496
        new failure (last pass: next-20231207)

    2023-12-12T09:05:38.154605  /lava-404666/1/../bin/lava-test-case   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
kontron-sl28-var3-ads2       | arm64 | lab-kontron   | gcc-10   | defconfig                    | 2          

  Details:     https://kernelci.org/test/plan/id/65782d37c9d56e1879e13484

  Results:     101 PASS, 2 FAIL, 1 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-sl28-var3-ads2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-sl28-var3-ads2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782d38c9d56e1879e13487
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:15.856489  / # #
    2023-12-12T09:51:15.957145  export SHELL=/bin/sh
    2023-12-12T09:51:15.957395  #
    2023-12-12T09:51:16.057909  / # export SHELL=/bin/sh. /lava-404696/environment
    2023-12-12T09:51:16.058169  
    2023-12-12T09:51:16.158705  / # . /lava-404696/environment/lava-404696/bin/lava-test-runner /lava-404696/1
    2023-12-12T09:51:16.159065  
    2023-12-12T09:51:16.171885  / # /lava-404696/bin/lava-test-runner /lava-404696/1
    2023-12-12T09:51:16.231825  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:51:16.231932  + <8>[   20.958945] <LAVA_SIGNAL_STARTRUN 1_bootrr 404696_1.5.2.4.5> 
    ... (10 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/65782d38c9d56e1879e1349a
        failing since 1 day (last pass: next-20231207, first fail: next-20231211)

    2023-12-12T09:51:18.667558  /lava-404696/1/../bin/lava-test-case
    2023-12-12T09:51:18.668193  <8>[   23.383407] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-12-12T09:51:18.668459  /lava-404696/1/../bin/lava-test-case
    2023-12-12T09:51:18.668759  <8>[   23.401156] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc_mdio-driver-present RESULT=pass>
    2023-12-12T09:51:18.669028  /lava-404696/1/../bin/lava-test-case
    2023-12-12T09:51:18.669282  <8>[   23.420595] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc_mdio-probed RESULT=pass>
    2023-12-12T09:51:18.669537  /lava-404696/1/../bin/lava-test-case   

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
meson-gxm-khadas-vim2        | arm64 | lab-baylibre  | gcc-10   | defconfig                    | 1          

  Details:     https://kernelci.org/test/plan/id/65782e3452d23e1fcce134d4

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65782e3452d23e1fcce134d5
        new failure (last pass: next-20231211) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/657824849597a865fae134af

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-panda.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-panda.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/657824849597a865fae134b8
        failing since 5 days (last pass: next-20221017, first fail: next-20231207)

    2023-12-12T09:14:18.363731  <8>[   19.957092] <LAVA_SIGNAL_ENDRUN 0_dmesg 3876416_1.5.2.4.1>
    2023-12-12T09:14:18.468605  / # #
    2023-12-12T09:14:18.569821  export SHELL=/bin/sh
    2023-12-12T09:14:18.570155  #
    2023-12-12T09:14:18.670912  / # export SHELL=/bin/sh. /lava-3876416/environment
    2023-12-12T09:14:18.671275  
    2023-12-12T09:14:18.772086  / # . /lava-3876416/environment/lava-3876416/bin/lava-test-runner /lava-3876416/1
    2023-12-12T09:14:18.772707  
    2023-12-12T09:14:18.777361  / # /lava-3876416/bin/lava-test-runner /lava-3876416/1
    2023-12-12T09:14:18.828621  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/6578255aa4fab3383ce13486

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-panda.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-panda.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578255aa4fab3383ce1348f
        failing since 5 days (last pass: next-20221014, first fail: next-20231207)

    2023-12-12T09:17:52.847805  <8>[   14.531036] <LAVA_SIGNAL_ENDRUN 0_dmesg 3876440_1.5.2.4.1>
    2023-12-12T09:17:52.954433  / # #
    2023-12-12T09:17:53.055687  export SHELL=/bin/sh
    2023-12-12T09:17:53.056069  #
    2023-12-12T09:17:53.156863  / # export SHELL=/bin/sh. /lava-3876440/environment
    2023-12-12T09:17:53.157301  
    2023-12-12T09:17:53.258098  / # . /lava-3876440/environment/lava-3876440/bin/lava-test-runner /lava-3876440/1
    2023-12-12T09:17:53.258722  
    2023-12-12T09:17:53.263782  / # /lava-3876440/bin/lava-test-runner /lava-3876440/1
    2023-12-12T09:17:53.328475  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
panda                        | arm   | lab-baylibre  | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/65782b28af3533d791e13536

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-panda.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-panda.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782b28af3533d791e1353f
        failing since 5 days (last pass: next-20221014, first fail: next-20231207)

    2023-12-12T09:42:36.328976  <8>[   14.508239] <LAVA_SIGNAL_ENDRUN 0_dmesg 3876552_1.5.2.4.1>
    2023-12-12T09:42:36.435842  / # #
    2023-12-12T09:42:36.537016  export SHELL=/bin/sh
    2023-12-12T09:42:36.537376  #
    2023-12-12T09:42:36.638178  / # export SHELL=/bin/sh. /lava-3876552/environment
    2023-12-12T09:42:36.638534  
    2023-12-12T09:42:36.739353  / # . /lava-3876552/environment/lava-3876552/bin/lava-test-runner /lava-3876552/1
    2023-12-12T09:42:36.739966  
    2023-12-12T09:42:36.745269  / # /lava-3876552/bin/lava-test-runner /lava-3876552/1
    2023-12-12T09:42:36.805741  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3        | arm64 | lab-baylibre  | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6578267af1eb15a742e134b3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6578267af1eb15a742e134b4
        new failure (last pass: next-20231211) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3        | arm64 | lab-collabora | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/65782656f1eb15a742e13475

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65782656f1eb15a742e13476
        new failure (last pass: next-20231211) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3-uefi   | arm64 | lab-baylibre  | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/65782678f1eb15a742e134ae

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65782678f1eb15a742e134af
        new failure (last pass: next-20231211) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3-uefi   | arm64 | lab-collabora | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/6578266af1eb15a742e134a6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm64/defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm64-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6578266af1eb15a742e134a7
        new failure (last pass: next-20231211) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
r8a7743-iwg20d-q7            | arm   | lab-cip       | gcc-10   | shmobile_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/65781fa200616be763e134ad

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: shmobile_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/65781fa200616be763e134ae
        failing since 5 days (last pass: next-20231205, first fail: next-20231206) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
rk3288-rock2-square          | arm   | lab-collabora | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/657821a33282ea75b1e13476

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/657821a33282ea75b1e13477
        failing since 384 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
rk3288-veyron-jaq            | arm   | lab-collabora | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/657820a0006b05aecfe13493

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/657820a0006b05aecfe13494
        failing since 384 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | clang-17 | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6578283f408099969ce134a8

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    clang-17 (Debian clang version 17.0.6 (++20231208085823+6009708b4367-1~exp1~20231208085918.75))
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/clang-17/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/clang-17/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578283f408099969ce134b1
        failing since 188 days (last pass: next-20230525, first fail: next-20230607)

    2023-12-12T09:30:32.192147  / # #
    2023-12-12T09:30:32.295277  export SHELL=/bin/sh
    2023-12-12T09:30:32.296353  #
    2023-12-12T09:30:32.398301  / # export SHELL=/bin/sh. /lava-3876515/environment
    2023-12-12T09:30:32.399687  
    2023-12-12T09:30:32.501289  / # . /lava-3876515/environment/lava-3876515/bin/lava-test-runner /lava-3876515/1
    2023-12-12T09:30:32.502647  
    2023-12-12T09:30:32.514818  / # /lava-3876515/bin/lava-test-runner /lava-3876515/1
    2023-12-12T09:30:32.611107  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:30:32.645852  + cd /lava-3876515/1/tests/1_bootrr 
    ... (10 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | clang-17 | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/657828aeef43fdf85ae13476

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    clang-17 (Debian clang version 17.0.6 (++20231208085823+6009708b4367-1~exp1~20231208085918.75))
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/clang-17/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/clang-17/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/657828afef43fdf85ae1347f
        failing since 188 days (last pass: next-20230525, first fail: next-20230607)

    2023-12-12T09:32:10.345793  + set +x
    2023-12-12T09:32:10.347496  [   21.703237] <LAVA_SIGNAL_ENDRUN 0_dmesg 447728_1.5.2.4.1>
    2023-12-12T09:32:10.455889  / # #
    2023-12-12T09:32:10.557471  export SHELL=/bin/sh
    2023-12-12T09:32:10.558090  #
    2023-12-12T09:32:10.659063  / # export SHELL=/bin/sh. /lava-447728/environment
    2023-12-12T09:32:10.659629  
    2023-12-12T09:32:10.760618  / # . /lava-447728/environment/lava-447728/bin/lava-test-runner /lava-447728/1
    2023-12-12T09:32:10.761468  
    2023-12-12T09:32:10.764382  / # /lava-447728/bin/lava-test-runner /lava-447728/1 
    ... (12 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/65782455742c76006ee13499

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782456742c76006ee134a2
        failing since 194 days (last pass: next-20230530, first fail: next-20230601)

    2023-12-12T09:13:38.243512  / # #
    2023-12-12T09:13:38.347010  export SHELL=/bin/sh
    2023-12-12T09:13:38.348294  #
    2023-12-12T09:13:38.450326  / # export SHELL=/bin/sh. /lava-3876407/environment
    2023-12-12T09:13:38.451534  
    2023-12-12T09:13:38.553676  / # . /lava-3876407/environment/lava-3876407/bin/lava-test-runner /lava-3876407/1
    2023-12-12T09:13:38.555186  
    2023-12-12T09:13:38.566851  / # /lava-3876407/bin/lava-test-runner /lava-3876407/1
    2023-12-12T09:13:38.692228  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:13:38.692653  + cd /lava-3876407/1/tests/1_bootrr 
    ... (10 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/6578268bf72116d2d4e13477

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578268bf72116d2d4e13480
        failing since 188 days (last pass: next-20230517, first fail: next-20230607)

    2023-12-12T09:23:01.198064  <8>[   13.404844] <LAVA_SIGNAL_ENDRUN 0_dmesg 3876438_1.5.2.4.1>
    2023-12-12T09:23:01.305943  / # #
    2023-12-12T09:23:01.408996  export SHELL=/bin/sh
    2023-12-12T09:23:01.410175  #
    2023-12-12T09:23:01.512453  / # export SHELL=/bin/sh. /lava-3876438/environment
    2023-12-12T09:23:01.513553  
    2023-12-12T09:23:01.615393  / # . /lava-3876438/environment/lava-3876438/bin/lava-test-runner /lava-3876438/1
    2023-12-12T09:23:01.617168  
    2023-12-12T09:23:01.629614  / # /lava-3876438/bin/lava-test-runner /lava-3876438/1
    2023-12-12T09:23:01.750137  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-baylibre  | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/65782aab13240201f6e134dd

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782aab13240201f6e134e6
        failing since 194 days (last pass: next-20230530, first fail: next-20230601)

    2023-12-12T09:40:45.627270  + set +x<8>[   14.456400] <LAVA_SIGNAL_ENDRUN 0_dmesg 3876551_1.5.2.4.1>
    2023-12-12T09:40:45.627425  
    2023-12-12T09:40:45.730131  / # #
    2023-12-12T09:40:45.831172  export SHELL=/bin/sh
    2023-12-12T09:40:45.831534  #
    2023-12-12T09:40:45.932263  / # export SHELL=/bin/sh. /lava-3876551/environment
    2023-12-12T09:40:45.932630  
    2023-12-12T09:40:46.033362  / # . /lava-3876551/environment/lava-3876551/bin/lava-test-runner /lava-3876551/1
    2023-12-12T09:40:46.033881  
    2023-12-12T09:40:46.075977  / # /lava-3876551/bin/lava-test-runner /lava-3876551/1 
    ... (12 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/6578244d6c37de8e18e134bd

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578244d6c37de8e18e134c6
        failing since 194 days (last pass: next-20230530, first fail: next-20230601)

    2023-12-12T09:13:31.486515  + set +x
    2023-12-12T09:13:31.488180  [   24.007060] <LAVA_SIGNAL_ENDRUN 0_dmesg 447706_1.5.2.4.1>
    2023-12-12T09:13:31.595841  / # #
    2023-12-12T09:13:31.697506  export SHELL=/bin/sh
    2023-12-12T09:13:31.698096  #
    2023-12-12T09:13:31.799091  / # export SHELL=/bin/sh. /lava-447706/environment
    2023-12-12T09:13:31.799687  
    2023-12-12T09:13:31.900679  / # . /lava-447706/environment/lava-447706/bin/lava-test-runner /lava-447706/1
    2023-12-12T09:13:31.901558  
    2023-12-12T09:13:31.904160  / # /lava-447706/bin/lava-test-runner /lava-447706/1 
    ... (12 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/6578267da4cdc27cafe13483

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/6578267da4cdc27cafe1348c
        failing since 188 days (last pass: next-20230517, first fail: next-20230607)

    2023-12-12T09:22:51.209980  [   19.379755] <LAVA_SIGNAL_ENDRUN 0_dmesg 447716_1.5.2.4.1>
    2023-12-12T09:22:51.318849  / # #
    2023-12-12T09:22:51.420471  export SHELL=/bin/sh
    2023-12-12T09:22:51.421109  #
    2023-12-12T09:22:51.522233  / # export SHELL=/bin/sh. /lava-447716/environment
    2023-12-12T09:22:51.522892  
    2023-12-12T09:22:51.623958  / # . /lava-447716/environment/lava-447716/bin/lava-test-runner /lava-447716/1
    2023-12-12T09:22:51.624965  
    2023-12-12T09:22:51.628168  / # /lava-447716/bin/lava-test-runner /lava-447716/1
    2023-12-12T09:22:51.712961  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun7i-a20-cubieboard2        | arm   | lab-clabbe    | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/65782aa213240201f6e134d2

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/multi_v7_defconfig/gcc-10/lab-clabbe/baseline-sun7i-a20-cubieboard2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/65782aa213240201f6e134db
        failing since 194 days (last pass: next-20230530, first fail: next-20230601)

    2023-12-12T09:40:34.162358  + set +x
    2023-12-12T09:40:34.164197  [   21.214026] <LAVA_SIGNAL_ENDRUN 0_dmesg 447732_1.5.2.4.1>
    2023-12-12T09:40:34.274571  / # #
    2023-12-12T09:40:34.375917  export SHELL=/bin/sh
    2023-12-12T09:40:34.376424  #
    2023-12-12T09:40:34.477373  / # export SHELL=/bin/sh. /lava-447732/environment
    2023-12-12T09:40:34.477900  
    2023-12-12T09:40:34.578903  / # . /lava-447732/environment/lava-447732/bin/lava-test-runner /lava-447732/1
    2023-12-12T09:40:34.579986  
    2023-12-12T09:40:34.583534  / # /lava-447732/bin/lava-test-runner /lava-447732/1 
    ... (12 line(s) more)  

platform                     | arch  | lab           | compiler | defconfig                    | regressions
-----------------------------+-------+---------------+----------+------------------------------+------------
sun8i-h2-plus...ch-all-h3-cc | arm   | lab-baylibre  | gcc-10   | sunxi_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/657827119fa5d42b70e13476

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20231212/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20231212/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-libretech-all-h3-cc.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230623.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/657827119fa5d42b70e1347f
        failing since 1 day (last pass: next-20231208, first fail: next-20231211)

    2023-12-12T09:24:43.942382  / # #
    2023-12-12T09:24:44.043619  export SHELL=/bin/sh
    2023-12-12T09:24:44.043997  #
    2023-12-12T09:24:44.144851  / # export SHELL=/bin/sh. /lava-3876492/environment
    2023-12-12T09:24:44.145230  
    2023-12-12T09:24:44.246064  / # . /lava-3876492/environment/lava-3876492/bin/lava-test-runner /lava-3876492/1
    2023-12-12T09:24:44.246682  
    2023-12-12T09:24:44.254008  / # /lava-3876492/bin/lava-test-runner /lava-3876492/1
    2023-12-12T09:24:44.397892  + export 'TESTRUN_ID=1_bootrr'
    2023-12-12T09:24:44.398362  + cd /lava-3876492/1/tests/1_bootrr 
    ... (10 line(s) more)