Subject: next/pending-fixes baseline: 663 runs, 61 regressions (v6.3-rc3-248-g01078db9d658)

From: "kernelci.org bot" <[email-masked]>
Date: Tue, 21 Mar 2023 20:07:52 -0700 (PDT)

next/pending-fixes baseline: 663 runs, 61 regressions (v6.3-rc3-248-g01078db9d658)

Regressions Summary
-------------------

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          
bcm2835-rpi-b-rev2           | arm    | lab-broonie     | gcc-10   | bcm2835_defconfig            | 1          
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | bcm2835_defconfig            | 1          
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+crypto    | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+ima       | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 2          
fsl-lx2160a-rdb              | arm64  | lab-nxp         | gcc-10   | defconfig+ima                | 1          
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          
imx6ul-14x14-evk             | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          
imx7d-sdb                    | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          
imx7ulp-evk                  | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+crypto             | 5          
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig                    | 2          
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ook+videodec | 3          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ook+videodec | 3          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_mips-malta              | mips   | lab-collabora   | gcc-10   | malta_defconfig              | 1          
r8a7743-iwg20d-q7            | arm    | lab-cip         | gcc-10   | shmobile_defconfig           | 1          
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sc7180-trogdor-kingoftown    | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:  https://kernelci.org/test/job/next/branch/pending-fixes/kernel/v6.3-rc3-248-g01078db9d658/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   pending-fixes
  Describe: v6.3-rc3-248-g01078db9d658
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      01078db9d6580a65008c607c1eb0d2b04eea65a4 

Test Regressions
---------------- 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/641a3a6842ab6973dd9c951a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3a6842ab6973dd9c951b
        failing since 162 days (last pass: v6.0-6424-g1af0c6a154d32, first fail: v6.0-9666-g02c05e0b8d5c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3d9aa708334bd99c950a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3d9aa708334bd99c950b
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a372d15450186de9c950f

  Results:     6 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+kselftest
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/641a372d15450186de9c9512
        failing since 6 days (last pass: v6.3-rc2-254-g7f35d1f08bb0, first fail: v6.3-rc2-307-g900e881df1ea3)
        3 lines

    2023-03-21T23:00:37.040835  kern  :emerg : __common_interrupt: 1.55 No irq handler for vector

    2023-03-21T23:00:37.057419  kern  :emerg : __common_interrupt: 2.55 No irq handler for vector<8>[   13.770282] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=3>

    2023-03-21T23:00:37.057506  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
bcm2835-rpi-b-rev2           | arm    | lab-broonie     | gcc-10   | bcm2835_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/641a3a2233aef6504e9c95c1

  Results:     50 PASS, 2 FAIL, 1 SKIP
  Full config: bcm2835_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/bcm2835_defconfig/gcc-10/lab-broonie/baseline-bcm2835-rpi-b-rev2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/bcm2835_defconfig/gcc-10/lab-broonie/baseline-bcm2835-rpi-b-rev2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3a2233aef6504e9c95f6
        failing since 7 days (last pass: v6.3-rc1-336-gbec2983895c5, first fail: v6.3-rc2-254-g7f35d1f08bb0)

    2023-03-21T23:13:06.367712  + set +x
    2023-03-21T23:13:06.371627  <8>[   19.037174] <LAVA_SIGNAL_ENDRUN 0_dmesg 203889_1.5.2.4.1>
    2023-03-21T23:13:06.504527  / # #
    2023-03-21T23:13:06.606556  export SHELL=/bin/sh
    2023-03-21T23:13:06.606842  #
    2023-03-21T23:13:06.708038  / # export SHELL=/bin/sh. /lava-203889/environment
    2023-03-21T23:13:06.708570  
    2023-03-21T23:13:06.810095  / # . /lava-203889/environment/lava-203889/bin/lava-test-runner /lava-203889/1
    2023-03-21T23:13:06.810482  
    2023-03-21T23:13:06.817779  / # /lava-203889/bin/lava-test-runner /lava-203889/1 
    ... (14 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | bcm2835_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/641a3a5503022c063e9c9514

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: bcm2835_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3a5503022c063e9c951d
        failing since 69 days (last pass: v6.2-rc3-243-g3145d9dfed32, first fail: v6.2-rc3-296-g79fa898551af)

    2023-03-21T23:14:07.448661  / # #

    2023-03-21T23:14:07.550924  export SHELL=/bin/sh

    2023-03-21T23:14:07.551591  #

    2023-03-21T23:14:07.653246  / # export SHELL=/bin/sh. /lava-9723091/environment

    2023-03-21T23:14:07.653819  

    2023-03-21T23:14:07.755438  / # . /lava-9723091/environment/lava-9723091/bin/lava-test-runner /lava-9723091/1

    2023-03-21T23:14:07.756410  

    2023-03-21T23:14:07.770270  / # /lava-9723091/bin/lava-test-runner /lava-9723091/1

    2023-03-21T23:14:07.880056  + export 'TESTRUN_ID=1_bootrr'

    2023-03-21T23:14:07.880505  + cd /lava-9723091/1/tests/1_bootrr
 
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a3c0e4c7ccc01a29c950c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3c0e4c7ccc01a29c950d
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+crypto    | 1          

  Details:     https://kernelci.org/test/plan/id/641a390de495d006df9c957a

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+crypto
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a390de495d006df9c9583
        failing since 63 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-21T23:08:33.794154  <8>[   24.144767] <LAVA_SIGNAL_ENDRUN 0_dmesg 3433203_1.5.2.4.1>
    2023-03-21T23:08:33.903983  / # #
    2023-03-21T23:08:34.008313  export SHELL=/bin/sh
    2023-03-21T23:08:34.009514  #
    2023-03-21T23:08:34.112498  / # export SHELL=/bin/sh. /lava-3433203/environment
    2023-03-21T23:08:34.113574  
    2023-03-21T23:08:34.215962  / # . /lava-3433203/environment/lava-3433203/bin/lava-test-runner /lava-3433203/1
    2023-03-21T23:08:34.217269  
    2023-03-21T23:08:34.222106  / # /lava-3433203/bin/lava-test-runner /lava-3433203/1
    2023-03-21T23:08:34.315124  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+ima       | 1          

  Details:     https://kernelci.org/test/plan/id/641a39a177ec0cdc229c9506

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+ima
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+ima/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+ima/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a39a177ec0cdc229c950f
        failing since 63 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-21T23:11:13.840983  <8>[   14.024236] <LAVA_SIGNAL_ENDRUN 0_dmesg 3433221_1.5.2.4.1>
    2023-03-21T23:11:13.952785  / # #
    2023-03-21T23:11:14.055494  export SHELL=/bin/sh
    2023-03-21T23:11:14.056236  #
    2023-03-21T23:11:14.157828  / # export SHELL=/bin/sh. /lava-3433221/environment
    2023-03-21T23:11:14.159003  
    2023-03-21T23:11:14.276650  / # . /lava-3433221/environment/lava-3433221/bin/lava-test-runner /lava-3433221/1
    2023-03-21T23:11:14.277451  
    2023-03-21T23:11:14.277675  / # /lava-3433221/bin/lava-test-runner /lava-3433221/1<3>[   14.446964] Bluetooth: hci0: command 0x0c03 tx timeout
    2023-03-21T23:11:14.284197   
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/641a3e03fefa590a339c9515

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3e03fefa590a339c951e
        failing since 63 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-21T23:29:51.299137  <8>[   16.635074] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=pass UNITS=lines MEASUREMENT=0>
    2023-03-21T23:29:51.299471  + set +x
    2023-03-21T23:29:51.305402  <8>[   16.645414] <LAVA_SIGNAL_ENDRUN 0_dmesg 3433302_1.5.2.4.1>
    2023-03-21T23:29:51.412456  / # #
    2023-03-21T23:29:51.514627  export SHELL=/bin/sh
    2023-03-21T23:29:51.515393  #
    2023-03-21T23:29:51.617086  / # export SHELL=/bin/sh. /lava-3433302/environment
    2023-03-21T23:29:51.617523  
    2023-03-21T23:29:51.718761  / # . /lava-3433302/environment/lava-3433302/bin/lava-test-runner /lava-3433302/1
    2023-03-21T23:29:51.719372   
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a3ea507cadee4a19c951f

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3ea507cadee4a19c9528
        failing since 63 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-21T23:32:29.226087  <8>[   13.495054] <LAVA_SIGNAL_ENDRUN 0_dmesg 3433326_1.5.2.4.1>
    2023-03-21T23:32:29.336483  / # #
    2023-03-21T23:32:29.439780  export SHELL=/bin/sh
    2023-03-21T23:32:29.440752  #
    2023-03-21T23:32:29.542979  / # export SHELL=/bin/sh. /lava-3433326/environment
    2023-03-21T23:32:29.544120  
    2023-03-21T23:32:29.544570  / # . /lava-3433326/environment<3>[   13.805971] Bluetooth: hci0: command 0xfc18 tx timeout
    2023-03-21T23:32:29.646497  /lava-3433326/bin/lava-test-runner /lava-3433326/1
    2023-03-21T23:32:29.647818  
    2023-03-21T23:32:29.652884  / # /lava-3433326/bin/lava-test-runner /lava-3433326/1 
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/641a40aa98a04cea489c950a

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a40aa98a04cea489c9513
        failing since 63 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-21T23:41:18.741451  <8>[   14.847791] <LAVA_SIGNAL_ENDRUN 0_dmesg 3433397_1.5.2.4.1>
    2023-03-21T23:41:18.852846  / # #
    2023-03-21T23:41:18.956469  export SHELL=/bin/sh
    2023-03-21T23:41:18.957620  #
    2023-03-21T23:41:19.060085  / # export SHELL=/bin/sh. /lava-3433397/environment
    2023-03-21T23:41:19.061313  
    2023-03-21T23:41:19.164096  / # . /lava-3433397/environment/lava-3433397/bin/lava-test-runner /lava-3433397/1
    2023-03-21T23:41:19.165822  
    2023-03-21T23:41:19.170825  / # /lava-3433397/bin/lava-test-runner /lava-3433397/1
    2023-03-21T23:41:19.265557  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 2          

  Details:     https://kernelci.org/test/plan/id/641a3847b09e0616f99c9516

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: davinci_all_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/641a3847b09e0616f99c9519
        failing since 18 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-13370-g5872d227b73f)
        30 lines

    2023-03-21T23:05:02.081442  kern  :alert : Register r6 information: NULL pointer
    2023-03-21T23:05:02.123717  kern  :alert : Register r7 information: non-paged memory
    2023-03-21T23:05:02.124459  kern  :alert : Register r8 information: slab kmalloc-512 start c4bdc200 pointer offset 4 size 512
    2023-03-21T23:05:02.125224  kern  :alert : Register r9 information: non-paged memory
    2023-03-21T23:05:02.126439  kern  :alert : Register r10 information: NULL pointer
    2023-03-21T23:05:02.127195  kern  :alert : Register r11 information: 5-page vmalloc region starting at 0xbf3e2000 allocated at load_module+0x898/0x1b10
    2023-03-21T23:05:02.127897  kern  :alert : Register r12 information: non-paged memory
    2023-03-21T23:05:02.216113  kern  :emerg : Internal error: Oops: 17 [#1] PREEMPT ARM
    2023-03-21T23:05:02.216919  kern  :emerg : Process udevd (pid: 64, stack limit = 0xb8972522)
    2023-03-21T23:05:02.217522  kern  :emerg : Stack: (0xc88fdcf8 to 0xc88fe000) 
    ... (8 line(s) more)  

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/641a3847b09e0616f99c951a
        failing since 422 days (last pass: v5.16-11577-gffd79fec234d, first fail: v5.17-rc1-180-g86539e2bdb99)
        18 lines

    2023-03-21T23:05:02.033289  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3400
    2023-03-21T23:05:02.034338  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3800
    2023-03-21T23:05:02.035928  kern  :alert : 8<--- cut here ---
    2023-03-21T23:05:02.036713  kern  :alert : Unable to handle kernel NULL pointer dereference at virtual address 00000060 when read
    2023-03-21T23:05:02.037682  kern  :alert : [00000060] *pgd=c49f1831, *pte=00000000, *ppte=00000000
    2023-03-21T23:05:02.038524  kern  :alert : Register r0 information: non-paged memory
    2023-03-21T23:05:02.076288  kern  :alert : Register r1 information: 2-page vmalloc region starting at 0xc88fc000 allocated at kernel_clone+0x98/0x3a0
    2023-03-21T23:05:02.076919  kern  :alert : Register r2 information: non-slab/vmalloc memory
    2023-03-21T23:05:02.077437  kern  :alert : Register r3 information: non-paged memory
    2023-03-21T23:05:02.078179  kern  :alert : R<8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=18> 
    ... (2 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
fsl-lx2160a-rdb              | arm64  | lab-nxp         | gcc-10   | defconfig+ima                | 1          

  Details:     https://kernelci.org/test/plan/id/641a3a801e325b950a9c9524

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+ima
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+ima/gcc-10/lab-nxp/baseline-fsl-lx2160a-rdb.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+ima/gcc-10/lab-nxp/baseline-fsl-lx2160a-rdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3a801e325b950a9c9525
        new failure (last pass: v6.3-rc3-188-g9f7814c09ae60) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3d96ae768171199c951d

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3d96ae768171199c951e
        failing since 187 days (last pass: v6.0-rc4-291-g83a56f559828, first fail: v6.0-rc5-197-ga6a750a2f4166) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6ul-14x14-evk             | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          

  Details:     https://kernelci.org/test/plan/id/641a392d77f18618d79c95ca

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+crypto
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx6ul-14x14-evk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx6ul-14x14-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a392d77f18618d79c95d3
        failing since 76 days (last pass: v6.1-rc4-339-g185da88d03ed, first fail: v6.2-rc2-332-g2d3ca5d07c8e)

    2023-03-21T23:09:06.805187  + set +x
    2023-03-21T23:09:06.806341  <8>[   58.117844] <LAVA_SIGNAL_ENDRUN 0_dmesg 1180910_1.5.2.4.1>
    2023-03-21T23:09:06.915009  
    2023-03-21T23:09:08.074835  / # #export SHELL=/bin/sh
    2023-03-21T23:09:08.080550  
    2023-03-21T23:09:09.628460  / # export SHELL=/bin/sh. /lava-1180910/environment
    2023-03-21T23:09:09.634155  
    2023-03-21T23:09:12.456473  / # . /lava-1180910/environment/lava-1180910/bin/lava-test-runner /lava-1180910/1
    2023-03-21T23:09:12.462574  
    2023-03-21T23:09:12.463130  / # /lava-1180910/bin/lava-test-runner /lava-1180910/1 
    ... (18 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx7d-sdb                    | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          

  Details:     https://kernelci.org/test/plan/id/641a3910ef384751099c9560

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+crypto
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx7d-sdb.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx7d-sdb.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3910ef384751099c9569
        failing since 76 days (last pass: v6.1-rc6-391-gf445421fe4c7, first fail: v6.2-rc2-332-g2d3ca5d07c8e)

    2023-03-21T23:08:43.887896  <8>[   37.877608] <LAVA_SIGNAL_ENDRUN 0_dmesg 1180911_1.5.2.4.1>
    2023-03-21T23:08:43.993310  / # #
    2023-03-21T23:08:45.152673  export SHELL=/bin/sh
    2023-03-21T23:08:45.158243  #
    2023-03-21T23:08:46.704513  / # export SHELL=/bin/sh. /lava-1180911/environment
    2023-03-21T23:08:46.710102  
    2023-03-21T23:08:49.531792  / # . /lava-1180911/environment/lava-1180911/bin/lava-test-runner /lava-1180911/1
    2023-03-21T23:08:49.537829  
    2023-03-21T23:08:49.547316  / # /lava-1180911/bin/lava-test-runner /lava-1180911/1
    2023-03-21T23:08:49.631508  + export 'TESTRUN_ID=1_bootrr' 
    ... (22 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx7ulp-evk                  | arm    | lab-nxp         | gcc-10   | multi_v7_defconfig+crypto    | 1          

  Details:     https://kernelci.org/test/plan/id/641a393c43482d5a329c953e

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+crypto
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx7ulp-evk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+crypto/gcc-10/lab-nxp/baseline-imx7ulp-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a393c43482d5a329c9547
        failing since 76 days (last pass: v6.1-rc6-391-gf445421fe4c7, first fail: v6.2-rc2-332-g2d3ca5d07c8e)

    2023-03-21T23:09:25.739005  + set +x<8>[   66.951644] <LAVA_SIGNAL_ENDRUN 0_dmesg 1180912_1.5.2.4.1>
    2023-03-21T23:09:25.739166  
    2023-03-21T23:09:25.848193  / # #
    2023-03-21T23:09:27.007265  export SHELL=/bin/sh
    2023-03-21T23:09:27.012691  #
    2023-03-21T23:09:27.012856  / # export SHELL=/bin/sh
    2023-03-21T23:09:28.559033  / # . /lava-1180912/environment
    2023-03-21T23:09:31.386487  /lava-1180912/bin/lava-test-runner /lava-1180912/1
    2023-03-21T23:09:31.392572  . /lava-1180912/environment
    2023-03-21T23:09:31.392830  / # /lava-1180912/bin/lava-test-runner /lava-1180912/1 
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a38753284c1cd879c9533

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a38753284c1cd879c9534
        failing since 326 days (last pass: v5.18-rc2-366-ga3e1163f7eb1a, first fail: v5.18-rc4-464-g425675974eb3) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+crypto             | 5          

  Details:     https://kernelci.org/test/plan/id/641a383ebc4f52d4429c955a

  Results:     90 PASS, 5 FAIL, 1 SKIP
  Full config: defconfig+crypto
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+crypto/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+crypto/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a383ebc4f52d4429c9561
        failing since 26 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-6789-g74c3b2e0a611)

    2023-03-21T23:05:16.749426  / # #
    2023-03-21T23:05:16.852049  export SHELL=/bin/sh
    2023-03-21T23:05:16.852794  #
    2023-03-21T23:05:16.954622  / # export SHELL=/bin/sh. /lava-300333/environment
    2023-03-21T23:05:16.955201  
    2023-03-21T23:05:17.056860  / # . /lava-300333/environment/lava-300333/bin/lava-test-runner /lava-300333/1
    2023-03-21T23:05:17.058034  
    2023-03-21T23:05:17.075740  / # /lava-300333/bin/lava-test-runner /lava-300333/1
    2023-03-21T23:05:17.129962  + export 'TESTRUN_ID=1_bootrr'
    2023-03-21T23:05:17.130449  + <8>[   44.166109] <LAVA_SIGNAL_STARTRUN 1_bootrr 300333_1.5.2.4.5> 
    ... (15 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc2-probed: https://kernelci.org/test/case/id/641a383ebc4f52d4429c9565
        failing since 26 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-6789-g74c3b2e0a611)

    2023-03-21T23:05:19.224087  /lava-300333/1/../bin/lava-test-case
    2023-03-21T23:05:19.224577  <8>[   46.271537] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc2-probed RESULT=fail>
    2023-03-21T23:05:19.224995  /lava-300333/1/../bin/lava-test-case   

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/641a383ebc4f52d4429c9567
        failing since 26 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-6789-g74c3b2e0a611)

    2023-03-21T23:05:20.282615  /lava-300333/1/../bin/lava-test-case
    2023-03-21T23:05:20.283094  <8>[   47.311994] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>
    2023-03-21T23:05:20.283469  /lava-300333/1/../bin/lava-test-case
    2023-03-21T23:05:20.283729  <8>[   47.328581] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-driver-present RESULT=pass>
    2023-03-21T23:05:20.283970  /lava-300333/1/../bin/lava-test-case
    2023-03-21T23:05:20.284202  <8>[   47.347119] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-probed RESULT=pass>   

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/641a383ebc4f52d4429c956c
        failing since 26 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-6789-g74c3b2e0a611)

    2023-03-21T23:05:21.355717  /lava-300333/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc1-probed: https://kernelci.org/test/case/id/641a383ebc4f52d4429c956d
        failing since 26 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-6789-g74c3b2e0a611)

    2023-03-21T23:05:21.358878  <8>[   48.420717] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-03-21T23:05:22.412483  /lava-300333/1/../bin/lava-test-case
    2023-03-21T23:05:22.413000  <8>[   49.441512] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc1-probed RESULT=fail>
    2023-03-21T23:05:22.413384  /lava-300333/1/../bin/lava-test-case   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig                    | 2          

  Details:     https://kernelci.org/test/plan/id/641a3d18e9c145900e9c957f

  Results:     51 PASS, 2 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a3d18e9c145900e9c9586
        new failure (last pass: v6.3-rc2-423-g0e8bb497921e)

    2023-03-21T23:26:08.065302  / # #
    2023-03-21T23:26:08.168083  export SHELL=/bin/sh
    2023-03-21T23:26:08.168905  #
    2023-03-21T23:26:08.270966  / # export SHELL=/bin/sh. /lava-300387/environment
    2023-03-21T23:26:08.271787  
    2023-03-21T23:26:08.373745  / # . /lava-300387/environment/lava-300387/bin/lava-test-runner /lava-300387/1
    2023-03-21T23:26:08.374994  
    2023-03-21T23:26:08.390818  / # /lava-300387/bin/lava-test-runner /lava-300387/1
    2023-03-21T23:26:08.447571  + export 'TESTRUN_ID=1_bootrr'
    2023-03-21T23:26:08.448057  + cd /l<8>[   15.155646] <LAVA_SIGNAL_STARTRUN 1_bootrr 300387_1.5.2.4.5> 
    ... (10 line(s) more)  

  * baseline.bootrr.dwc3-usb1-probed: https://kernelci.org/test/case/id/641a3d18e9c145900e9c9596
        new failure (last pass: v6.3-rc2-423-g0e8bb497921e)

    2023-03-21T23:26:10.819628  /lava-300387/1/../bin/lava-test-case
    2023-03-21T23:26:10.820115  <8>[   17.625677] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=dwc3-usb1-probed RESULT=fail>
    2023-03-21T23:26:10.820529  /lava-300387/1/../bin/lava-test-case   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a4156a3bbdacfd99c950e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a4156a3bbdacfd99c950f
        failing since 56 days (last pass: v6.2-rc5-157-g1f16e03afb18, first fail: v6.2-rc5-174-gc5f142de047d) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ook+videodec | 3          

  Details:     https://kernelci.org/test/plan/id/641a3b420abc7239029c9517

  Results:     166 PASS, 6 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+videodec/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+videodec/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.panel-edp-probed: https://kernelci.org/test/case/id/641a3b420abc7239029c956b
        failing since 25 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-8700-gab98cc06b683)

    2023-03-21T23:18:12.569371  /lava-9723198/1/../bin/lava-test-case

    2023-03-21T23:18:12.575920  <8>[   23.425354] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=panel-edp-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-pericfg-probed: https://kernelci.org/test/case/id/641a3b420abc7239029c95bf
        failing since 16 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-13684-gf93c476d9769)

    2023-03-21T23:18:09.706389  /lava-9723198/1/../bin/lava-test-case

    2023-03-21T23:18:09.717377  <8>[   20.563858] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-infracfg-probed: https://kernelci.org/test/case/id/641a3b420abc7239029c95c0
        failing since 16 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-13684-gf93c476d9769)

    2023-03-21T23:18:08.674291  /lava-9723198/1/../bin/lava-test-case

    2023-03-21T23:18:08.684762  <8>[   19.532107] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-infracfg-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          

  Details:     https://kernelci.org/test/plan/id/641a3cfb04b327a7be9c9508

  Results:     166 PASS, 6 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.panel-edp-probed: https://kernelci.org/test/case/id/641a3cfb04b327a7be9c955c
        failing since 27 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-21T23:25:20.459097  /lava-9723259/1/../bin/lava-test-case

    2023-03-21T23:25:20.465553  <8>[   23.409840] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=panel-edp-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-pericfg-probed: https://kernelci.org/test/case/id/641a3cfb04b327a7be9c95ae
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-21T23:25:17.607989  /lava-9723259/1/../bin/lava-test-case

    2023-03-21T23:25:17.617591  <8>[   20.559526] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-infracfg-probed: https://kernelci.org/test/case/id/641a3cfb04b327a7be9c95af
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-21T23:25:15.564439  <8>[   18.504386] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-topckgen-probed RESULT=pass>

    2023-03-21T23:25:16.525574  <3>[   19.470913] Bluetooth: hci0: Opcode 0x c14 failed: -110

    2023-03-21T23:25:16.532332  <3>[   19.471065] Bluetooth: hci0: command 0x0c14 tx timeout

    2023-03-21T23:25:16.580213  /lava-9723259/1/../bin/lava-test-case

    2023-03-21T23:25:16.590418  <8>[   19.531886] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-infracfg-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a413f8f1f5ecf519c950e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a413f8f1f5ecf519c950f
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ook+videodec | 3          

  Details:     https://kernelci.org/test/plan/id/641a3b917987b42a759c9561

  Results:     177 PASS, 9 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+videodec/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+videodec/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.clk-mt8192-pericfg-probed: https://kernelci.org/test/case/id/641a3b917987b42a759c9617
        failing since 18 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-13370-g5872d227b73f)

    2023-03-21T23:19:28.054025  /lava-9723192/1/../bin/lava-test-case

    2023-03-21T23:19:28.063336  <8>[   20.964509] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-infracfg-probed: https://kernelci.org/test/case/id/641a3b917987b42a759c9618
        failing since 18 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-13370-g5872d227b73f)

    2023-03-21T23:19:27.037466  /lava-9723192/1/../bin/lava-test-case

    2023-03-21T23:19:27.044283  <8>[   19.944837] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-infracfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-topckgen-probed: https://kernelci.org/test/case/id/641a3b917987b42a759c9619
        failing since 18 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-13370-g5872d227b73f)

    2023-03-21T23:19:26.017890  /lava-9723192/1/../bin/lava-test-case

    2023-03-21T23:19:26.024228  <8>[   18.925476] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-topckgen-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          

  Details:     https://kernelci.org/test/plan/id/641a3d20de45c1c2af9c9571

  Results:     177 PASS, 9 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.clk-mt8192-pericfg-probed: https://kernelci.org/test/case/id/641a3d20de45c1c2af9c9626
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-21T23:25:50.639765  /lava-9723307/1/../bin/lava-test-case

    2023-03-21T23:25:50.645431  <8>[   21.046266] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-infracfg-probed: https://kernelci.org/test/case/id/641a3d20de45c1c2af9c9627
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-21T23:25:49.619498  /lava-9723307/1/../bin/lava-test-case

    2023-03-21T23:25:49.625994  <8>[   20.026628] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-infracfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-topckgen-probed: https://kernelci.org/test/case/id/641a3d20de45c1c2af9c9628
        failing since 23 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-21T23:25:48.600460  /lava-9723307/1/../bin/lava-test-case

    2023-03-21T23:25:48.606860  <8>[   19.007300] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-topckgen-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a4146a3bbdacfd99c9508

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a4146a3bbdacfd99c9509
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/641a3ab32ba5399c239c963e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: oxnas_v6_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3ab32ba5399c239c963f
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a44dd1fa9b4489d9c950f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a44dd1fa9b4489d9c9510
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3ddc8e49e0a0bb9c9505

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3ddc8e49e0a0bb9c9506
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3ef306ea57f10d9c954b

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3ef306ea57f10d9c954c
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3dda54ab8385299c95d6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3dda54ab8385299c95d7
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3dee8e49e0a0bb9c9518

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3dee8e49e0a0bb9c9519
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3dd8c5f1905cf49c95c7

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3dd8c5f1905cf49c95c8
        failing since 160 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3dda59dec282a69c9530

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3dda59dec282a69c9531
        failing since 160 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3d7147ce27faba9c953c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3d7147ce27faba9c953d
        failing since 160 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3e018e49e0a0bb9c955e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3e018e49e0a0bb9c955f
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/641a3f07e8f41501199c959c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a3f07e8f41501199c959d
        failing since 160 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_mips-malta              | mips   | lab-collabora   | gcc-10   | malta_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/641a380c335181f3ee9c9508

  Results:     4 PASS, 1 FAIL, 2 SKIP
  Full config: malta_defconfig
  Compiler:    gcc-10 (mips-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/mips/malta_defconfig/gcc-10/lab-collabora/baseline-qemu_mips-malta.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/mips/malta_defconfig/gcc-10/lab-collabora/baseline-qemu_mips-malta.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/mipsel/rootfs.cpio.gz 

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/641a380c335181f3ee9c950c
        failing since 0 day (last pass: v6.3-rc3-188-g9f7814c09ae60, first fail: v6.3-rc3-224-ga0d6a906c6a2)
        1 lines

    2023-03-21T23:04:39.026538  kern  :alert : CPU 0 Unable to handle kernel paging request at virtual address 0005cce4, epc == 80200868, ra == 8020085c
    2023-03-21T23:04:39.026748  

    2023-03-21T23:04:39.050530  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=1>
    2023-03-21T23:04:39.050738  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
r8a7743-iwg20d-q7            | arm    | lab-cip         | gcc-10   | shmobile_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/641a370e10b60763849c950c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: shmobile_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/shmobile_defconfig/gcc-10/lab-cip/baseline-r8a7743-iwg20d-q7.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a370e10b60763849c950d
        failing since 4 days (last pass: v6.3-rc2-369-g71c206ed429b, first fail: v6.3-rc2-423-g0e8bb497921e) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a398935317968679c9511

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a398935317968679c9512
        failing since 96 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a38861ca0d3540b9c9524

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a38861ca0d3540b9c9525
        failing since 96 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a4169b6b8b6800f9c951d

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a4169b6b8b6800f9c951e
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdor-kingoftown    | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a41438f1f5ecf519c9513

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-kingoftown.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-kingoftown.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a41438f1f5ecf519c9514
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641a4143b6b8b6800f9c9505

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a4143b6b8b6800f9c9506
        failing since 42 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/641a367f14aebdfb639c9560

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641a367f14aebdfb639c9569
        failing since 18 days (last pass: v6.2-12625-g13efc3a9f23b, first fail: v6.2-13370-g5872d227b73f)

    2023-03-21T22:57:44.874341  / # #
    2023-03-21T22:57:44.979878  export SHELL=/bin/sh
    2023-03-21T22:57:44.981450  #
    2023-03-21T22:57:45.084725  / # export SHELL=/bin/sh. /lava-3433113/environment
    2023-03-21T22:57:45.086319  
    2023-03-21T22:57:45.189651  / # . /lava-3433113/environment/lava-3433113/bin/lava-test-runner /lava-3433113/1
    2023-03-21T22:57:45.192296  
    2023-03-21T22:57:45.200050  / # /lava-3433113/bin/lava-test-runner /lava-3433113/1
    2023-03-21T22:57:45.344744  + export 'TESTRUN_ID=1_bootrr'
    2023-03-21T22:57:45.345929  + cd /lava-3433113/1/tests/1_bootrr 
    ... (10 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/641a399e31fff6c48c9c9505

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc3-248-g01078db9d658/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641a399e31fff6c48c9c9506
        failing since 96 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da)