Subject: next/pending-fixes baseline: 539 runs, 51 regressions (v6.3-rc2-369-g71c206ed429b)

From: "kernelci.org bot" <[email-masked]>
Date: Wed, 15 Mar 2023 20:59:43 -0700 (PDT)

next/pending-fixes baseline: 539 runs, 51 regressions (v6.3-rc2-369-g71c206ed429b)

Regressions Summary
-------------------

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          
bcm2835-rpi-b-rev2           | arm    | lab-broonie     | gcc-10   | bcm2835_defconfig            | 1          
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | bcm2835_defconfig            | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+ima       | 1          
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 2          
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...6-chromebook | 1          
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
jh7100-starfi...isionfive-v1 | riscv  | lab-clabbe      | gcc-10   | defconfig+kselftest          | 1          
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+CON...OMIZE_BASE=y | 5          
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+ima                | 4          
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig+videodec           | 2          
meson-gxm-q200               | arm64  | lab-baylibre    | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sc7180-trogdor-kingoftown    | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:  https://kernelci.org/test/job/next/branch/pending-fixes/kernel/v6.3-rc2-369-g71c206ed429b/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   pending-fixes
  Describe: v6.3-rc2-369-g71c206ed429b
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      71c206ed429bcd060918a00616d993244294315e 

Test Regressions
---------------- 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125d3a674ebb1ebd8c867b

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125d3a674ebb1ebd8c867c
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
asus-CM1400CXA-dalboz        | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641258a2afa035d8a98c8655

  Results:     6 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+kselftest
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/x86_64/x86_64_defconfig+x86-chromebook+kselftest/gcc-10/lab-collabora/baseline-asus-CM1400CXA-dalboz.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/641258a2afa035d8a98c8658
        failing since 1 day (last pass: v6.3-rc2-254-g7f35d1f08bb0, first fail: v6.3-rc2-307-g900e881df1ea3)
        3 lines

    2023-03-15T23:45:20.189568  kern  :emerg : __common_interrupt: 1.55 No irq handler for vector

    2023-03-15T23:45:20.196239  kern  :emerg : __common_interrupt: 2.55 No irq handler for vector

    2023-03-15T23:45:20.203002  kern  :emerg : __common_interrupt: 3.55 No irq handler for vector

    2023-03-15T23:45:20.209671  <8>[   96.561681] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=fail UNITS=lines MEASUREMENT=3>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
bcm2835-rpi-b-rev2           | arm    | lab-broonie     | gcc-10   | bcm2835_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/64125b249e6132fd488c8697

  Results:     50 PASS, 2 FAIL, 1 SKIP
  Full config: bcm2835_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/bcm2835_defconfig/gcc-10/lab-broonie/baseline-bcm2835-rpi-b-rev2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/bcm2835_defconfig/gcc-10/lab-broonie/baseline-bcm2835-rpi-b-rev2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125b249e6132fd488c86cd
        failing since 1 day (last pass: v6.3-rc1-336-gbec2983895c5, first fail: v6.3-rc2-254-g7f35d1f08bb0)

    2023-03-15T23:56:00.860624  + set +x
    2023-03-15T23:56:00.865552  <8>[   18.464808] <LAVA_SIGNAL_ENDRUN 0_dmesg 169788_1.5.2.4.1>
    2023-03-15T23:56:01.002458  / # #
    2023-03-15T23:56:01.105273  export SHELL=/bin/sh
    2023-03-15T23:56:01.106092  #
    2023-03-15T23:56:01.208211  / # export SHELL=/bin/sh. /lava-169788/environment
    2023-03-15T23:56:01.209222  
    2023-03-15T23:56:01.311367  / # . /lava-169788/environment/lava-169788/bin/lava-test-runner /lava-169788/1
    2023-03-15T23:56:01.312658  
    2023-03-15T23:56:01.318469  / # /lava-169788/bin/lava-test-runner /lava-169788/1 
    ... (14 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
bcm2836-rpi-2-b              | arm    | lab-collabora   | gcc-10   | bcm2835_defconfig            | 1          

  Details:     https://kernelci.org/test/plan/id/64125b36fc336943fa8c8633

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: bcm2835_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/bcm2835_defconfig/gcc-10/lab-collabora/baseline-bcm2836-rpi-2-b.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125b36fc336943fa8c863c
        failing since 63 days (last pass: v6.2-rc3-243-g3145d9dfed32, first fail: v6.2-rc3-296-g79fa898551af)

    2023-03-15T23:56:21.118075  / # #

    2023-03-15T23:56:21.220153  export SHELL=/bin/sh

    2023-03-15T23:56:21.220932  #

    2023-03-15T23:56:21.322598  / # export SHELL=/bin/sh. /lava-9641518/environment

    2023-03-15T23:56:21.323099  

    2023-03-15T23:56:21.424541  / # . /lava-9641518/environment/lava-9641518/bin/lava-test-runner /lava-9641518/1

    2023-03-15T23:56:21.425891  

    2023-03-15T23:56:21.438180  / # /lava-9641518/bin/lava-test-runner /lava-9641518/1

    2023-03-15T23:56:21.549206  + export 'TESTRUN_ID=1_bootrr'

    2023-03-15T23:56:21.549654  + cd /lava-9641518/1/tests/1_bootrr
 
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+ima       | 1          

  Details:     https://kernelci.org/test/plan/id/64125a9de3179c1cd38c8870

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig+ima
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+ima/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+ima/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125a9de3179c1cd38c8879
        failing since 57 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-15T23:53:37.044114  <8>[   15.183629] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=emerg RESULT=pass UNITS=lines MEASUREMENT=0>
    2023-03-15T23:53:37.050564  + set +x<8>[   15.194600] <LAVA_SIGNAL_ENDRUN 0_dmesg 3415595_1.5.2.4.1>
    2023-03-15T23:53:37.050858  
    2023-03-15T23:53:37.161743  / # #
    2023-03-15T23:53:37.263671  export SHELL=/bin/sh
    2023-03-15T23:53:37.264321  #
    2023-03-15T23:53:37.264559  <3>[   15.317736] Bluetooth: hci0: command 0x0c03 tx timeout
    2023-03-15T23:53:37.365807  / # export SHELL=/bin/sh. /lava-3415595/environment
    2023-03-15T23:53:37.366653  
    2023-03-15T23:53:37.468443  / # . /lava-3415595/environment/lava-3415595/bin/lava-test-runner /lava-3415595/1 
    ... (14 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
cubietruck                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/64125d673441c23bb48c86cd

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig/gcc-10/lab-baylibre/baseline-cubietruck.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125d673441c23bb48c86d6
        failing since 57 days (last pass: v6.1-13341-g16cba743aec0, first fail: v6.2-rc4-302-g48ed69cf21b2)

    2023-03-16T00:05:33.631421  <8>[   14.988189] <LAVA_SIGNAL_ENDRUN 0_dmesg 3415679_1.5.2.4.1>
    2023-03-16T00:05:33.740851  / # #
    2023-03-16T00:05:33.844412  export SHELL=/bin/sh
    2023-03-16T00:05:33.845111  #
    2023-03-16T00:05:33.946675  / # export SHELL=/bin/sh. /lava-3415679/environment
    2023-03-16T00:05:33.947044  
    2023-03-16T00:05:33.947213  / # <3>[   15.237627] Bluetooth: hci0: command 0x0c03 tx timeout
    2023-03-16T00:05:34.048432  . /lava-3415679/environment/lava-3415679/bin/lava-test-runner /lava-3415679/1
    2023-03-16T00:05:34.049757  
    2023-03-16T00:05:34.055035  / # /lava-3415679/bin/lava-test-runner /lava-3415679/1 
    ... (12 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
da850-lcdk                   | arm    | lab-baylibre    | gcc-10   | davinci_all_defconfig        | 2          

  Details:     https://kernelci.org/test/plan/id/6412591d9e2fe6e2448c8649

  Results:     3 PASS, 3 FAIL, 1 SKIP
  Full config: davinci_all_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.dmesg.emerg: https://kernelci.org/test/case/id/6412591d9e2fe6e2448c864c
        failing since 12 days (last pass: v6.2-1218-g2d4a54ee969d, first fail: v6.2-13370-g5872d227b73f)
        30 lines

    2023-03-15T23:47:16.127769  kern  :alert : Register r5 information: non-slab/vmalloc memory
    2023-03-15T23:47:16.129327  kern  :alert : Register r6 information: NULL pointer
    2023-03-15T23:47:16.174108  kern  :alert : Register r7 information: non-paged memory
    2023-03-15T23:47:16.174415  kern  :alert : Register r8 information: slab kmalloc-512 start c4c25200 pointer offset 4 size 512
    2023-03-15T23:47:16.174647  kern  :alert : Register r9 information: non-paged memory
    2023-03-15T23:47:16.174837  kern  :alert : Register r10 information: NULL pointer
    2023-03-15T23:47:16.175060  kern  :alert : Register r11 information: 5-page vmalloc region starting at 0xbf3e8000 allocated at load_module+0x898/0x1b10
    2023-03-15T23:47:16.177145  kern  :alert : Register r12 information: non-paged memory
    2023-03-15T23:47:16.265514  kern  :emerg : Internal error: Oops: 17 [#1] PREEMPT ARM
    2023-03-15T23:47:16.265800  kern  :emerg : Process udevd (pid: 65, stack limit = 0xf9717a7d) 
    ... (10 line(s) more)  

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/6412591d9e2fe6e2448c864d
        failing since 416 days (last pass: v5.16-11577-gffd79fec234d, first fail: v5.17-rc1-180-g86539e2bdb99)
        18 lines

    2023-03-15T23:47:16.082586  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3400
    2023-03-15T23:47:16.083134  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3800
    2023-03-15T23:47:16.083405  kern  :alert : 8<--- cut here ---
    2023-03-15T23:47:16.083621  kern  :alert : Unable to handle kernel NULL pointer dereference at virtual address 00000060 when read
    2023-03-15T23:47:16.083802  kern  :alert : [00000060] *pgd=c49f7831, *pte=00000000, *ppte=00000000
    2023-03-15T23:47:16.084017  kern  :alert : Register r0 information: non-paged memory
    2023-03-15T23:47:16.126221  kern  :alert : Register r1 information: 2-page vmalloc region starting at 0xc8900000 allocated at kernel_clone+0x98/0x3a0
    2023-03-15T23:47:16.126469  kern  :alert : Register r2 information: non-slab/vmalloc memory
    2023-03-15T23:47:16.126643  kern  :alert : Register r3 information: non-paged memory
    2023-03-15T23:47:16.126791  kern  :alert : R<8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=18> 
    ... (1 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
hp-11A-G6-EE-grunt           | x86_64 | lab-collabora   | gcc-10   | x86_64_defcon...6-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/64125b08360f0c48668c8661

  Results:     18 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-collabora/baseline-hp-11A-G6-EE-grunt.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/x86/rootfs.cpio.gz 

  * baseline.bootrr.tpm-chip-is-online: https://kernelci.org/test/case/id/64125b08360f0c48668c8674
        new failure (last pass: v6.3-rc2-307-g900e881df1ea3)

    2023-03-15T23:55:31.308996  <8>[   10.581363] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=all-cpus-are-online RESULT=pass>

    2023-03-15T23:55:31.313061  /usr/bin/tpm2_getcap

    2023-03-15T23:55:41.524687  /lava-9641469/1/../bin/lava-test-case

    2023-03-15T23:55:41.531902  <8>[   20.804180] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=tpm-chip-is-online RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125e853e76250bf18c862f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125e853e76250bf18c8630
        failing since 181 days (last pass: v6.0-rc4-291-g83a56f559828, first fail: v6.0-rc5-197-ga6a750a2f4166) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
jetson-tk1                   | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/64125750a152319c968c8647

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-baylibre/baseline-jetson-tk1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125750a152319c968c8648
        failing since 320 days (last pass: v5.18-rc2-366-ga3e1163f7eb1a, first fail: v5.18-rc4-464-g425675974eb3) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
jh7100-starfi...isionfive-v1 | riscv  | lab-clabbe      | gcc-10   | defconfig+kselftest          | 1          

  Details:     https://kernelci.org/test/plan/id/64125c519cdd8367e28c87c6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+kselftest
  Compiler:    gcc-10 (riscv64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/riscv/defconfig+kselftest/gcc-10/lab-clabbe/baseline-jh7100-starfive-visionfive-v1.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/riscv/defconfig+kselftest/gcc-10/lab-clabbe/baseline-jh7100-starfive-visionfive-v1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/riscv/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125c519cdd8367e28c87c7
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+CON...OMIZE_BASE=y | 5          

  Details:     https://kernelci.org/test/plan/id/641259ca31e30a1e2b8c863b

  Results:     90 PASS, 5 FAIL, 1 SKIP
  Full config: defconfig+CONFIG_RANDOMIZE_BASE=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+CONFIG_RANDOMIZE_BASE=y/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/641259ca31e30a1e2b8c8642
        new failure (last pass: v6.3-rc2-307-g900e881df1ea3)

    2023-03-15T23:50:15.650470  #
    2023-03-15T23:50:15.651331  / # <6>[   22.173213] fsl_enetc 0000:00:00.2: enabling device (0400 -> 0402)
    2023-03-15T23:50:15.752814  #export SHELL=/bin/sh
    2023-03-15T23:50:15.753455  <6>[   22.249209] usb 1-1.6: new full-speed USB device number 5 using xhci-hcd
    2023-03-15T23:50:15.753645  
    2023-03-15T23:50:15.753787  / # <6>[   22.289198] fsl_enetc 0000:00:00.6: enabling device (0400 -> 0402)
    2023-03-15T23:50:15.855047  export SHELL=/bin/sh. /lava-296391/environment
    2023-03-15T23:50:15.855693  
    2023-03-15T23:50:15.855893  / # <3>[   22.391044] hid-generic 0003:064F:2AF9.0003: device has no listeners, quitting
    2023-03-15T23:50:15.856049  <6>[   22.401500] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) 
    ... (24 line(s) more)  

  * baseline.bootrr.fsl_enetc-enetc2-probed: https://kernelci.org/test/case/id/641259ca31e30a1e2b8c8646
        failing since 21 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-15T23:50:18.126221  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:18.126492  <8>[   24.691624] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc2-probed RESULT=fail>
    2023-03-15T23:50:18.126642  /lava-296391/1/../bin/lava-test-case   

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/641259ca31e30a1e2b8c8648
        failing since 21 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-15T23:50:19.183305  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:19.183599  <8>[   25.729538] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>
    2023-03-15T23:50:19.183747  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:19.183885  <8>[   25.746457] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-driver-present RESULT=pass>
    2023-03-15T23:50:19.184036  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:19.184169  <8>[   25.765056] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-probed RESULT=pass>
    2023-03-15T23:50:19.184301  /lava-296391/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/641259ca31e30a1e2b8c864d
        failing since 21 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-15T23:50:19.242754  <8>[   25.821304] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-driver-present RESULT=pass>
    2023-03-15T23:50:20.261347  /lava-296391/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc1-probed: https://kernelci.org/test/case/id/641259ca31e30a1e2b8c864e
        failing since 21 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-15T23:50:20.264412  <8>[   26.843147] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-03-15T23:50:21.319718  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:21.320018  <8>[   27.864277] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc1-probed RESULT=fail>
    2023-03-15T23:50:21.320168  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:21.320306  <8>[   27.880694] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc_mdio-driver-present RESULT=pass>
    2023-03-15T23:50:21.320443  /lava-296391/1/../bin/lava-test-case
    2023-03-15T23:50:21.320574  <8>[   27.899661] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc_mdio-probed RESULT=pass>
    2023-03-15T23:50:21.320707  /lava-296391/1/../bin/lava-test-case   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-kbox-a-230-ls        | arm64  | lab-kontron     | gcc-10   | defconfig+ima                | 4          

  Details:     https://kernelci.org/test/plan/id/64125b5c11b6deee4f8c8643

  Results:     91 PASS, 4 FAIL, 1 SKIP
  Full config: defconfig+ima
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+ima/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+ima/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.fsl_enetc-enetc2-probed: https://kernelci.org/test/case/id/64125b5c11b6deee4f8c864e
        failing since 17 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-15T23:56:56.821421  <6>[   22.777123] fsl_enetc 0000:00:00.6: enabling device (0400 -> 0402)
    2023-03-15T23:56:56.932024  <6>[   22.889130] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402)
    2023-03-15T23:56:57.051845  <6>[   23.005158] fsl_enetc 0000:00:00.1: enabling device (0400 -> 0402)
    2023-03-15T23:56:57.792638  /lava-296401/1/../bin/lava-test-case
    2023-03-15T23:56:57.792954  <8>[   23.734520] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc2-probed RESULT=fail>
    2023-03-15T23:56:57.793119  /lava-296401/1/../bin/lava-test-case   

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/64125b5c11b6deee4f8c8650
        failing since 17 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-15T23:56:58.855739  /lava-296401/1/../bin/lava-test-case
    2023-03-15T23:56:58.856044  <8>[   24.772422] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>
    2023-03-15T23:56:58.856197  /lava-296401/1/../bin/lava-test-case
    2023-03-15T23:56:58.856339  <8>[   24.791002] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-driver-present RESULT=pass>
    2023-03-15T23:56:58.856480  /lava-296401/1/../bin/lava-test-case
    2023-03-15T23:56:58.856615  <8>[   24.809526] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=leds-gpio-probed RESULT=pass>
    2023-03-15T23:56:58.856751  /lava-296401/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc0-probed: https://kernelci.org/test/case/id/64125b5c11b6deee4f8c8655
        failing since 17 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-15T23:56:59.929708  /lava-296401/1/../bin/lava-test-case   

  * baseline.bootrr.fsl_enetc-enetc1-probed: https://kernelci.org/test/case/id/64125b5c11b6deee4f8c8656
        failing since 17 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-15T23:56:59.933312  <8>[   25.884578] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc0-probed RESULT=fail>
    2023-03-15T23:57:00.969395  /lava-296401/1/../bin/lava-test-case
    2023-03-15T23:57:00.969696  <8>[   26.906050] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=fsl_enetc-enetc1-probed RESULT=fail>
    2023-03-15T23:57:00.969851  /lava-296401/1/../bin/lava-test-case   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig+videodec           | 2          

  Details:     https://kernelci.org/test/plan/id/64125d682970f9424b8c864a

  Results:     51 PASS, 2 FAIL, 0 SKIP
  Full config: defconfig+videodec
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+videodec/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125d682970f9424b8c8651
        new failure (last pass: v6.3-rc2-307-g900e881df1ea3)

    2023-03-16T00:05:34.907262  / # #
    2023-03-16T00:05:35.012013  export SHELL=/bin/sh
    2023-03-16T00:05:35.012600  #
    2023-03-16T00:05:35.115561  / # export SHELL=/bin/sh. /lava-296418/environment
    2023-03-16T00:05:35.115946  
    2023-03-16T00:05:35.217709  / # . /lava-296418/environment/lava-296418/bin/lava-test-runner /lava-296418/1
    2023-03-16T00:05:35.218301  
    2023-03-16T00:05:35.235286  / # /lava-296418/bin/lava-test-runner /lava-296418/1
    2023-03-16T00:05:35.290261  + export 'TESTRUN_ID=1_bootrr'
    2023-03-16T00:05:35.290570  + cd /l<8>[   15.608850] <LAVA_SIGNAL_STARTRUN 1_bootrr 296418_1.5.2.4.5> 
    ... (10 line(s) more)  

  * baseline.bootrr.dwc3-usb1-probed: https://kernelci.org/test/case/id/64125d682970f9424b8c8661
        new failure (last pass: v6.3-rc2-307-g900e881df1ea3)

    2023-03-16T00:05:37.664407  /lava-296418/1/../bin/lava-test-case
    2023-03-16T00:05:37.664809  <8>[   18.079575] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=dwc3-usb1-probed RESULT=fail>
    2023-03-16T00:05:37.665038  /lava-296418/1/../bin/lava-test-case   

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
meson-gxm-q200               | arm64  | lab-baylibre    | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          

  Details:     https://kernelci.org/test/plan/id/64125f9a232a12ce5d8c8659

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+CONFIG_CPU_BIG_ENDIAN=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-meson-gxm-q200.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-meson-gxm-q200.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64be/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125f9a232a12ce5d8c865a
        new failure (last pass: v6.3-rc2-307-g900e881df1ea3) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64125d613441c23bb48c86c8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125d613441c23bb48c86c9
        failing since 50 days (last pass: v6.2-rc5-157-g1f16e03afb18, first fail: v6.2-rc5-174-gc5f142de047d) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          

  Details:     https://kernelci.org/test/plan/id/64125c27c7c9441fba8c86f4

  Results:     166 PASS, 6 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.panel-edp-probed: https://kernelci.org/test/case/id/64125c27c7c9441fba8c8747
        failing since 21 days (last pass: v6.2-rc8-211-g05def2d212c6, first fail: v6.2-3304-g34939120e353)

    2023-03-16T00:00:26.840371  /lava-9641685/1/../bin/lava-test-case

    2023-03-16T00:00:26.846528  <8>[   23.260992] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=panel-edp-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-pericfg-probed: https://kernelci.org/test/case/id/64125c27c7c9441fba8c879d
        failing since 17 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-16T00:00:22.955248  <8>[   19.365797] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-infracfg-probed RESULT=fail>

    2023-03-16T00:00:23.056038  <3>[   19.471703] Bluetooth: hci0: command 0x0c14 tx timeout

    2023-03-16T00:00:23.064718  <3>[   19.471703] Bluetooth: hci0: Opcode 0x c14 failed: -110

    2023-03-16T00:00:23.973263  /lava-9641685/1/../bin/lava-test-case

    2023-03-16T00:00:23.983829  <8>[   20.395963] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8183-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8183-infracfg-probed: https://kernelci.org/test/case/id/64125c27c7c9441fba8c879e
        failing since 17 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-16T00:00:22.944969  /lava-9641685/1/../bin/lava-test-case

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64125d9fa7812b03f78c8670

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125d9fa7812b03f78c8671
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm64-chromebook   | 3          

  Details:     https://kernelci.org/test/plan/id/64125c79e940e6aeac8c86ec

  Results:     177 PASS, 9 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.clk-mt8192-pericfg-probed: https://kernelci.org/test/case/id/64125c7ae940e6aeac8c876e
        failing since 17 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-16T00:01:32.055499  /lava-9641700/1/../bin/lava-test-case

    2023-03-16T00:01:32.062262  <8>[   20.945061] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-pericfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-infracfg-probed: https://kernelci.org/test/case/id/64125c7ae940e6aeac8c876f
        failing since 17 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-16T00:01:31.036578  /lava-9641700/1/../bin/lava-test-case

    2023-03-16T00:01:31.043092  <8>[   19.925284] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-infracfg-probed RESULT=fail>

  * baseline.bootrr.clk-mt8192-topckgen-probed: https://kernelci.org/test/case/id/64125c7ae940e6aeac8c8770
        failing since 17 days (last pass: v6.2-8700-gab98cc06b683, first fail: v6.2-12625-g13efc3a9f23b)

    2023-03-16T00:01:30.016375  /lava-9641700/1/../bin/lava-test-case

    2023-03-16T00:01:30.022728  <8>[   18.905486] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clk-mt8192-topckgen-probed RESULT=fail>

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64125dc6baf3eb02d38c863b

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125dc6baf3eb02d38c863c
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/64125620894027bbc48c864a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: oxnas_v6_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125620894027bbc48c864b
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qcom-qdf2400                 | arm64  | lab-linaro-lkft | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/641264ecd309d8d9648c8701

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-linaro-lkft/baseline-qcom-qdf2400.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/641264edd309d8d9648c8702
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125e00976d4132608c8696

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125e00976d4132608c8697
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125deef9947077c28c8692

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125deef9947077c28c8693
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125e13976d4132608c8739

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125e13976d4132608c873a
        failing since 154 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125e3c97774b5b8f8c86aa

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125e3c97774b5b8f8c86ab
        failing since 154 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125d3b7b33c305688c874e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-collabora/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125d3b7b33c305688c874f
        failing since 154 days (last pass: v6.0-5324-g7871897dadfa9, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125dff976d4132608c8693

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125dff976d4132608c8694
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125dedbdbd7d78b48c8673

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125dedbdbd7d78b48c8674
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125dfe976d4132608c8690

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125dfe976d4132608c8691
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/64125decbdbd7d78b48c8670

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125decbdbd7d78b48c8671
        failing since 154 days (last pass: v6.0-9666-g02c05e0b8d5c, first fail: v6.0-11312-g1778d6da389c) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6412586460507e08a78c8672

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6412586460507e08a78c8673
        failing since 90 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/64125760a152319c968c8666

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125760a152319c968c8667
        failing since 90 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3399-gru-kevin             | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64125dd9bdbd7d78b48c8644

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125dd9bdbd7d78b48c8645
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdor-kingoftown    | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64126044df35b7e6778c8632

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-kingoftown.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-kingoftown.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64126044df35b7e6778c8633
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | gcc-10   | defconfig+arm...ok+kselftest | 1          

  Details:     https://kernelci.org/test/plan/id/64125ef198b0e1eb738c864d

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook+kselftest
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm64/defconfig+arm64-chromebook+kselftest/gcc-10/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/64125ef198b0e1eb738c864e
        failing since 36 days (last pass: v6.2-rc7-120-g11b3628314a5, first fail: v6.2-rc7-136-g39693c3ba989) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun8i-h2-plus-orangepi-zero  | arm    | lab-baylibre    | gcc-10   | sunxi_defconfig              | 1          

  Details:     https://kernelci.org/test/plan/id/64125688ea7c3c23b88c866b

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: sunxi_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/sunxi_defconfig/gcc-10/lab-baylibre/baseline-sun8i-h2-plus-orangepi-zero.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.bootrr.deferred-probe-empty: https://kernelci.org/test/case/id/64125688ea7c3c23b88c8674
        failing since 12 days (last pass: v6.2-12625-g13efc3a9f23b, first fail: v6.2-13370-g5872d227b73f)

    2023-03-15T23:36:13.322995  <8>[    9.587769] <LAVA_SIGNAL_ENDRUN 0_dmesg 3415495_1.5.2.4.1>
    2023-03-15T23:36:13.443614  / # #
    2023-03-15T23:36:13.549484  export SHELL=/bin/sh
    2023-03-15T23:36:13.551073  #
    2023-03-15T23:36:13.654749  / # export SHELL=/bin/sh. /lava-3415495/environment
    2023-03-15T23:36:13.656347  
    2023-03-15T23:36:13.760085  / # . /lava-3415495/environment/lava-3415495/bin/lava-test-runner /lava-3415495/1
    2023-03-15T23:36:13.762968  
    2023-03-15T23:36:13.767796  / # /lava-3415495/bin/lava-test-runner /lava-3415495/1
    2023-03-15T23:36:13.893156  + export 'TESTRUN_ID=1_bootrr' 
    ... (11 line(s) more)  

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6412579cbc55e2189c8c866d

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v6.3-rc2-369-g71c206ed429b/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20230310.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6412579cbc55e2189c8c866e
        failing since 90 days (last pass: v6.1-6378-g44d433ee7540, first fail: v6.1-10792-g244d284981da)