Subject: next/pending-fixes baseline: 477 runs, 29 regressions (v5.19-rc6-441-g4822e65f1f95)

From: "kernelci.org bot" <[email-masked]>
Date: Wed, 13 Jul 2022 21:01:14 -0700 (PDT)

next/pending-fixes baseline: 477 runs, 29 regressions (v5.19-rc6-441-g4822e65f1f95)

Regressions Summary
-------------------

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
da850-lcdk                 | arm    | lab-baylibre  | gcc-10   | davinci_all_defconfig        | 1          
imx8mn-ddr4-evk            | arm64  | lab-baylibre  | gcc-10   | defconfig                    | 1          
kontron-kbox-a-230-ls      | arm64  | lab-kontron   | gcc-10   | defconfig+debug              | 1          
mt8173-elm-hana            | arm64  | lab-collabora | gcc-10   | defconfig+arm64-chromebook   | 1          
qemu_arm64-virt-gicv2-uefi | arm64  | lab-baylibre  | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv2-uefi | arm64  | lab-broonie   | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv3-uefi | arm64  | lab-baylibre  | gcc-10   | defconfig+debug              | 1          
qemu_arm64-virt-gicv3-uefi | arm64  | lab-broonie   | gcc-10   | defconfig+debug              | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+crypto      | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+ima         | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig             | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...6-chromebook | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...86_kvm_guest | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+debug       | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+amdgpu      | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+crypto      | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+ima         | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig             | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...6-chromebook | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...86_kvm_guest | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+debug       | 1          
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+amdgpu      | 1          
r8a77950-salvator-x        | arm64  | lab-baylibre  | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          
rk3399-gru-kevin           | arm64  | lab-collabora | gcc-10   | defconfig+arm64-chromebook   | 4          

  Details:  https://kernelci.org/test/job/next/branch/pending-fixes/kernel/v5.19-rc6-441-g4822e65f1f95/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   pending-fixes
  Describe: v5.19-rc6-441-g4822e65f1f95
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      4822e65f1f95faeed22b61d56cd912fd6d49222a 

Test Regressions
---------------- 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
da850-lcdk                 | arm    | lab-baylibre  | gcc-10   | davinci_all_defconfig        | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5b5180d1ce5382a39bd1

  Results:     5 PASS, 1 FAIL, 1 SKIP
  Full config: davinci_all_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm/davinci_all_defconfig/gcc-10/lab-baylibre/baseline-da850-lcdk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/armel/rootfs.cpio.gz 

  * baseline.dmesg.alert: https://kernelci.org/test/case/id/62cf5b5280d1ce5382a39bd9
        failing since 171 days (last pass: v5.16-11577-gffd79fec234d, first fail: v5.17-rc1-180-g86539e2bdb99)
        3 lines

    2022-07-13T23:54:27.596341  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3000
    2022-07-13T23:54:27.596809  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3400
    2022-07-13T23:54:27.599458  kern  :alert : BUG: Bad page state in process kworker/u2:0  pfn:c3800
    2022-07-13T23:54:27.642972  <8><LAVA_SIGNAL_TESTCASE TEST_CASE_ID=alert RESULT=fail UNITS=lines MEASUREMENT=3>   

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
imx8mn-ddr4-evk            | arm64  | lab-baylibre  | gcc-10   | defconfig                    | 1          

  Details:     https://kernelci.org/test/plan/id/62cf65acb7618ab8eca39bda

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig/gcc-10/lab-baylibre/baseline-imx8mn-ddr4-evk.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig/gcc-10/lab-baylibre/baseline-imx8mn-ddr4-evk.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf65acb7618ab8eca39bdb
        new failure (last pass: v5.19-rc5-383-g73ad9bd963c1) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
kontron-kbox-a-230-ls      | arm64  | lab-kontron   | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5d538ce243d2aaa39c04

  Results:     94 PASS, 1 FAIL, 1 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-kontron/baseline-kontron-kbox-a-230-ls.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.mscc_felix-probed: https://kernelci.org/test/case/id/62cf5d538ce243d2aaa39c0d
        failing since 44 days (last pass: v5.18-5613-ge079cf8f6817, first fail: v5.18-11575-gceccc06b624ed)

    2022-07-14T00:03:04.828121  /lava-140517/1/../bin/lava-test-case
    2022-07-14T00:03:04.918004  <8>[   41.201350][  T311] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=mscc_felix-probed RESULT=fail>   

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
mt8173-elm-hana            | arm64  | lab-collabora | gcc-10   | defconfig+arm64-chromebook   | 1          

  Details:     https://kernelci.org/test/plan/id/62cf6280cd890584bca39bdc

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf6280cd890584bca39bdd
        new failure (last pass: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64  | lab-baylibre  | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5e7455cbf6b0b0a39bec

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5e7455cbf6b0b0a39bed
        failing since 63 days (last pass: v5.18-rc5-178-g63184bc90d435, first fail: v5.18-rc6-217-gd854612e01df) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv2-uefi | arm64  | lab-broonie   | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5e8355cbf6b0b0a39c4b

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5e8355cbf6b0b0a39c4c
        failing since 63 days (last pass: v5.18-rc5-178-g63184bc90d435, first fail: v5.18-rc6-217-gd854612e01df) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64  | lab-baylibre  | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5e7255cbf6b0b0a39be6

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm64-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5e7255cbf6b0b0a39be7
        failing since 63 days (last pass: v5.18-rc5-178-g63184bc90d435, first fail: v5.18-rc6-217-gd854612e01df) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_arm64-virt-gicv3-uefi | arm64  | lab-broonie   | gcc-10   | defconfig+debug              | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5e5b415a458631a39c12

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+debug
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm64-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5e5b415a458631a39c13
        failing since 63 days (last pass: v5.18-rc5-178-g63184bc90d435, first fail: v5.18-rc6-217-gd854612e01df) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+crypto      | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5acbcae67c09ffa39c0a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+crypto
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+crypto/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+crypto/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5acbcae67c09ffa39c0b
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+ima         | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5b6b236d98d049a39be1

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+ima
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+ima/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+ima/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5b6b236d98d049a39be2
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig             | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5be51c7a763323a39bda

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5be51c7a763323a39bdb
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5be8e92e97e4b3a39bfd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+amdgpu
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5be8e92e97e4b3a39bfe
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...6-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5d156d774901cba39be5

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5d156d774901cba39be6
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defcon...86_kvm_guest | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5dba71b993ac6ea39bce

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86_kvm_guest
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86_kvm_guest/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86_kvm_guest/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5dba71b993ac6ea39bcf
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+debug       | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5f6a50268c44efa39bcf

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+debug
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5f6a50268c44efa39bd0
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-baylibre  | gcc-10   | x86_64_defconfig+amdgpu      | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5fc476475d4d51a39c3a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+amdgpu
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+amdgpu/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+amdgpu/gcc-10/lab-baylibre/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5fc476475d4d51a39c3b
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+crypto      | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5aaecae67c09ffa39bdb

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+crypto
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+crypto/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+crypto/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5aaecae67c09ffa39bdc
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+ima         | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5b775c6791b90ca39be3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+ima
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+ima/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+ima/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5b775c6791b90ca39be4
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig             | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5c0468f542f8e8a39bdb

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5c0468f542f8e8a39bdc
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...ebook+amdgpu | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5c3f61239b19a8a39bf3

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook+amdgpu
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook+amdgpu/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5c3f61239b19a8a39bf4
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...6-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5d2f7ee71fc80da39c17

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86-chromebook
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86-chromebook/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5d2f7ee71fc80da39c18
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defcon...86_kvm_guest | 1          

  Details:     https://kernelci.org/test/plan/id/62cf5f37b213b59347a39bce

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+x86_kvm_guest
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86_kvm_guest/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+x86_kvm_guest/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf5f37b213b59347a39bcf
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+debug       | 1          

  Details:     https://kernelci.org/test/plan/id/62cf60b52eb85002dfa39bcd

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+debug
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf60b52eb85002dfa39bce
        new failure (last pass: v5.19-rc6-297-g1e717e841472) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
qemu_x86_64-uefi-mixed     | x86_64 | lab-broonie   | gcc-10   | x86_64_defconfig+amdgpu      | 1          

  Details:     https://kernelci.org/test/plan/id/62cf621d59572625cda39bf8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+amdgpu
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+amdgpu/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/x86_64/x86_64_defconfig+amdgpu/gcc-10/lab-broonie/baseline-qemu_x86_64-uefi-mixed.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf621d59572625cda39bf9
        failing since 0 day (last pass: v5.19-rc6-297-g1e717e841472, first fail: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
r8a77950-salvator-x        | arm64  | lab-baylibre  | gcc-10   | defconfig+CON...BIG_ENDIAN=y | 1          

  Details:     https://kernelci.org/test/plan/id/62cf6785c8b30a2dffa39bd8

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+CONFIG_CPU_BIG_ENDIAN=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-r8a77950-salvator-x.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+CONFIG_CPU_BIG_ENDIAN=y/gcc-10/lab-baylibre/baseline-r8a77950-salvator-x.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64be/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/62cf6785c8b30a2dffa39bd9
        new failure (last pass: v5.19-rc6-382-gec832b259e17) 

platform                   | arch   | lab           | compiler | defconfig                    | regressions
---------------------------+--------+---------------+----------+------------------------------+------------
rk3399-gru-kevin           | arm64  | lab-collabora | gcc-10   | defconfig+arm64-chromebook   | 4          

  Details:     https://kernelci.org/test/plan/id/62cf620b8c6fbbab72a39bef

  Results:     85 PASS, 7 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.txt
  HTML log:    https://storage.kernelci.org//next/pending-fixes/v5.19-rc6-441-g4822e65f1f95/arm64/defconfig+arm64-chromebook/gcc-10/lab-collabora/baseline-rk3399-gru-kevin.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20220708.0/arm64/rootfs.cpio.gz 

  * baseline.bootrr.rockchip-i2s1-probed: https://kernelci.org/test/case/id/62cf620b8c6fbbab72a39c11
        failing since 135 days (last pass: v5.17-rc5-244-gd77a1b37f796, first fail: v5.17-rc6-176-gb4e03e0dde48)

    2022-07-14T00:23:21.525536  <8>[   53.919783] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=rockchip-i2s0-probed RESULT=pass>
    2022-07-14T00:23:22.549386  /lava-6814398/1/../bin/lava-test-case   

  * baseline.bootrr.cros-ec-sensors-gyro0-probed: https://kernelci.org/test/case/id/62cf620b8c6fbbab72a39c34
        failing since 44 days (last pass: v5.18-5613-ge079cf8f6817, first fail: v5.18-11575-gceccc06b624ed)

    2022-07-14T00:23:19.349246  <8>[   51.742486] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=cros-ec-sensors-accel1-probed RESULT=fail>
    2022-07-14T00:23:20.378410  /lava-6814398/1/../bin/lava-test-case   

  * baseline.bootrr.cros-ec-sensors-accel1-probed: https://kernelci.org/test/case/id/62cf620b8c6fbbab72a39c35
        failing since 44 days (last pass: v5.18-5613-ge079cf8f6817, first fail: v5.18-11575-gceccc06b624ed)

    2022-07-14T00:23:18.305293  <8>[   50.698675] <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=cros-ec-sensors-accel0-probed RESULT=fail>
    2022-07-14T00:23:19.336812  /lava-6814398/1/../bin/lava-test-case   

  * baseline.bootrr.cros-ec-sensors-accel0-probed: https://kernelci.org/test/case/id/62cf620b8c6fbbab72a39c36
        failing since 44 days (last pass: v5.18-5613-ge079cf8f6817, first fail: v5.18-11575-gceccc06b624ed)

    2022-07-14T00:23:18.293515  /lava-6814398/1/../bin/lava-test-case