Subject: next/master baseline: 657 runs, 32 regressions (next-20221208)

From: "kernelci.org bot" <[email-masked]>
Date: Thu, 08 Dec 2022 01:34:12 -0800 (PST)

next/master baseline: 657 runs, 32 regressions (next-20221208)

Regressions Summary
-------------------

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          
beagle-xm                    | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          
imx6dl-riotboard             | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          
kontron-pitx-imx8m           | arm64  | lab-kontron     | clang-16 | defconfig+CON..._64K_PAGES=y | 1          
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig+CON..._64K_PAGES=y | 1          
meson-gxm-khadas-vim2        | arm64  | lab-baylibre    | gcc-10   | defconfig+ima                | 1          
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          
odroid-xu3                   | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig           | 1          
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          
qemu_x86_64                  | x86_64 | lab-broonie     | gcc-10   | x86_64_defconfig+debug       | 1          
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          
sun50i-a64-bananapi-m64      | arm64  | lab-clabbe      | clang-16 | defconfig                    | 1          
sun50i-a64-bananapi-m64      | arm64  | lab-clabbe      | gcc-10   | defconfig+crypto             | 1          
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:  https://kernelci.org/test/job/next/branch/master/kernel/next-20221208/plan/baseline/

  Test:     baseline
  Tree:     next
  Branch:   master
  Describe: next-20221208
  URL:      https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
  SHA:      f925116b24c0c42dc6d5ab5111c55fd7f74e8dc7 

Test Regressions
---------------- 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defc...CONFIG_SMP=n | 1          

  Details:     https://kernelci.org/test/plan/id/63918726f07f5df7702abcfa

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_SMP=n
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63918726f07f5df7702abcfb
        failing since 82 days (last pass: next-20220915, first fail: next-20220916) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
am57xx-beagle-x15            | arm    | lab-linaro-lkft | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63918902c7cccd749c2abd02

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-linaro-lkft/baseline-am57xx-beagle-x15.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63918902c7cccd749c2abd03
        failing since 84 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
beagle-xm                    | arm    | lab-baylibre    | gcc-10   | multi_v7_defc...MB2_KERNEL=y | 1          

  Details:     https://kernelci.org/test/plan/id/63917b80c9cdc09b822abd57

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-beagle-xm.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_THUMB2_KERNEL=y/gcc-10/lab-baylibre/baseline-beagle-xm.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917b80c9cdc09b822abd58
        new failure (last pass: next-20221017) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6dl-riotboard             | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63917bd0d9f185b0a62abd1c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6dl-riotboard.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6dl-riotboard.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917bd0d9f185b0a62abd1d
        failing since 120 days (last pass: next-20220808, first fail: next-20220810) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
imx6qp-wandboard-revd1       | arm    | lab-pengutronix | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/639179f0a5b3ccec9e2abcfa

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-pengutronix/baseline-imx6qp-wandboard-revd1.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639179f0a5b3ccec9e2abcfb
        failing since 107 days (last pass: next-20220810, first fail: next-20220822) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-pitx-imx8m           | arm64  | lab-kontron     | clang-16 | defconfig+CON..._64K_PAGES=y | 1          

  Details:     https://kernelci.org/test/plan/id/63917d9dc7d6952c0a2abd8e

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+CONFIG_ARM64_64K_PAGES=y
  Compiler:    clang-16 (Debian clang version 16.0.0 (++20221205071824+e95ca5bb05ec-1~exp1~20221205071939.483))
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+CONFIG_ARM64_64K_PAGES=y/clang-16/lab-kontron/baseline-kontron-pitx-imx8m.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+CONFIG_ARM64_64K_PAGES=y/clang-16/lab-kontron/baseline-kontron-pitx-imx8m.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917d9dc7d6952c0a2abd8f
        failing since 36 days (last pass: next-20221028, first fail: next-20221101) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
kontron-pitx-imx8m           | arm64  | lab-kontron     | gcc-10   | defconfig+CON..._64K_PAGES=y | 1          

  Details:     https://kernelci.org/test/plan/id/63917a07a5b3ccec9e2abd27

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+CONFIG_ARM64_64K_PAGES=y
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+CONFIG_ARM64_64K_PAGES=y/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+CONFIG_ARM64_64K_PAGES=y/gcc-10/lab-kontron/baseline-kontron-pitx-imx8m.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917a07a5b3ccec9e2abd28
        failing since 28 days (last pass: next-20221107, first fail: next-20221109) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
meson-gxm-khadas-vim2        | arm64  | lab-baylibre    | gcc-10   | defconfig+ima                | 1          

  Details:     https://kernelci.org/test/plan/id/63917c4f170772e1a92abd10

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+ima
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+ima/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+ima/gcc-10/lab-baylibre/baseline-meson-gxm-khadas-vim2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917c4f170772e1a92abd11
        new failure (last pass: next-20221206) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63917f3b93ef761bc62abcfa

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: cros://chromeos-5.10/arm64/chromiumos-arm64.flavour.config+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917f3b93ef761bc62abcfb
        failing since 1 day (last pass: next-20221205, first fail: next-20221206) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/639181903805b4cc5d2abd89

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: cros://chromeos-5.10/arm64/chromiumos-mediatek.flavour.config+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-mediatek.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-mediatek.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639181903805b4cc5d2abd8a
        failing since 1 day (last pass: next-20221205, first fail: next-20221206) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8173-elm-hana              | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          

  Details:     https://kernelci.org/test/plan/id/63918217e866e1b67a2abd12

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-mt8173-elm-hana.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63918217e866e1b67a2abd13
        failing since 1 day (last pass: next-20221205, first fail: next-20221206) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63917f397f385cee222abcfe

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: cros://chromeos-5.10/arm64/chromiumos-arm64.flavour.config+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917f397f385cee222abcff
        failing since 21 days (last pass: next-20221114, first fail: next-20221116) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63918197821bdc96742abd61

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: cros://chromeos-5.10/arm64/chromiumos-mediatek.flavour.config+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-mediatek.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-mediatek.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63918197821bdc96742abd62
        failing since 21 days (last pass: next-20221114, first fail: next-20221116) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8183-kukui-...uniper-sku16 | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          

  Details:     https://kernelci.org/test/plan/id/63918155e434656f032abd11

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-mt8183-kukui-jacuzzi-juniper-sku16.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63918155e434656f032abd12
        failing since 209 days (last pass: next-20220506, first fail: next-20220512) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
mt8192-asurada-spherion-r0   | arm64  | lab-collabora   | clang-13 | cros://chrome...4-chromebook | 1          

  Details:     https://kernelci.org/test/plan/id/63917e997e37bfe2682abd09

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: cros://chromeos-5.10/arm64/chromiumos-arm64.flavour.config+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8192-asurada-spherion-r0.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/cros---chromeos-5.10-arm64-chromiumos-arm64.flavour.config+arm64-chromebook/clang-13/lab-collabora/baseline-mt8192-asurada-spherion-r0.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917e997e37bfe2682abd0a
        new failure (last pass: next-20221207) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
odroid-xu3                   | arm    | lab-collabora   | gcc-10   | multi_v7_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/63917770dccfe9378c2abcfc

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig/gcc-10/lab-collabora/baseline-odroid-xu3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917770dccfe9378c2abcfd
        new failure (last pass: next-20221207) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
ox820-clouden...lug-series-3 | arm    | lab-baylibre    | gcc-10   | oxnas_v6_defconfig           | 1          

  Details:     https://kernelci.org/test/plan/id/6391719be88208b6ac2abcfe

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: oxnas_v6_defconfig
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/oxnas_v6_defconfig/gcc-10/lab-baylibre/baseline-ox820-cloudengines-pogoplug-series-3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6391719be88208b6ac2abcff
        failing since 106 days (last pass: next-20220822, first fail: next-20220823) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/639179d12f875990402abd27

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639179d12f875990402abd28
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63917b106c5c2b892f2abd4d

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917b106c5c2b892f2abd4e
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/639179d46d50d7c7ad2abd17

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639179d46d50d7c7ad2abd18
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv2-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63917b3842a304fbd52abd07

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv2-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917b3842a304fbd52abd08
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/639179d078c96a9a6d2abd07

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639179d078c96a9a6d2abd08
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3          | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63917afca0d61204012abd0a

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917afca0d61204012abd0b
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-baylibre    | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/639179d26d50d7c7ad2abd12

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-baylibre/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639179d26d50d7c7ad2abd13
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_arm-virt-gicv3-uefi     | arm    | lab-broonie     | gcc-10   | multi_v7_defconfig+debug     | 1          

  Details:     https://kernelci.org/test/plan/id/63917b255be504080a2abd15

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+debug
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_arm-virt-gicv3-uefi.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917b255be504080a2abd16
        failing since 85 days (last pass: next-20220907, first fail: next-20220913) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
qemu_x86_64                  | x86_64 | lab-broonie     | gcc-10   | x86_64_defconfig+debug       | 1          

  Details:     https://kernelci.org/test/plan/id/639178a4a3a8f8170c2abcfe

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: x86_64_defconfig+debug
  Compiler:    gcc-10 (gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/x86_64/x86_64_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_x86_64.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/x86_64/x86_64_defconfig+debug/gcc-10/lab-broonie/baseline-qemu_x86_64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/x86/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639178a4a3a8f8170c2abcff
        failing since 8 days (last pass: next-20221128, first fail: next-20221129) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-rock2-square          | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/639181c1821bdc96742abd7c

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-rock2-square.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639181c1821bdc96742abd7d
        failing since 15 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
rk3288-veyron-jaq            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/6391a74732545b787f2abd00

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-rk3288-veyron-jaq.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6391a74732545b787f2abd01
        failing since 15 days (last pass: next-20221121, first fail: next-20221122) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sc7180-trogdo...zor-limozeen | arm64  | lab-collabora   | clang-13 | defconfig+arm64-chromebook   | 1          

  Details:     https://kernelci.org/test/plan/id/6391816ae434656f032abd1f

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+arm64-chromebook
  Compiler:    clang-13 (Debian clang version 13.0.1-6~deb11u1)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+arm64-chromebook/clang-13/lab-collabora/baseline-sc7180-trogdor-lazor-limozeen.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/6391816ae434656f032abd20
        new failure (last pass: next-20221206) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun50i-a64-bananapi-m64      | arm64  | lab-clabbe      | clang-16 | defconfig                    | 1          

  Details:     https://kernelci.org/test/plan/id/63917d4cf4980298142abd95

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig
  Compiler:    clang-16 (Debian clang version 16.0.0 (++20221205071824+e95ca5bb05ec-1~exp1~20221205071939.483))
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig/clang-16/lab-clabbe/baseline-sun50i-a64-bananapi-m64.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig/clang-16/lab-clabbe/baseline-sun50i-a64-bananapi-m64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/63917d4cf4980298142abd96
        new failure (last pass: next-20221109) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
sun50i-a64-bananapi-m64      | arm64  | lab-clabbe      | gcc-10   | defconfig+crypto             | 1          

  Details:     https://kernelci.org/test/plan/id/639174b60f42b5495d2abd63

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: defconfig+crypto
  Compiler:    gcc-10 (aarch64-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+crypto/gcc-10/lab-clabbe/baseline-sun50i-a64-bananapi-m64.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm64/defconfig+crypto/gcc-10/lab-clabbe/baseline-sun50i-a64-bananapi-m64.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/arm64/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639174b60f42b5495d2abd64
        new failure (last pass: next-20221109) 

platform                     | arch   | lab             | compiler | defconfig                    | regressions
-----------------------------+--------+-----------------+----------+------------------------------+------------
tegra124-nyan-big            | arm    | lab-collabora   | gcc-10   | multi_v7_defc...G_ARM_LPAE=y | 1          

  Details:     https://kernelci.org/test/plan/id/639185d388a742a37a2abd17

  Results:     0 PASS, 1 FAIL, 0 SKIP
  Full config: multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y
  Compiler:    gcc-10 (arm-linux-gnueabihf-gcc (Debian 10.2.1-6) 10.2.1 20210110)
  Plain log:   https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.txt
  HTML log:    https://storage.kernelci.org//next/master/next-20221208/arm/multi_v7_defconfig+CONFIG_EFI=y+CONFIG_ARM_LPAE=y/gcc-10/lab-collabora/baseline-tegra124-nyan-big.html
  Rootfs:      http://storage.kernelci.org/images/rootfs/buildroot/buildroot-baseline/20221125.0/armel/rootfs.cpio.gz 

  * baseline.login: https://kernelci.org/test/case/id/639185d388a742a37a2abd18
        failing since 14 days (last pass: next-20221121, first fail: next-20221123)