linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [RFC v4] drm: Add initial ci/ subdirectory
       [not found] <20220511061533.56881-1-tomeu.vizoso@collabora.com>
@ 2022-05-17  8:16 ` Tomeu Vizoso
  2022-05-17  9:18   ` Neil Armstrong
  2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
  0 siblings, 2 replies; 13+ messages in thread
From: Tomeu Vizoso @ 2022-05-17  8:16 UTC (permalink / raw)
  To: David Airlie, Jonathan Corbet, Carlo Caione, Kevin Hilman,
	Heiko Stuebner, Matthias Brugger
  Cc: Tomeu Vizoso, dri-devel, linux-doc, linux-kernel,
	linux-arm-kernel, linux-amlogic, linux-rockchip, linux-mediatek

And use it to store expectations about what the DRM drivers are
supposed to pass in the IGT test suite.

Also include a configuration file that points to the out-of-tree CI
scripts.

By storing the test expectations along the code we can make sure both
stay in sync with each other, and so we can know when a code change
breaks those expectations.

This will allow all contributors to drm to reuse the infrastructure
already in gitlab.freedesktop.org to test the driver on several
generations of the hardware.

v2:
  - Fix names of result expectation files to match SoC
  - Don't execute tests that are going to skip on all boards

v3:
  - Remove tracking of dmesg output during test execution

v4:
  - Move up to drivers/gpu/drm
  - Add support for a bunch of other drivers
  - Explain how to incorporate fixes for CI from a
    ${TARGET_BRANCH}-external-fixes branch
  - Remove tests that pass from expected results file, to reduce the
    size of in-tree files
  - Add docs about how to deal with outages in automated testing labs
  - Specify the exact SHA of the CI scripts to be used

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
---
 Documentation/gpu/automated_testing.rst       |  84 +++++++
 drivers/gpu/drm/ci/amdgpu.testlist            | 130 +++++++++++
 drivers/gpu/drm/ci/amdgpu_stoney_results.txt  |  17 ++
 drivers/gpu/drm/ci/gitlab-ci.yml              |  13 ++
 drivers/gpu/drm/ci/i915.testlist              | 219 ++++++++++++++++++
 drivers/gpu/drm/ci/i915_amly_results.txt      |   9 +
 drivers/gpu/drm/ci/i915_apl_results.txt       |   9 +
 drivers/gpu/drm/ci/i915_cml_results.txt       |  10 +
 drivers/gpu/drm/ci/i915_glk_results.txt       |  13 ++
 drivers/gpu/drm/ci/i915_kbl_results.txt       |   9 +
 drivers/gpu/drm/ci/i915_tgl_results.txt       |  12 +
 drivers/gpu/drm/ci/i915_whl_results.txt       |   9 +
 drivers/gpu/drm/ci/mediatek.testlist          | 102 ++++++++
 .../gpu/drm/ci/mediatek_mt8173_results.txt    |  11 +
 .../gpu/drm/ci/mediatek_mt8183_results.txt    |  11 +
 .../gpu/drm/ci/mediatek_mt8192_results.txt    |  11 +
 drivers/gpu/drm/ci/meson.testlist             |  71 ++++++
 drivers/gpu/drm/ci/meson_g12b_results.txt     |  11 +
 drivers/gpu/drm/ci/msm.testlist               | 144 ++++++++++++
 drivers/gpu/drm/ci/msm_apq8016_results.txt    |  77 ++++++
 drivers/gpu/drm/ci/msm_apq8096_results.txt    |  76 ++++++
 drivers/gpu/drm/ci/msm_sc7180_results.txt     |  18 ++
 drivers/gpu/drm/ci/msm_sdm845_results.txt     |  33 +++
 drivers/gpu/drm/ci/rockchip.testlist          | 188 +++++++++++++++
 .../gpu/drm/ci/rockchip_rk3288_results.txt    |  11 +
 .../gpu/drm/ci/rockchip_rk3399_results.txt    |  14 ++
 drivers/gpu/drm/ci/virtio_gpu.testlist        |  92 ++++++++
 drivers/gpu/drm/ci/virtio_gpu_results.txt     |  28 +++
 28 files changed, 1432 insertions(+)
 create mode 100644 Documentation/gpu/automated_testing.rst
 create mode 100644 drivers/gpu/drm/ci/amdgpu.testlist
 create mode 100644 drivers/gpu/drm/ci/amdgpu_stoney_results.txt
 create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
 create mode 100644 drivers/gpu/drm/ci/i915.testlist
 create mode 100644 drivers/gpu/drm/ci/i915_amly_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_apl_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_cml_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_glk_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_kbl_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_tgl_results.txt
 create mode 100644 drivers/gpu/drm/ci/i915_whl_results.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek.testlist
 create mode 100644 drivers/gpu/drm/ci/mediatek_mt8173_results.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek_mt8183_results.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek_mt8192_results.txt
 create mode 100644 drivers/gpu/drm/ci/meson.testlist
 create mode 100644 drivers/gpu/drm/ci/meson_g12b_results.txt
 create mode 100644 drivers/gpu/drm/ci/msm.testlist
 create mode 100644 drivers/gpu/drm/ci/msm_apq8016_results.txt
 create mode 100644 drivers/gpu/drm/ci/msm_apq8096_results.txt
 create mode 100644 drivers/gpu/drm/ci/msm_sc7180_results.txt
 create mode 100644 drivers/gpu/drm/ci/msm_sdm845_results.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip.testlist
 create mode 100644 drivers/gpu/drm/ci/rockchip_rk3288_results.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip_rk3399_results.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu.testlist
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu_results.txt

diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
new file mode 100644
index 000000000000..61003ecc7b6e
--- /dev/null
+++ b/Documentation/gpu/automated_testing.rst
@@ -0,0 +1,84 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+=========================================
+Automated testing of the DRM subsystem
+=========================================
+
+
+Introduction
+============
+
+Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
+
+Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
+
+There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
+
+
+Relevant files
+==============
+
+drivers/gpu/drm/ci/gitlab-ci.yml
+--------------------------------
+
+Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
+------------------------------------------
+
+Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
+----------------------------------------
+
+Specifies the expected results of running this specific kernel revision on a given hardware configuration.
+
+
+How to enable automated testing on your tree
+============================================
+
+1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
+
+2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
+
+3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
+
+4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
+
+
+How to update test expectations
+===============================
+
+If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
+
+If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
+
+
+How to expand coverage
+======================
+
+If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
+
+If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
+
+
+How to test your changes to the scripts
+==========================================
+
+For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
+
+
+How to incorporate external fixes in your testing
+=================================================
+
+Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
+
+If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
+
+
+How to deal with automated testing labs that may be down
+========================================================
+
+If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
diff --git a/drivers/gpu/drm/ci/amdgpu.testlist b/drivers/gpu/drm/ci/amdgpu.testlist
new file mode 100644
index 000000000000..c311ad590e2f
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu.testlist
@@ -0,0 +1,130 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@drm_read@fault-buffer
+igt@drm_read@empty-block
+igt@drm_read@empty-nonblock
+igt@drm_read@short-buffer-block
+igt@drm_read@short-buffer-nonblock
+igt@drm_read@short-buffer-wakeup
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_hdr@bpc-switch
+igt@kms_hdr@bpc-switch-dpms
+igt@kms_pipe_crc_basic@bad-source
+igt@kms_pipe_crc_basic@read-crc-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-b
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b
+igt@kms_plane_lowres@pipe-a-tiling-none
+igt@kms_plane_lowres@pipe-b-tiling-none
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+
+# Flaky on Stoney
+# [drm] REG_WAIT timeout 10us * 3500 tries - dce_mi_free_dmif line:782
+# igt@kms_rotation_crc@primary-rotation-180
+
+
+# Get a warning some times on Stoney
+# igt@kms_setmode@basic@pipe-a-edp-1
+# igt@kms_setmode@basic@pipe-b-edp-1
+
+igt@kms_sysfs_edid_timing
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-modeset
+igt@kms_vblank@pipe-b-accuracy-idle
+igt@kms_vblank@pipe-b-query-idle
+igt@kms_vblank@pipe-b-query-forked
+igt@kms_vblank@pipe-b-query-busy
+igt@kms_vblank@pipe-b-query-forked-busy
+igt@kms_vblank@pipe-b-wait-idle
+igt@kms_vblank@pipe-b-wait-forked
+igt@kms_vblank@pipe-b-wait-busy
+igt@kms_vblank@pipe-b-wait-forked-busy
+igt@kms_vblank@pipe-b-ts-continuation-idle
+igt@kms_vblank@pipe-b-ts-continuation-modeset
diff --git a/drivers/gpu/drm/ci/amdgpu_stoney_results.txt b/drivers/gpu/drm/ci/amdgpu_stoney_results.txt
new file mode 100644
index 000000000000..8f9fb6201011
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu_stoney_results.txt
@@ -0,0 +1,17 @@
+igt@kms_addfb_basic@bad-pitch-65536,fail
+igt@kms_addfb_basic@too-high,fail
+igt@kms_addfb_basic@bo-too-small,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdr@bpc-switch@bpc-switch-edp-1-pipe-a,skip
+igt@kms_hdr@bpc-switch-dpms@bpc-switch-dpms-edp-1-pipe-a,skip
+igt@kms_plane_lowres@pipe-a-tiling-none,incomplete
+igt@kms_plane_lowres@pipe-b-tiling-none,incomplete
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a,fail
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,fail
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
new file mode 100644
index 000000000000..e7464d8d442e
--- /dev/null
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -0,0 +1,13 @@
+variables:
+  # Change this to use your fork of drm-ci
+  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
+  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 6e01c5e9f0eea910e9fb7ce475cdef6cc719caf3
+
+  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
+  TARGET_BRANCH: drm-next
+
+include:
+  - project: *drm-ci-project-path
+    ref: *drm-ci-commit-sha
+    file:
+      - '.gitlab-ci.yml'
diff --git a/drivers/gpu/drm/ci/i915.testlist b/drivers/gpu/drm/ci/i915.testlist
new file mode 100644
index 000000000000..92fa28e77f5d
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915.testlist
@@ -0,0 +1,219 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@drm_read@fault-buffer
+igt@drm_read@empty-block
+igt@drm_read@empty-nonblock
+igt@drm_read@short-buffer-block
+igt@drm_read@short-buffer-nonblock
+igt@drm_read@short-buffer-wakeup
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@clobberred-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@bo-too-small-due-to-tiling
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@addfb25-x-tiled-mismatch-legacy
+igt@kms_addfb_basic@addfb25-x-tiled-legacy
+igt@kms_addfb_basic@addfb25-framebuffer-vs-set-tiling
+igt@kms_addfb_basic@addfb25-y-tiled-legacy
+igt@kms_addfb_basic@addfb25-yf-tiled-legacy
+igt@kms_addfb_basic@addfb25-y-tiled-small-legacy
+igt@kms_addfb_basic@addfb25-4-tiled
+igt@kms_addfb_basic@basic-x-tiled-legacy
+igt@kms_addfb_basic@framebuffer-vs-set-tiling
+igt@kms_addfb_basic@tile-pitch-mismatch
+igt@kms_addfb_basic@basic-y-tiled-legacy
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb-reject-ccs
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-accept-ccs
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_hdmi_inject@inject-4k
+igt@kms_hdr@bpc-switch
+igt@kms_hdr@bpc-switch-dpms
+igt@kms_pipe_crc_basic@bad-source
+igt@kms_pipe_crc_basic@read-crc-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a
+igt@kms_pipe_crc_basic@hang-read-crc-pipe-a
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-b
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b
+igt@kms_pipe_crc_basic@hang-read-crc-pipe-b
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b
+igt@kms_pipe_crc_basic@read-crc-pipe-c
+igt@kms_pipe_crc_basic@read-crc-pipe-c-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-c
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-c-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-c
+igt@kms_pipe_crc_basic@hang-read-crc-pipe-c
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+igt@kms_rotation_crc@primary-rotation-90
+igt@kms_rotation_crc@primary-rotation-180
+igt@kms_rotation_crc@primary-rotation-270
+igt@kms_rotation_crc@sprite-rotation-90
+igt@kms_rotation_crc@sprite-rotation-180
+igt@kms_rotation_crc@sprite-rotation-270
+igt@kms_rotation_crc@cursor-rotation-180
+igt@kms_rotation_crc@sprite-rotation-90-pos-100-0
+igt@kms_rotation_crc@bad-pixel-format
+igt@kms_rotation_crc@bad-tiling
+igt@kms_rotation_crc@multiplane-rotation
+igt@kms_rotation_crc@multiplane-rotation-cropping-top
+igt@kms_rotation_crc@multiplane-rotation-cropping-bottom
+igt@kms_rotation_crc@exhaust-fences
+igt@kms_setmode@basic@pipe-b-edp-1
+igt@kms_sysfs_edid_timing
+igt@kms_universal_plane@universal-plane-pipe-a-functional
+igt@kms_universal_plane@universal-plane-pipe-a-sanity
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a
+igt@kms_universal_plane@cursor-fb-leak-pipe-a
+igt@kms_universal_plane@universal-plane-gen9-features-pipe-a
+igt@kms_universal_plane@universal-plane-pipe-b-functional
+igt@kms_universal_plane@universal-plane-pipe-b-sanity
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b
+igt@kms_universal_plane@cursor-fb-leak-pipe-b
+igt@kms_universal_plane@universal-plane-gen9-features-pipe-b
+igt@kms_universal_plane@universal-plane-pipe-c-functional
+igt@kms_universal_plane@universal-plane-pipe-c-sanity
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-c
+igt@kms_universal_plane@cursor-fb-leak-pipe-c
+igt@kms_universal_plane@universal-plane-gen9-features-pipe-c
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-idle-hang
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-forked-hang
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-busy-hang
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-query-forked-busy-hang
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-idle-hang
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-forked-hang
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-busy-hang
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-wait-forked-busy-hang
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-idle-hang
+igt@kms_vblank@pipe-a-ts-continuation-modeset
+igt@kms_vblank@pipe-a-ts-continuation-modeset-hang
+igt@kms_vblank@pipe-b-accuracy-idle
+igt@kms_vblank@pipe-b-query-idle
+igt@kms_vblank@pipe-b-query-idle-hang
+igt@kms_vblank@pipe-b-query-forked
+igt@kms_vblank@pipe-b-query-forked-hang
+igt@kms_vblank@pipe-b-query-busy
+igt@kms_vblank@pipe-b-query-busy-hang
+igt@kms_vblank@pipe-b-query-forked-busy
+igt@kms_vblank@pipe-b-query-forked-busy-hang
+igt@kms_vblank@pipe-b-wait-idle
+igt@kms_vblank@pipe-b-wait-idle-hang
+igt@kms_vblank@pipe-b-wait-forked
+igt@kms_vblank@pipe-b-wait-forked-hang
+igt@kms_vblank@pipe-b-wait-busy
+igt@kms_vblank@pipe-b-wait-busy-hang
+igt@kms_vblank@pipe-b-wait-forked-busy
+igt@kms_vblank@pipe-b-wait-forked-busy-hang
+igt@kms_vblank@pipe-b-ts-continuation-idle
+igt@kms_vblank@pipe-b-ts-continuation-idle-hang
+igt@kms_vblank@pipe-b-ts-continuation-modeset
+igt@kms_vblank@pipe-b-ts-continuation-modeset-hang
+igt@kms_vblank@pipe-c-accuracy-idle
+igt@kms_vblank@pipe-c-query-idle
+igt@kms_vblank@pipe-c-query-idle-hang
+igt@kms_vblank@pipe-c-query-forked
+igt@kms_vblank@pipe-c-query-forked-hang
+igt@kms_vblank@pipe-c-query-busy
+igt@kms_vblank@pipe-c-query-busy-hang
+igt@kms_vblank@pipe-c-query-forked-busy
+igt@kms_vblank@pipe-c-query-forked-busy-hang
+igt@kms_vblank@pipe-c-wait-idle
+igt@kms_vblank@pipe-c-wait-idle-hang
+igt@kms_vblank@pipe-c-wait-forked
+igt@kms_vblank@pipe-c-wait-forked-hang
+igt@kms_vblank@pipe-c-wait-busy
+igt@kms_vblank@pipe-c-wait-busy-hang
+igt@kms_vblank@pipe-c-wait-forked-busy
+igt@kms_vblank@pipe-c-wait-forked-busy-hang
+igt@kms_vblank@pipe-c-ts-continuation-idle
+igt@kms_vblank@pipe-c-ts-continuation-idle-hang
+igt@kms_vblank@pipe-c-ts-continuation-modeset
+igt@kms_vblank@pipe-c-ts-continuation-modeset-hang
diff --git a/drivers/gpu/drm/ci/i915_amly_results.txt b/drivers/gpu/drm/ci/i915_amly_results.txt
new file mode 100644
index 000000000000..2937755a9901
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_amly_results.txt
@@ -0,0 +1,9 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_sysfs_edid_timing,fail
diff --git a/drivers/gpu/drm/ci/i915_apl_results.txt b/drivers/gpu/drm/ci/i915_apl_results.txt
new file mode 100644
index 000000000000..2937755a9901
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_apl_results.txt
@@ -0,0 +1,9 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_sysfs_edid_timing,fail
diff --git a/drivers/gpu/drm/ci/i915_cml_results.txt b/drivers/gpu/drm/ci/i915_cml_results.txt
new file mode 100644
index 000000000000..e19c38f6b154
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_cml_results.txt
@@ -0,0 +1,10 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,skip
+igt@kms_setmode@basic@pipe-b-edp-1,fail
diff --git a/drivers/gpu/drm/ci/i915_glk_results.txt b/drivers/gpu/drm/ci/i915_glk_results.txt
new file mode 100644
index 000000000000..cb5b1ad47183
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_glk_results.txt
@@ -0,0 +1,13 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,skip
+igt@kms_rotation_crc@multiplane-rotation,fail
+igt@kms_rotation_crc@multiplane-rotation-cropping-top,fail
+igt@kms_rotation_crc@multiplane-rotation-cropping-bottom,fail
+igt@kms_setmode@basic@pipe-b-edp-1,fail
diff --git a/drivers/gpu/drm/ci/i915_kbl_results.txt b/drivers/gpu/drm/ci/i915_kbl_results.txt
new file mode 100644
index 000000000000..2937755a9901
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_kbl_results.txt
@@ -0,0 +1,9 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_sysfs_edid_timing,fail
diff --git a/drivers/gpu/drm/ci/i915_tgl_results.txt b/drivers/gpu/drm/ci/i915_tgl_results.txt
new file mode 100644
index 000000000000..e8b70ddd96c7
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_tgl_results.txt
@@ -0,0 +1,12 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_rotation_crc@bad-pixel-format,fail
+igt@kms_rotation_crc@multiplane-rotation,fail
+igt@kms_rotation_crc@multiplane-rotation-cropping-top,fail
+igt@kms_rotation_crc@multiplane-rotation-cropping-bottom,fail
diff --git a/drivers/gpu/drm/ci/i915_whl_results.txt b/drivers/gpu/drm/ci/i915_whl_results.txt
new file mode 100644
index 000000000000..2937755a9901
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915_whl_results.txt
@@ -0,0 +1,9 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_sysfs_edid_timing,fail
diff --git a/drivers/gpu/drm/ci/mediatek.testlist b/drivers/gpu/drm/ci/mediatek.testlist
new file mode 100644
index 000000000000..dd1ad1801b0b
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek.testlist
@@ -0,0 +1,102 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@drm_read@fault-buffer
+igt@drm_read@empty-block
+igt@drm_read@empty-nonblock
+igt@drm_read@short-buffer-block
+igt@drm_read@short-buffer-nonblock
+igt@drm_read@short-buffer-wakeup
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+
+# Flaky on mt8173
+# igt@kms_hdmi_inject@inject-4k
+
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+igt@kms_setmode@basic@pipe-a-edp-1
+igt@kms_setmode@basic@pipe-b-edp-1
+igt@kms_sysfs_edid_timing
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-modeset
diff --git a/drivers/gpu/drm/ci/mediatek_mt8173_results.txt b/drivers/gpu/drm/ci/mediatek_mt8173_results.txt
new file mode 100644
index 000000000000..74fd881fdd87
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek_mt8173_results.txt
@@ -0,0 +1,11 @@
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_rmfb@close-fd,fail
+igt@kms_setmode@basic,skip
diff --git a/drivers/gpu/drm/ci/mediatek_mt8183_results.txt b/drivers/gpu/drm/ci/mediatek_mt8183_results.txt
new file mode 100644
index 000000000000..74fd881fdd87
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek_mt8183_results.txt
@@ -0,0 +1,11 @@
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_rmfb@close-fd,fail
+igt@kms_setmode@basic,skip
diff --git a/drivers/gpu/drm/ci/mediatek_mt8192_results.txt b/drivers/gpu/drm/ci/mediatek_mt8192_results.txt
new file mode 100644
index 000000000000..74fd881fdd87
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek_mt8192_results.txt
@@ -0,0 +1,11 @@
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_rmfb@close-fd,fail
+igt@kms_setmode@basic,skip
diff --git a/drivers/gpu/drm/ci/meson.testlist b/drivers/gpu/drm/ci/meson.testlist
new file mode 100644
index 000000000000..23744b3a2fb0
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson.testlist
@@ -0,0 +1,71 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_hdmi_inject@inject-4k
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_setmode@basic@pipe-a-edp-1
+igt@kms_setmode@basic@pipe-b-edp-1
+igt@kms_sysfs_edid_timing
diff --git a/drivers/gpu/drm/ci/meson_g12b_results.txt b/drivers/gpu/drm/ci/meson_g12b_results.txt
new file mode 100644
index 000000000000..6997a75aa1ef
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson_g12b_results.txt
@@ -0,0 +1,11 @@
+igt@drm_read@invalid-buffer,fail
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_setmode@basic,skip
diff --git a/drivers/gpu/drm/ci/msm.testlist b/drivers/gpu/drm/ci/msm.testlist
new file mode 100644
index 000000000000..568051b1dc91
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm.testlist
@@ -0,0 +1,144 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@drm_read@fault-buffer
+igt@drm_read@empty-block
+igt@drm_read@empty-nonblock
+igt@drm_read@short-buffer-block
+igt@drm_read@short-buffer-nonblock
+igt@drm_read@short-buffer-wakeup
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-primary-overlay-mutable-zpos
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic@atomic_plane_damage
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_hdmi_inject@inject-4k
+igt@kms_multipipe_modeset@basic-max-pipe-crc-check
+igt@kms_pipe_crc_basic@bad-source
+igt@kms_pipe_crc_basic@read-crc-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a
+igt@kms_pipe_crc_basic@read-crc-pipe-b
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+igt@kms_setmode@basic@pipe-a-edp-1
+igt@kms_setmode@basic@pipe-b-edp-1
+igt@msm_mapping@ring
+igt@msm_mapping@sqefw
+igt@msm_mapping@shadow
+igt@msm_recovery@hangcheck
+igt@msm_recovery@gpu-fault
+igt@msm_recovery@iova-fault
+igt@msm_submit@empty-submit
+igt@msm_submit@invalid-queue-submit
+igt@msm_submit@invalid-flags-submit
+igt@msm_submit@invalid-in-fence-submit
+igt@msm_submit@invalid-duplicate-bo-submit
+igt@msm_submit@invalid-cmd-idx-submit
+igt@msm_submit@invalid-cmd-type-submit
+igt@msm_submit@valid-submit
+igt@kms_sysfs_edid_timing
+igt@kms_universal_plane@universal-plane-pipe-a-functional
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-modeset
+igt@kms_vblank@pipe-b-accuracy-idle
+igt@kms_vblank@pipe-b-query-idle
+igt@kms_vblank@pipe-b-query-forked
+igt@kms_vblank@pipe-b-query-busy
+igt@kms_vblank@pipe-b-query-forked-busy
+igt@kms_vblank@pipe-b-wait-idle
+igt@kms_vblank@pipe-b-wait-forked
+igt@kms_vblank@pipe-b-wait-busy
+igt@kms_vblank@pipe-b-wait-forked-busy
+igt@kms_vblank@pipe-b-ts-continuation-idle
+igt@kms_vblank@pipe-b-ts-continuation-modeset
diff --git a/drivers/gpu/drm/ci/msm_apq8016_results.txt b/drivers/gpu/drm/ci/msm_apq8016_results.txt
new file mode 100644
index 000000000000..b9f664b43220
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm_apq8016_results.txt
@@ -0,0 +1,77 @@
+igt@drm_read@invalid-buffer,skip
+igt@drm_read@fault-buffer,skip
+igt@drm_read@empty-block,skip
+igt@drm_read@empty-nonblock,skip
+igt@drm_read@short-buffer-block,skip
+igt@drm_read@short-buffer-nonblock,skip
+igt@drm_read@short-buffer-wakeup,skip
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic@plane-overlay-legacy,skip
+igt@kms_atomic@plane-primary-legacy,skip
+igt@kms_atomic@plane-primary-overlay-mutable-zpos,skip
+igt@kms_atomic@plane-immutable-zpos,skip
+igt@kms_atomic@test-only,skip
+igt@kms_atomic@plane-cursor-legacy,skip
+igt@kms_atomic@plane-invalid-params,skip
+igt@kms_atomic@crtc-invalid-params,skip
+igt@kms_atomic@atomic-invalid-params,skip
+igt@kms_atomic@atomic_plane_damage,skip
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_flip_event_leak,skip
+igt@kms_hdmi_inject@inject-4k,fail
+igt@kms_multipipe_modeset@basic-max-pipe-crc-check,skip
+igt@kms_pipe_crc_basic@bad-source,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-a,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence,skip
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a,skip
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b,skip
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b,skip
+igt@kms_rmfb@rmfb-ioctl,skip
+igt@kms_rmfb@close-fd,skip
+igt@kms_setmode@basic,skip
+igt@msm_mapping@ring,fail
+igt@msm_mapping@sqefw,skip
+igt@msm_mapping@shadow,skip
+igt@msm_recovery@hangcheck,skip
+igt@msm_recovery@gpu-fault,skip
+igt@msm_recovery@iova-fault,skip
+igt@kms_universal_plane@universal-plane-pipe-a-functional,skip
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a,skip
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,skip
+igt@kms_vblank@invalid,skip
+igt@kms_vblank@crtc-id,skip
+igt@kms_vblank@pipe-a-query-idle,skip
+igt@kms_vblank@pipe-a-query-forked,skip
+igt@kms_vblank@pipe-a-query-busy,skip
+igt@kms_vblank@pipe-a-query-forked-busy,skip
+igt@kms_vblank@pipe-a-wait-idle,skip
+igt@kms_vblank@pipe-a-wait-forked,skip
+igt@kms_vblank@pipe-a-wait-busy,skip
+igt@kms_vblank@pipe-a-wait-forked-busy,skip
+igt@kms_vblank@pipe-a-ts-continuation-idle,skip
+igt@kms_vblank@pipe-a-ts-continuation-modeset,skip
+igt@kms_vblank@pipe-b-accuracy-idle,skip
+igt@kms_vblank@pipe-b-query-idle,skip
+igt@kms_vblank@pipe-b-query-forked,skip
+igt@kms_vblank@pipe-b-query-busy,skip
+igt@kms_vblank@pipe-b-query-forked-busy,skip
+igt@kms_vblank@pipe-b-wait-idle,skip
+igt@kms_vblank@pipe-b-wait-forked,skip
+igt@kms_vblank@pipe-b-wait-busy,skip
+igt@kms_vblank@pipe-b-wait-forked-busy,skip
+igt@kms_vblank@pipe-b-ts-continuation-idle,skip
+igt@kms_vblank@pipe-b-ts-continuation-modeset,skip
diff --git a/drivers/gpu/drm/ci/msm_apq8096_results.txt b/drivers/gpu/drm/ci/msm_apq8096_results.txt
new file mode 100644
index 000000000000..c3896d752bb8
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm_apq8096_results.txt
@@ -0,0 +1,76 @@
+igt@drm_read@invalid-buffer,skip
+igt@drm_read@fault-buffer,skip
+igt@drm_read@empty-block,skip
+igt@drm_read@empty-nonblock,skip
+igt@drm_read@short-buffer-block,skip
+igt@drm_read@short-buffer-nonblock,skip
+igt@drm_read@short-buffer-wakeup,skip
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic@plane-overlay-legacy,skip
+igt@kms_atomic@plane-primary-legacy,skip
+igt@kms_atomic@plane-primary-overlay-mutable-zpos,skip
+igt@kms_atomic@plane-immutable-zpos,skip
+igt@kms_atomic@test-only,skip
+igt@kms_atomic@plane-cursor-legacy,skip
+igt@kms_atomic@plane-invalid-params,skip
+igt@kms_atomic@crtc-invalid-params,skip
+igt@kms_atomic@atomic-invalid-params,skip
+igt@kms_atomic@atomic_plane_damage,skip
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_flip_event_leak,skip
+igt@kms_multipipe_modeset@basic-max-pipe-crc-check,skip
+igt@kms_pipe_crc_basic@bad-source,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-a,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence,skip
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a,skip
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b,skip
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b,skip
+igt@kms_rmfb@rmfb-ioctl,skip
+igt@kms_rmfb@close-fd,skip
+igt@kms_setmode@basic,skip
+igt@msm_mapping@ring,fail
+igt@msm_mapping@sqefw,skip
+igt@msm_mapping@shadow,skip
+igt@msm_recovery@hangcheck,skip
+igt@msm_recovery@gpu-fault,skip
+igt@msm_recovery@iova-fault,skip
+igt@kms_universal_plane@universal-plane-pipe-a-functional,skip
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a,skip
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,skip
+igt@kms_vblank@invalid,skip
+igt@kms_vblank@crtc-id,skip
+igt@kms_vblank@pipe-a-query-idle,skip
+igt@kms_vblank@pipe-a-query-forked,skip
+igt@kms_vblank@pipe-a-query-busy,skip
+igt@kms_vblank@pipe-a-query-forked-busy,skip
+igt@kms_vblank@pipe-a-wait-idle,skip
+igt@kms_vblank@pipe-a-wait-forked,skip
+igt@kms_vblank@pipe-a-wait-busy,skip
+igt@kms_vblank@pipe-a-wait-forked-busy,skip
+igt@kms_vblank@pipe-a-ts-continuation-idle,skip
+igt@kms_vblank@pipe-a-ts-continuation-modeset,skip
+igt@kms_vblank@pipe-b-accuracy-idle,skip
+igt@kms_vblank@pipe-b-query-idle,skip
+igt@kms_vblank@pipe-b-query-forked,skip
+igt@kms_vblank@pipe-b-query-busy,skip
+igt@kms_vblank@pipe-b-query-forked-busy,skip
+igt@kms_vblank@pipe-b-wait-idle,skip
+igt@kms_vblank@pipe-b-wait-forked,skip
+igt@kms_vblank@pipe-b-wait-busy,skip
+igt@kms_vblank@pipe-b-wait-forked-busy,skip
+igt@kms_vblank@pipe-b-ts-continuation-idle,skip
+igt@kms_vblank@pipe-b-ts-continuation-modeset,skip
diff --git a/drivers/gpu/drm/ci/msm_sc7180_results.txt b/drivers/gpu/drm/ci/msm_sc7180_results.txt
new file mode 100644
index 000000000000..ca801d6ad2e7
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm_sc7180_results.txt
@@ -0,0 +1,18 @@
+igt@kms_atomic@plane-overlay-legacy,skip
+igt@kms_atomic@plane-primary-overlay-mutable-zpos,skip
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,skip
+igt@kms_multipipe_modeset@basic-max-pipe-crc-check,skip
+igt@kms_rmfb@close-fd,fail
+igt@msm_mapping@ring,fail
+igt@msm_mapping@sqefw,fail
+igt@msm_mapping@shadow,fail
+igt@kms_universal_plane@universal-plane-pipe-a-functional,skip
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,fail
diff --git a/drivers/gpu/drm/ci/msm_sdm845_results.txt b/drivers/gpu/drm/ci/msm_sdm845_results.txt
new file mode 100644
index 000000000000..eaf5fe125fbd
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm_sdm845_results.txt
@@ -0,0 +1,33 @@
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b,skip
+igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence,skip
+igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b,skip
+igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b,skip
+igt@kms_rmfb@close-fd,fail
+igt@kms_setmode@basic,skip
+igt@msm_mapping@ring,fail
+igt@msm_mapping@sqefw,fail
+igt@msm_mapping@shadow,fail
+igt@kms_universal_plane@universal-plane-pipe-a-functional,fail
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b,skip
+igt@kms_vblank@pipe-b-accuracy-idle,skip
+igt@kms_vblank@pipe-b-query-idle,skip
+igt@kms_vblank@pipe-b-query-forked,skip
+igt@kms_vblank@pipe-b-query-busy,skip
+igt@kms_vblank@pipe-b-query-forked-busy,skip
+igt@kms_vblank@pipe-b-wait-idle,skip
+igt@kms_vblank@pipe-b-wait-forked,skip
+igt@kms_vblank@pipe-b-wait-busy,skip
+igt@kms_vblank@pipe-b-wait-forked-busy,skip
+igt@kms_vblank@pipe-b-ts-continuation-idle,skip
+igt@kms_vblank@pipe-b-ts-continuation-modeset,skip
diff --git a/drivers/gpu/drm/ci/rockchip.testlist b/drivers/gpu/drm/ci/rockchip.testlist
new file mode 100644
index 000000000000..c261e0677200
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip.testlist
@@ -0,0 +1,188 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@drm_read@invalid-buffer
+igt@drm_read@fault-buffer
+igt@drm_read@empty-block
+igt@drm_read@empty-nonblock
+igt@drm_read@short-buffer-block
+igt@drm_read@short-buffer-nonblock
+igt@drm_read@short-buffer-wakeup
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-overlay-legacy
+
+# Occasional dmesg-warn on rk3288
+# rockchip-dp ff970000.dp: AUX CH cmd reply timeout!
+# igt@kms_atomic@plane-primary-legacy
+
+
+# Machine hangs on rk3399
+# igt@kms_atomic@plane-immutable-zpos
+
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_hdmi_inject@inject-4k
+igt@kms_pipe_crc_basic@bad-source
+
+# Machine hangs on rk3399
+# igt@kms_pipe_crc_basic@read-crc-pipe-a
+# igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-a-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-a
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-a
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a
+# igt@kms_pipe_crc_basic@read-crc-pipe-b
+# igt@kms_pipe_crc_basic@read-crc-pipe-b-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-b-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-b
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-b
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b
+# igt@kms_pipe_crc_basic@read-crc-pipe-c
+# igt@kms_pipe_crc_basic@read-crc-pipe-c-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-c
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-c-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-c
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-c
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-c
+# igt@kms_pipe_crc_basic@read-crc-pipe-d
+# igt@kms_pipe_crc_basic@read-crc-pipe-d-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-d-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-d
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-d
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d
+# igt@kms_pipe_crc_basic@read-crc-pipe-e
+# igt@kms_pipe_crc_basic@read-crc-pipe-e-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-e
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-e-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-e
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-e
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-e
+# igt@kms_pipe_crc_basic@read-crc-pipe-f
+# igt@kms_pipe_crc_basic@read-crc-pipe-f-frame-sequence
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-f
+# igt@kms_pipe_crc_basic@nonblocking-crc-pipe-f-frame-sequence
+# igt@kms_pipe_crc_basic@disable-crc-after-crtc-pipe-f
+# igt@kms_pipe_crc_basic@hang-read-crc-pipe-f
+# igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-f
+
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+
+# Machine hangs on rk3399
+# igt@kms_rotation_crc@primary-x-tiled-reflect-x-0
+# igt@kms_rotation_crc@primary-y-tiled-reflect-x-0
+# igt@kms_rotation_crc@primary-yf-tiled-reflect-x-0
+# igt@kms_rotation_crc@primary-4-tiled-reflect-x-0
+# igt@kms_rotation_crc@primary-4-tiled-reflect-x-180
+
+
+# Flaky on rk3399
+# igt@kms_setmode@basic@pipe-a-edp-1
+# igt@kms_setmode@basic@pipe-b-edp-1
+
+igt@kms_sysfs_edid_timing
+igt@kms_universal_plane@universal-plane-pipe-a-functional
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-a
+
+# Machine hangs on rk3399 after a load of the following:
+# rockchip-dp ff970000.edp: CR Max reached (5,0,0)
+# rockchip-dp ff970000.edp: LT CR failed!
+# rockchip-dp ff970000.edp: eDP link training failed (-5)
+# rockchip-dp ff970000.edp: unable to do link train, ret=-5
+# [drm:analogix_dp_bridge_atomic_enable] *ERROR* dp commit error, ret = -5
+#
+# igt@kms_universal_plane@universal-plane-pipe-b-functional
+
+igt@kms_universal_plane@disable-primary-vs-flip-pipe-b
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-modeset
+igt@kms_vblank@pipe-b-accuracy-idle
+igt@kms_vblank@pipe-b-query-idle
+igt@kms_vblank@pipe-b-query-forked
+igt@kms_vblank@pipe-b-query-busy
+igt@kms_vblank@pipe-b-query-forked-busy
+igt@kms_vblank@pipe-b-wait-idle
+igt@kms_vblank@pipe-b-wait-forked
+igt@kms_vblank@pipe-b-wait-busy
+igt@kms_vblank@pipe-b-wait-forked-busy
+igt@kms_vblank@pipe-b-ts-continuation-idle
+igt@kms_vblank@pipe-b-ts-continuation-modeset
diff --git a/drivers/gpu/drm/ci/rockchip_rk3288_results.txt b/drivers/gpu/drm/ci/rockchip_rk3288_results.txt
new file mode 100644
index 000000000000..c6bec0324414
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip_rk3288_results.txt
@@ -0,0 +1,11 @@
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,fail
+igt@kms_rmfb@close-fd,fail
diff --git a/drivers/gpu/drm/ci/rockchip_rk3399_results.txt b/drivers/gpu/drm/ci/rockchip_rk3399_results.txt
new file mode 100644
index 000000000000..c6d9208c2754
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip_rk3399_results.txt
@@ -0,0 +1,14 @@
+igt@drm_read@short-buffer-wakeup,fail
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic@plane-overlay-legacy,skip
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_hdmi_inject@inject-4k,skip
+igt@kms_rmfb@close-fd,fail
+igt@kms_universal_plane@universal-plane-pipe-a-functional,skip
diff --git a/drivers/gpu/drm/ci/virtio_gpu.testlist b/drivers/gpu/drm/ci/virtio_gpu.testlist
new file mode 100644
index 000000000000..d0df58d23dae
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu.testlist
@@ -0,0 +1,92 @@
+# Keep alphabetically sorted by default
+
+igt@core_auth@getclient-simple
+igt@core_auth@getclient-master-drop
+igt@core_auth@basic-auth
+igt@core_auth@many-magics
+igt@core_getclient
+igt@core_getstats
+igt@core_getversion
+igt@core_setmaster_vs_auth
+igt@kms_addfb_basic@unused-handle
+igt@kms_addfb_basic@unused-pitches
+igt@kms_addfb_basic@unused-offsets
+igt@kms_addfb_basic@unused-modifier
+igt@kms_addfb_basic@legacy-format
+igt@kms_addfb_basic@no-handle
+igt@kms_addfb_basic@basic
+igt@kms_addfb_basic@bad-pitch-0
+igt@kms_addfb_basic@bad-pitch-32
+igt@kms_addfb_basic@bad-pitch-63
+igt@kms_addfb_basic@bad-pitch-128
+igt@kms_addfb_basic@bad-pitch-256
+igt@kms_addfb_basic@bad-pitch-1024
+igt@kms_addfb_basic@bad-pitch-999
+igt@kms_addfb_basic@bad-pitch-65536
+igt@kms_addfb_basic@size-max
+igt@kms_addfb_basic@too-wide
+igt@kms_addfb_basic@too-high
+igt@kms_addfb_basic@bo-too-small
+igt@kms_addfb_basic@small-bo
+igt@kms_addfb_basic@addfb25-modifier-no-flag
+igt@kms_addfb_basic@addfb25-bad-modifier
+igt@kms_addfb_basic@invalid-get-prop-any
+igt@kms_addfb_basic@invalid-get-prop
+igt@kms_addfb_basic@invalid-set-prop-any
+igt@kms_addfb_basic@invalid-set-prop
+igt@kms_addfb_basic@master-rmfb
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@atomic-invalid-params
+igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
+igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
+igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
+igt@kms_content_protection@LIC
+igt@kms_flip_event_leak
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_rmfb@rmfb-ioctl
+igt@kms_rmfb@close-fd
+igt@kms_setmode@basic@pipe-a-edp-1
+igt@kms_setmode@basic@pipe-b-edp-1
+igt@kms_sysfs_edid_timing
+igt@kms_vblank@invalid
+igt@kms_vblank@crtc-id
+igt@kms_vblank@pipe-a-query-idle
+igt@kms_vblank@pipe-a-query-forked
+igt@kms_vblank@pipe-a-query-busy
+igt@kms_vblank@pipe-a-query-forked-busy
+igt@kms_vblank@pipe-a-wait-idle
+igt@kms_vblank@pipe-a-wait-forked
+igt@kms_vblank@pipe-a-wait-busy
+igt@kms_vblank@pipe-a-wait-forked-busy
+igt@kms_vblank@pipe-a-ts-continuation-idle
+igt@kms_vblank@pipe-a-ts-continuation-dpms-rpm
+igt@kms_vblank@pipe-a-ts-continuation-modeset
+igt@kms_vblank@pipe-a-ts-continuation-modeset-rpm
diff --git a/drivers/gpu/drm/ci/virtio_gpu_results.txt b/drivers/gpu/drm/ci/virtio_gpu_results.txt
new file mode 100644
index 000000000000..1453f49880dd
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu_results.txt
@@ -0,0 +1,28 @@
+igt@kms_addfb_basic@bad-pitch-65536,fail
+igt@kms_addfb_basic@size-max,fail
+igt@kms_addfb_basic@too-high,fail
+igt@kms_addfb_basic@bo-too-small,fail
+igt@kms_addfb_basic@addfb25-bad-modifier,fail
+igt@kms_atomic_interruptible@legacy-setmode,skip
+igt@kms_atomic_interruptible@atomic-setmode,skip
+igt@kms_atomic_interruptible@legacy-dpms,skip
+igt@kms_atomic_interruptible@legacy-pageflip,skip
+igt@kms_atomic_interruptible@legacy-cursor,skip
+igt@kms_atomic_interruptible@universal-setplane-primary,skip
+igt@kms_atomic_interruptible@universal-setplane-cursor,skip
+igt@kms_content_protection@lic,skip
+igt@kms_setmode@basic,skip
+igt@kms_vblank@invalid,fail
+igt@kms_vblank@crtc-id,fail
+igt@kms_vblank@pipe-a-query-idle,fail
+igt@kms_vblank@pipe-a-query-forked,fail
+igt@kms_vblank@pipe-a-query-busy,fail
+igt@kms_vblank@pipe-a-query-forked-busy,fail
+igt@kms_vblank@pipe-a-wait-idle,fail
+igt@kms_vblank@pipe-a-wait-forked,fail
+igt@kms_vblank@pipe-a-wait-busy,fail
+igt@kms_vblank@pipe-a-wait-forked-busy,fail
+igt@kms_vblank@pipe-a-ts-continuation-idle,fail
+igt@kms_vblank@pipe-a-ts-continuation-dpms-rpm,fail
+igt@kms_vblank@pipe-a-ts-continuation-modeset,fail
+igt@kms_vblank@pipe-a-ts-continuation-modeset-rpm,fail
-- 
2.31.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [RFC v4] drm: Add initial ci/ subdirectory
  2022-05-17  8:16 ` [RFC v4] drm: Add initial ci/ subdirectory Tomeu Vizoso
@ 2022-05-17  9:18   ` Neil Armstrong
  2022-05-17  9:24     ` Tomeu Vizoso
  2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
  1 sibling, 1 reply; 13+ messages in thread
From: Neil Armstrong @ 2022-05-17  9:18 UTC (permalink / raw)
  To: Tomeu Vizoso, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger
  Cc: linux-doc, linux-kernel, dri-devel, linux-rockchip,
	linux-mediatek, linux-amlogic, linux-arm-kernel

On 17/05/2022 10:16, Tomeu Vizoso wrote:
> And use it to store expectations about what the DRM drivers are
> supposed to pass in the IGT test suite.
> 
> Also include a configuration file that points to the out-of-tree CI
> scripts.
> 
> By storing the test expectations along the code we can make sure both
> stay in sync with each other, and so we can know when a code change
> breaks those expectations.
> 
> This will allow all contributors to drm to reuse the infrastructure
> already in gitlab.freedesktop.org to test the driver on several
> generations of the hardware.
> 
> v2:
>    - Fix names of result expectation files to match SoC
>    - Don't execute tests that are going to skip on all boards
> 
> v3:
>    - Remove tracking of dmesg output during test execution
> 
> v4:
>    - Move up to drivers/gpu/drm
>    - Add support for a bunch of other drivers
>    - Explain how to incorporate fixes for CI from a
>      ${TARGET_BRANCH}-external-fixes branch
>    - Remove tests that pass from expected results file, to reduce the
>      size of in-tree files
>    - Add docs about how to deal with outages in automated testing labs
>    - Specify the exact SHA of the CI scripts to be used
> 
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> ---
>   Documentation/gpu/automated_testing.rst       |  84 +++++++
>   drivers/gpu/drm/ci/amdgpu.testlist            | 130 +++++++++++
>   drivers/gpu/drm/ci/amdgpu_stoney_results.txt  |  17 ++
>   drivers/gpu/drm/ci/gitlab-ci.yml              |  13 ++
>   drivers/gpu/drm/ci/i915.testlist              | 219 ++++++++++++++++++
>   drivers/gpu/drm/ci/i915_amly_results.txt      |   9 +
>   drivers/gpu/drm/ci/i915_apl_results.txt       |   9 +
>   drivers/gpu/drm/ci/i915_cml_results.txt       |  10 +
>   drivers/gpu/drm/ci/i915_glk_results.txt       |  13 ++
>   drivers/gpu/drm/ci/i915_kbl_results.txt       |   9 +
>   drivers/gpu/drm/ci/i915_tgl_results.txt       |  12 +
>   drivers/gpu/drm/ci/i915_whl_results.txt       |   9 +
>   drivers/gpu/drm/ci/mediatek.testlist          | 102 ++++++++
>   .../gpu/drm/ci/mediatek_mt8173_results.txt    |  11 +
>   .../gpu/drm/ci/mediatek_mt8183_results.txt    |  11 +
>   .../gpu/drm/ci/mediatek_mt8192_results.txt    |  11 +
>   drivers/gpu/drm/ci/meson.testlist             |  71 ++++++
>   drivers/gpu/drm/ci/meson_g12b_results.txt     |  11 +
>   drivers/gpu/drm/ci/msm.testlist               | 144 ++++++++++++
>   drivers/gpu/drm/ci/msm_apq8016_results.txt    |  77 ++++++
>   drivers/gpu/drm/ci/msm_apq8096_results.txt    |  76 ++++++
>   drivers/gpu/drm/ci/msm_sc7180_results.txt     |  18 ++
>   drivers/gpu/drm/ci/msm_sdm845_results.txt     |  33 +++
>   drivers/gpu/drm/ci/rockchip.testlist          | 188 +++++++++++++++
>   .../gpu/drm/ci/rockchip_rk3288_results.txt    |  11 +
>   .../gpu/drm/ci/rockchip_rk3399_results.txt    |  14 ++
>   drivers/gpu/drm/ci/virtio_gpu.testlist        |  92 ++++++++
>   drivers/gpu/drm/ci/virtio_gpu_results.txt     |  28 +++
>   28 files changed, 1432 insertions(+)
>   create mode 100644 Documentation/gpu/automated_testing.rst
>   create mode 100644 drivers/gpu/drm/ci/amdgpu.testlist
>   create mode 100644 drivers/gpu/drm/ci/amdgpu_stoney_results.txt
>   create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>   create mode 100644 drivers/gpu/drm/ci/i915.testlist
>   create mode 100644 drivers/gpu/drm/ci/i915_amly_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_apl_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_cml_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_glk_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_kbl_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_tgl_results.txt
>   create mode 100644 drivers/gpu/drm/ci/i915_whl_results.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek.testlist
>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8173_results.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8183_results.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8192_results.txt
>   create mode 100644 drivers/gpu/drm/ci/meson.testlist
>   create mode 100644 drivers/gpu/drm/ci/meson_g12b_results.txt
>   create mode 100644 drivers/gpu/drm/ci/msm.testlist
>   create mode 100644 drivers/gpu/drm/ci/msm_apq8016_results.txt
>   create mode 100644 drivers/gpu/drm/ci/msm_apq8096_results.txt
>   create mode 100644 drivers/gpu/drm/ci/msm_sc7180_results.txt
>   create mode 100644 drivers/gpu/drm/ci/msm_sdm845_results.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip.testlist
>   create mode 100644 drivers/gpu/drm/ci/rockchip_rk3288_results.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip_rk3399_results.txt
>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu.testlist
>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu_results.txt
> 
> diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
> new file mode 100644
> index 000000000000..61003ecc7b6e
> --- /dev/null
> +++ b/Documentation/gpu/automated_testing.rst
> @@ -0,0 +1,84 @@
> +.. SPDX-License-Identifier: GPL-2.0+
> +
> +=========================================
> +Automated testing of the DRM subsystem
> +=========================================
> +
> +
> +Introduction
> +============
> +
> +Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
> +
> +Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
> +
> +There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
> +
> +
> +Relevant files
> +==============
> +
> +drivers/gpu/drm/ci/gitlab-ci.yml
> +--------------------------------
> +
> +Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
> +------------------------------------------
> +
> +Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.
> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
> +----------------------------------------
> +
> +Specifies the expected results of running this specific kernel revision on a given hardware configuration.
> +
> +
> +How to enable automated testing on your tree
> +============================================
> +
> +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
> +
> +2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
> +
> +3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
> +
> +4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
> +
> +
> +How to update test expectations
> +===============================
> +
> +If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
> +
> +If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
> +
> +
> +How to expand coverage
> +======================
> +
> +If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
> +
> +If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
> +
> +
> +How to test your changes to the scripts
> +==========================================
> +
> +For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
> +
> +
> +How to incorporate external fixes in your testing
> +=================================================
> +
> +Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
> +
> +If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
> +
> +
> +How to deal with automated testing labs that may be down
> +========================================================
> +
> +If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.


[..]

> diff --git a/drivers/gpu/drm/ci/meson.testlist b/drivers/gpu/drm/ci/meson.testlist
> new file mode 100644
> index 000000000000..23744b3a2fb0
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson.testlist
> @@ -0,0 +1,71 @@
> +# Keep alphabetically sorted by default
> +
> +igt@core_auth@getclient-simple
> +igt@core_auth@getclient-master-drop
> +igt@core_auth@basic-auth
> +igt@core_auth@many-magics
> +igt@core_getclient
> +igt@core_getstats
> +igt@core_getversion
> +igt@core_setmaster_vs_auth
> +igt@drm_read@invalid-buffer
> +igt@kms_addfb_basic@unused-handle
> +igt@kms_addfb_basic@unused-pitches
> +igt@kms_addfb_basic@unused-offsets
> +igt@kms_addfb_basic@unused-modifier
> +igt@kms_addfb_basic@legacy-format
> +igt@kms_addfb_basic@no-handle
> +igt@kms_addfb_basic@basic
> +igt@kms_addfb_basic@bad-pitch-0
> +igt@kms_addfb_basic@bad-pitch-32
> +igt@kms_addfb_basic@bad-pitch-63
> +igt@kms_addfb_basic@bad-pitch-128
> +igt@kms_addfb_basic@bad-pitch-256
> +igt@kms_addfb_basic@bad-pitch-1024
> +igt@kms_addfb_basic@bad-pitch-999
> +igt@kms_addfb_basic@bad-pitch-65536
> +igt@kms_addfb_basic@size-max
> +igt@kms_addfb_basic@too-wide
> +igt@kms_addfb_basic@too-high
> +igt@kms_addfb_basic@bo-too-small
> +igt@kms_addfb_basic@small-bo
> +igt@kms_addfb_basic@addfb25-modifier-no-flag
> +igt@kms_addfb_basic@addfb25-bad-modifier
> +igt@kms_addfb_basic@invalid-get-prop-any
> +igt@kms_addfb_basic@invalid-get-prop
> +igt@kms_addfb_basic@invalid-set-prop-any
> +igt@kms_addfb_basic@invalid-set-prop
> +igt@kms_addfb_basic@master-rmfb
> +igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
> +igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
> +igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
> +igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
> +igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
> +igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
> +igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
> +igt@kms_content_protection@LIC
> +igt@kms_getfb@getfb-handle-zero
> +igt@kms_getfb@getfb-handle-valid
> +igt@kms_getfb@getfb-handle-closed
> +igt@kms_getfb@getfb-handle-not-fb
> +igt@kms_getfb@getfb-addfb-different-handles
> +igt@kms_getfb@getfb-repeated-different-handles
> +igt@kms_getfb@getfb2-handle-zero
> +igt@kms_getfb@getfb2-handle-closed
> +igt@kms_getfb@getfb2-handle-not-fb
> +igt@kms_getfb@getfb2-into-addfb2
> +igt@kms_getfb@getfb-handle-protection
> +igt@kms_getfb@getfb2-handle-protection
> +igt@kms_hdmi_inject@inject-4k
> +igt@kms_prop_blob@basic
> +igt@kms_prop_blob@blob-prop-core
> +igt@kms_prop_blob@blob-prop-validate
> +igt@kms_prop_blob@blob-prop-lifetime
> +igt@kms_prop_blob@blob-multiple
> +igt@kms_prop_blob@invalid-get-prop-any
> +igt@kms_prop_blob@invalid-get-prop
> +igt@kms_prop_blob@invalid-set-prop-any
> +igt@kms_prop_blob@invalid-set-prop
> +igt@kms_setmode@basic@pipe-a-edp-1
> +igt@kms_setmode@basic@pipe-b-edp-1
> +igt@kms_sysfs_edid_timing
> diff --git a/drivers/gpu/drm/ci/meson_g12b_results.txt b/drivers/gpu/drm/ci/meson_g12b_results.txt
> new file mode 100644
> index 000000000000..6997a75aa1ef
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson_g12b_results.txt
> @@ -0,0 +1,11 @@
> +igt@drm_read@invalid-buffer,fail
> +igt@kms_addfb_basic@addfb25-bad-modifier,fail
> +igt@kms_atomic_interruptible@legacy-setmode,skip
> +igt@kms_atomic_interruptible@atomic-setmode,skip
> +igt@kms_atomic_interruptible@legacy-dpms,skip
> +igt@kms_atomic_interruptible@legacy-pageflip,skip
> +igt@kms_atomic_interruptible@legacy-cursor,skip
> +igt@kms_atomic_interruptible@universal-setplane-primary,skip
> +igt@kms_atomic_interruptible@universal-setplane-cursor,skip
> +igt@kms_content_protection@lic,skip
> +igt@kms_setmode@basic,skip

Can't those be removed from testlist instead ?
I suppose those should not skip, right ?

Anyway, this is great, please take my
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
for stuff related to the meson driver.

Neil

[..]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [RFC v4] drm: Add initial ci/ subdirectory
  2022-05-17  9:18   ` Neil Armstrong
@ 2022-05-17  9:24     ` Tomeu Vizoso
  0 siblings, 0 replies; 13+ messages in thread
From: Tomeu Vizoso @ 2022-05-17  9:24 UTC (permalink / raw)
  To: Neil Armstrong, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger
  Cc: linux-doc, linux-kernel, dri-devel, linux-rockchip,
	linux-mediatek, linux-amlogic, linux-arm-kernel

On 5/17/22 11:18 AM, Neil Armstrong wrote:
> On 17/05/2022 10:16, Tomeu Vizoso wrote:
>> And use it to store expectations about what the DRM drivers are
>> supposed to pass in the IGT test suite.
>>
>> Also include a configuration file that points to the out-of-tree CI
>> scripts.
>>
>> By storing the test expectations along the code we can make sure both
>> stay in sync with each other, and so we can know when a code change
>> breaks those expectations.
>>
>> This will allow all contributors to drm to reuse the infrastructure
>> already in gitlab.freedesktop.org to test the driver on several
>> generations of the hardware.
>>
>> v2:
>>    - Fix names of result expectation files to match SoC
>>    - Don't execute tests that are going to skip on all boards
>>
>> v3:
>>    - Remove tracking of dmesg output during test execution
>>
>> v4:
>>    - Move up to drivers/gpu/drm
>>    - Add support for a bunch of other drivers
>>    - Explain how to incorporate fixes for CI from a
>>      ${TARGET_BRANCH}-external-fixes branch
>>    - Remove tests that pass from expected results file, to reduce the
>>      size of in-tree files
>>    - Add docs about how to deal with outages in automated testing labs
>>    - Specify the exact SHA of the CI scripts to be used
>>
>> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>> ---
>>   Documentation/gpu/automated_testing.rst       |  84 +++++++
>>   drivers/gpu/drm/ci/amdgpu.testlist            | 130 +++++++++++
>>   drivers/gpu/drm/ci/amdgpu_stoney_results.txt  |  17 ++
>>   drivers/gpu/drm/ci/gitlab-ci.yml              |  13 ++
>>   drivers/gpu/drm/ci/i915.testlist              | 219 ++++++++++++++++++
>>   drivers/gpu/drm/ci/i915_amly_results.txt      |   9 +
>>   drivers/gpu/drm/ci/i915_apl_results.txt       |   9 +
>>   drivers/gpu/drm/ci/i915_cml_results.txt       |  10 +
>>   drivers/gpu/drm/ci/i915_glk_results.txt       |  13 ++
>>   drivers/gpu/drm/ci/i915_kbl_results.txt       |   9 +
>>   drivers/gpu/drm/ci/i915_tgl_results.txt       |  12 +
>>   drivers/gpu/drm/ci/i915_whl_results.txt       |   9 +
>>   drivers/gpu/drm/ci/mediatek.testlist          | 102 ++++++++
>>   .../gpu/drm/ci/mediatek_mt8173_results.txt    |  11 +
>>   .../gpu/drm/ci/mediatek_mt8183_results.txt    |  11 +
>>   .../gpu/drm/ci/mediatek_mt8192_results.txt    |  11 +
>>   drivers/gpu/drm/ci/meson.testlist             |  71 ++++++
>>   drivers/gpu/drm/ci/meson_g12b_results.txt     |  11 +
>>   drivers/gpu/drm/ci/msm.testlist               | 144 ++++++++++++
>>   drivers/gpu/drm/ci/msm_apq8016_results.txt    |  77 ++++++
>>   drivers/gpu/drm/ci/msm_apq8096_results.txt    |  76 ++++++
>>   drivers/gpu/drm/ci/msm_sc7180_results.txt     |  18 ++
>>   drivers/gpu/drm/ci/msm_sdm845_results.txt     |  33 +++
>>   drivers/gpu/drm/ci/rockchip.testlist          | 188 +++++++++++++++
>>   .../gpu/drm/ci/rockchip_rk3288_results.txt    |  11 +
>>   .../gpu/drm/ci/rockchip_rk3399_results.txt    |  14 ++
>>   drivers/gpu/drm/ci/virtio_gpu.testlist        |  92 ++++++++
>>   drivers/gpu/drm/ci/virtio_gpu_results.txt     |  28 +++
>>   28 files changed, 1432 insertions(+)
>>   create mode 100644 Documentation/gpu/automated_testing.rst
>>   create mode 100644 drivers/gpu/drm/ci/amdgpu.testlist
>>   create mode 100644 drivers/gpu/drm/ci/amdgpu_stoney_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>>   create mode 100644 drivers/gpu/drm/ci/i915.testlist
>>   create mode 100644 drivers/gpu/drm/ci/i915_amly_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_apl_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_cml_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_glk_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_kbl_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_tgl_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915_whl_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek.testlist
>>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8173_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8183_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek_mt8192_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/meson.testlist
>>   create mode 100644 drivers/gpu/drm/ci/meson_g12b_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm.testlist
>>   create mode 100644 drivers/gpu/drm/ci/msm_apq8016_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm_apq8096_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm_sc7180_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm_sdm845_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip.testlist
>>   create mode 100644 drivers/gpu/drm/ci/rockchip_rk3288_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip_rk3399_results.txt
>>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu.testlist
>>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu_results.txt
>>
>> diff --git a/Documentation/gpu/automated_testing.rst 
>> b/Documentation/gpu/automated_testing.rst
>> new file mode 100644
>> index 000000000000..61003ecc7b6e
>> --- /dev/null
>> +++ b/Documentation/gpu/automated_testing.rst
>> @@ -0,0 +1,84 @@
>> +.. SPDX-License-Identifier: GPL-2.0+
>> +
>> +=========================================
>> +Automated testing of the DRM subsystem
>> +=========================================
>> +
>> +
>> +Introduction
>> +============
>> +
>> +Making sure that changes to the core or drivers don't introduce 
>> regressions can be very time consuming when lots of different hardware 
>> configurations need to be tested. Moreover, it isn't practical for 
>> each person interested in this testing to have to acquire and maintain 
>> what can be a considerable amount of hardware.
>> +
>> +Also, it is desirable for developers to check for regressions in 
>> their code by themselves, instead of relying on the maintainers 
>> finding them and then reporting back.
>> +
>> +There are facilities in gitlab.freedesktop.org to automatically test 
>> Mesa that can be used as well for testing the DRM subsystem. This 
>> document explains how people interested in testing it can use this 
>> shared infrastructure to save quite some time and effort.
>> +
>> +
>> +Relevant files
>> +==============
>> +
>> +drivers/gpu/drm/ci/gitlab-ci.yml
>> +--------------------------------
>> +
>> +Specifies the specific version of the scripts to be used. GitLab CI 
>> will use the values defined in this file to fetch the right scripts.
>> +
>> +
>> +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
>> +------------------------------------------
>> +
>> +Specifies the tests that the current code is expected to be able to 
>> reliably run for ${DRIVER_NAME}. These tests are expected to not hang 
>> the DUT (Device Under Testing) when running on the revision they 
>> belong to, and to give consistent results.
>> +
>> +
>> +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
>> +----------------------------------------
>> +
>> +Specifies the expected results of running this specific kernel 
>> revision on a given hardware configuration.
>> +
>> +
>> +How to enable automated testing on your tree
>> +============================================
>> +
>> +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you 
>> don't have one yet
>> +
>> +2. In your kernel repo's configuration (eg. 
>> https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change 
>> the CI/CD configuration file from .gitlab-ci.yml to 
>> drivers/gpu/drm/ci/gitlab-ci.yml.
>> +
>> +3. Next time you push to this repository, you will see a CI pipeline 
>> being created (eg. 
>> https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
>> +
>> +4. The various jobs will be run and when the pipeline is finished, 
>> all jobs should be green unless a regression has been found.
>> +
>> +
>> +How to update test expectations
>> +===============================
>> +
>> +If your changes to the code fix any tests, you will have to update 
>> one or more of the files in 
>> drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test 
>> platforms affected by the change.
>> +
>> +If you have run a pipeline and it failed because of a mismatched test 
>> result, you will have been offered a patch at the end of the run that 
>> you can apply after reviewing it.
>> +
>> +
>> +How to expand coverage
>> +======================
>> +
>> +If your code changes makes it possible to run more tests (by solving 
>> reliability issues, for example), more tests can be added to this 
>> list, and then the expected results updated with the patch file that 
>> will be printed at the end of each job.
>> +
>> +If there is a need for updating the version of IGT being used (maybe 
>> you have added more tests to it), follow the instructions at 
>> https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
>> +
>> +
>> +How to test your changes to the scripts
>> +==========================================
>> +
>> +For testing changes to the scripts in the drm-ci repo, change the 
>> DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to 
>> match your fork of the project (eg. tomeu/drm-ci). This fork needs to 
>> be in https://gitlab.freedesktop.org/.
>> +
>> +
>> +How to incorporate external fixes in your testing
>> +=================================================
>> +
>> +Often, regressions in other trees will prevent testing changes local 
>> to the tree under test. These fixes will be automatically merged in 
>> during the build jobs from a branch in the target tree that is named 
>> as ${TARGET_BRANCH}-external-fixes.
>> +
>> +If the pipeline is not in a merge request and a branch with the same 
>> name exists in the local tree, commits from that branch will be merged 
>> in as well.
>> +
>> +
>> +How to deal with automated testing labs that may be down
>> +========================================================
>> +
>> +If a hardware farm is down and thus causing pipelines to fail that 
>> would otherwise pass, one can disable all jobs that would be submitted 
>> to that farm by editing the file at 
>> https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml. 
>>
> 
> 
> [..]
> 
>> diff --git a/drivers/gpu/drm/ci/meson.testlist 
>> b/drivers/gpu/drm/ci/meson.testlist
>> new file mode 100644
>> index 000000000000..23744b3a2fb0
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/meson.testlist
>> @@ -0,0 +1,71 @@
>> +# Keep alphabetically sorted by default
>> +
>> +igt@core_auth@getclient-simple
>> +igt@core_auth@getclient-master-drop
>> +igt@core_auth@basic-auth
>> +igt@core_auth@many-magics
>> +igt@core_getclient
>> +igt@core_getstats
>> +igt@core_getversion
>> +igt@core_setmaster_vs_auth
>> +igt@drm_read@invalid-buffer
>> +igt@kms_addfb_basic@unused-handle
>> +igt@kms_addfb_basic@unused-pitches
>> +igt@kms_addfb_basic@unused-offsets
>> +igt@kms_addfb_basic@unused-modifier
>> +igt@kms_addfb_basic@legacy-format
>> +igt@kms_addfb_basic@no-handle
>> +igt@kms_addfb_basic@basic
>> +igt@kms_addfb_basic@bad-pitch-0
>> +igt@kms_addfb_basic@bad-pitch-32
>> +igt@kms_addfb_basic@bad-pitch-63
>> +igt@kms_addfb_basic@bad-pitch-128
>> +igt@kms_addfb_basic@bad-pitch-256
>> +igt@kms_addfb_basic@bad-pitch-1024
>> +igt@kms_addfb_basic@bad-pitch-999
>> +igt@kms_addfb_basic@bad-pitch-65536
>> +igt@kms_addfb_basic@size-max
>> +igt@kms_addfb_basic@too-wide
>> +igt@kms_addfb_basic@too-high
>> +igt@kms_addfb_basic@bo-too-small
>> +igt@kms_addfb_basic@small-bo
>> +igt@kms_addfb_basic@addfb25-modifier-no-flag
>> +igt@kms_addfb_basic@addfb25-bad-modifier
>> +igt@kms_addfb_basic@invalid-get-prop-any
>> +igt@kms_addfb_basic@invalid-get-prop
>> +igt@kms_addfb_basic@invalid-set-prop-any
>> +igt@kms_addfb_basic@invalid-set-prop
>> +igt@kms_addfb_basic@master-rmfb
>> +igt@kms_atomic_interruptible@legacy-setmode@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@atomic-setmode@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-dpms@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-pageflip@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@legacy-cursor@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@universal-setplane-primary@pipe-a-edp-1
>> +igt@kms_atomic_interruptible@universal-setplane-cursor@pipe-a-edp-1
>> +igt@kms_content_protection@LIC
>> +igt@kms_getfb@getfb-handle-zero
>> +igt@kms_getfb@getfb-handle-valid
>> +igt@kms_getfb@getfb-handle-closed
>> +igt@kms_getfb@getfb-handle-not-fb
>> +igt@kms_getfb@getfb-addfb-different-handles
>> +igt@kms_getfb@getfb-repeated-different-handles
>> +igt@kms_getfb@getfb2-handle-zero
>> +igt@kms_getfb@getfb2-handle-closed
>> +igt@kms_getfb@getfb2-handle-not-fb
>> +igt@kms_getfb@getfb2-into-addfb2
>> +igt@kms_getfb@getfb-handle-protection
>> +igt@kms_getfb@getfb2-handle-protection
>> +igt@kms_hdmi_inject@inject-4k
>> +igt@kms_prop_blob@basic
>> +igt@kms_prop_blob@blob-prop-core
>> +igt@kms_prop_blob@blob-prop-validate
>> +igt@kms_prop_blob@blob-prop-lifetime
>> +igt@kms_prop_blob@blob-multiple
>> +igt@kms_prop_blob@invalid-get-prop-any
>> +igt@kms_prop_blob@invalid-get-prop
>> +igt@kms_prop_blob@invalid-set-prop-any
>> +igt@kms_prop_blob@invalid-set-prop
>> +igt@kms_setmode@basic@pipe-a-edp-1
>> +igt@kms_setmode@basic@pipe-b-edp-1
>> +igt@kms_sysfs_edid_timing
>> diff --git a/drivers/gpu/drm/ci/meson_g12b_results.txt 
>> b/drivers/gpu/drm/ci/meson_g12b_results.txt
>> new file mode 100644
>> index 000000000000..6997a75aa1ef
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/meson_g12b_results.txt
>> @@ -0,0 +1,11 @@
>> +igt@drm_read@invalid-buffer,fail
>> +igt@kms_addfb_basic@addfb25-bad-modifier,fail
>> +igt@kms_atomic_interruptible@legacy-setmode,skip
>> +igt@kms_atomic_interruptible@atomic-setmode,skip
>> +igt@kms_atomic_interruptible@legacy-dpms,skip
>> +igt@kms_atomic_interruptible@legacy-pageflip,skip
>> +igt@kms_atomic_interruptible@legacy-cursor,skip
>> +igt@kms_atomic_interruptible@universal-setplane-primary,skip
>> +igt@kms_atomic_interruptible@universal-setplane-cursor,skip
>> +igt@kms_content_protection@lic,skip
>> +igt@kms_setmode@basic,skip
> 
> Can't those be removed from testlist instead ?
> I suppose those should not skip, right ?

You are right, these slipped in. The only skips in the results should be 
for tests that do run on other boards for the same driver.

> Anyway, this is great, please take my
> Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
> for stuff related to the meson driver.

Thanks,

Tomeu

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH v5] drm: Add initial ci/ subdirectory
  2022-05-17  8:16 ` [RFC v4] drm: Add initial ci/ subdirectory Tomeu Vizoso
  2022-05-17  9:18   ` Neil Armstrong
@ 2022-07-26 18:16   ` Tomeu Vizoso
  2022-07-29 15:43     ` Rob Clark
                       ` (2 more replies)
  1 sibling, 3 replies; 13+ messages in thread
From: Tomeu Vizoso @ 2022-07-26 18:16 UTC (permalink / raw)
  To: David Airlie, Jonathan Corbet, Carlo Caione, Kevin Hilman,
	Heiko Stuebner, Matthias Brugger, Rob Clark
  Cc: dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Tomeu Vizoso, Neil Armstrong

And use it to store expectations about what the DRM drivers are
supposed to pass in the IGT test suite.

Also include a configuration file that points to the out-of-tree CI
scripts.

By storing the test expectations along the code we can make sure both
stay in sync with each other, and so we can know when a code change
breaks those expectations.

This will allow all contributors to drm to reuse the infrastructure
already in gitlab.freedesktop.org to test the driver on several
generations of the hardware.

v2:
  - Fix names of result expectation files to match SoC
  - Don't execute tests that are going to skip on all boards

v3:
  - Remove tracking of dmesg output during test execution

v4:
  - Move up to drivers/gpu/drm
  - Add support for a bunch of other drivers
  - Explain how to incorporate fixes for CI from a
    ${TARGET_BRANCH}-external-fixes branch
  - Remove tests that pass from expected results file, to reduce the
    size of in-tree files
  - Add docs about how to deal with outages in automated testing labs
  - Specify the exact SHA of the CI scripts to be used

v5:
  - Remove unneeded skips from Meson expectations file
  - Use a more advanced runner that detects flakes automatically
  - Use a more succint format for the expectations
  - Run many more tests (and use sharding to finish in time)
  - Use skip lists to avoid hanging machines
  - Add some build testing
  - Build IGT in each pipeline for faster uprevs
  - List failures in the GitLab UI

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
---
 Documentation/gpu/automated_testing.rst       | 84 ++++++++++++++++++
 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 13 +++
 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
 drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
 drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
 drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
 drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
 drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
 drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
 drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
 drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
 drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
 drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
 drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
 drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
 drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
 drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
 drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
 drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
 drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
 drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
 drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
 drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
 drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
 drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
 drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
 drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  5 ++
 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 25 ++++++
 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
 46 files changed, 882 insertions(+)
 create mode 100644 Documentation/gpu/automated_testing.rst
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
 create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
 create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt

diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
new file mode 100644
index 000000000000..61003ecc7b6e
--- /dev/null
+++ b/Documentation/gpu/automated_testing.rst
@@ -0,0 +1,84 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+=========================================
+Automated testing of the DRM subsystem
+=========================================
+
+
+Introduction
+============
+
+Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
+
+Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
+
+There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
+
+
+Relevant files
+==============
+
+drivers/gpu/drm/ci/gitlab-ci.yml
+--------------------------------
+
+Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
+------------------------------------------
+
+Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
+----------------------------------------
+
+Specifies the expected results of running this specific kernel revision on a given hardware configuration.
+
+
+How to enable automated testing on your tree
+============================================
+
+1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
+
+2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
+
+3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
+
+4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
+
+
+How to update test expectations
+===============================
+
+If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
+
+If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
+
+
+How to expand coverage
+======================
+
+If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
+
+If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
+
+
+How to test your changes to the scripts
+==========================================
+
+For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
+
+
+How to incorporate external fixes in your testing
+=================================================
+
+Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
+
+If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
+
+
+How to deal with automated testing labs that may be down
+========================================================
+
+If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
new file mode 100644
index 000000000000..e721568b6ec3
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
@@ -0,0 +1,13 @@
+kms_addfb_basic@bad-pitch-65536,Fail
+kms_addfb_basic@bo-too-small,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_hdr@bpc-switch,Fail
+kms_hdr@bpc-switch-dpms,Fail
+kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
new file mode 100644
index 000000000000..a2cbc339838c
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
@@ -0,0 +1,20 @@
+kms_addfb_basic@too-high
+kms_atomic_transition@plane-all-modeset-transition-internal-panels
+kms_atomic_transition@plane-all-transition
+kms_atomic_transition@plane-use-after-nonblocking-unbind
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-B-tiling-none
+kms_plane@pixel-format
+kms_plane_scaling@downscale-with-rotation-factor-0-5
+kms_universal_plane@disable-primary-vs-flip-pipe-A
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
new file mode 100644
index 000000000000..8898bbe0f180
--- /dev/null
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -0,0 +1,13 @@
+variables:
+  # Change this to use your fork of drm-ci
+  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
+  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha bccff240ca042820484096f0782e8f060bd83bca
+
+  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
+  TARGET_BRANCH: drm-next
+
+include:
+  - project: *drm-ci-project-path
+    ref: *drm-ci-commit-sha
+    file:
+      - '.gitlab-ci.yml'
diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
new file mode 100644
index 000000000000..67d78f10b671
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
@@ -0,0 +1,32 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
new file mode 100644
index 000000000000..a077f29d5cba
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
@@ -0,0 +1,29 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
new file mode 100644
index 000000000000..618bbe6bb793
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
@@ -0,0 +1 @@
+kms_frontbuffer_tracking@fbc-tiling-linear
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
new file mode 100644
index 000000000000..a214419c6dc5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
@@ -0,0 +1,36 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@flip-vs-suspend
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_hdr@bpc-switch-suspend
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_psr2_su@page_flip-NV12
+kms_psr2_su@page_flip-P010
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
new file mode 100644
index 000000000000..aedb3d2ef885
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
@@ -0,0 +1,40 @@
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@blocking-wf_vblank
+kms_flip@wf_vblank-ts-check
+kms_flip@wf_vblank-ts-check-interruptible
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_frontbuffer_tracking@fbc-tiling-linear
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_rotation_crc@multiplane-rotation
+kms_rotation_crc@multiplane-rotation-cropping-bottom
+kms_rotation_crc@multiplane-rotation-cropping-top
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
new file mode 100644
index 000000000000..f81f065e27e8
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
@@ -0,0 +1,8 @@
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
new file mode 100644
index 000000000000..f82017654cf5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
@@ -0,0 +1,24 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
new file mode 100644
index 000000000000..270644ce220c
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
@@ -0,0 +1,19 @@
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_bw@linear-tiling-5-displays-1920x1080p,Fail
+kms_bw@linear-tiling-5-displays-2560x1440p,Fail
+kms_bw@linear-tiling-5-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_rotation_crc@bad-pixel-format,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
new file mode 100644
index 000000000000..15fba77ff19d
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
@@ -0,0 +1,6 @@
+kms_draw_crc@draw-method-rgb565-blt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
+kms_flip@blocking-absolute-wf_vblank
+kms_flip@bo-too-big-interruptible
+kms_flip@flip-vs-rmfb-interruptible
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
new file mode 100644
index 000000000000..14f33c7ba03b
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
@@ -0,0 +1,8 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# GPU hangs, then the whole machine
+gem_eio.*
+
+# Whole machine hangs
+kms_flip@absolute-wf_vblank@a-edp1
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
new file mode 100644
index 000000000000..dbfedfa5d63f
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
@@ -0,0 +1,30 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_frontbuffer_tracking@fbc-tiling-linear,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
new file mode 100644
index 000000000000..6bcac5b84fbb
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
@@ -0,0 +1 @@
+kms_flip@flip-vs-suspend
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
new file mode 100644
index 000000000000..7f74fb9afc99
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
@@ -0,0 +1,29 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@plane-properties-atomic,Fail
+kms_properties@plane-properties-legacy,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
new file mode 100644
index 000000000000..6ff81d00e84e
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
@@ -0,0 +1,10 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
new file mode 100644
index 000000000000..208890b79eb0
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
@@ -0,0 +1,14 @@
+core_setmaster_vs_auth
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_cursor_legacy@cursor-vs-flip-atomic
+kms_plane_scaling@invalid-num-scalers
+kms_plane_scaling@planes-upscale-20x20
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
+kms_plane_scaling@upscale-with-modifier-20x20
+kms_plane_scaling@upscale-with-pixel-format-20x20
+kms_prop_blob@invalid-set-prop-any
+kms_properties@get_properties-sanity-atomic
+kms_properties@plane-properties-atomic
+kms_properties@plane-properties-legacy
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
new file mode 100644
index 000000000000..345e1426b3a3
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
@@ -0,0 +1,5 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
new file mode 100644
index 000000000000..16c2dfab3f16
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
@@ -0,0 +1,15 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_cursor_legacy@all-pipes-forked-move,Fail
+kms_cursor_legacy@all-pipes-torture-bo,Fail
+kms_cursor_legacy@all-pipes-forked-bo,Fail
+kms_cursor_legacy@all-pipes-single-bo,Fail
+kms_cursor_legacy@all-pipes-single-move,Fail
+kms_cursor_legacy@all-pipes-torture-move,Fail
+kms_cursor_legacy@pipe-A-forked-bo,Fail
+kms_cursor_legacy@pipe-A-forked-move,Fail
+kms_cursor_legacy@pipe-A-single-bo,Fail
+kms_cursor_legacy@pipe-A-single-move,Fail
+kms_cursor_legacy@pipe-A-torture-bo,Fail
+kms_cursor_legacy@pipe-A-torture-move,Fail
+kms_hdmi_inject@inject-4k,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
new file mode 100644
index 000000000000..88a1fc0a3b0d
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
@@ -0,0 +1,2 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
new file mode 100644
index 000000000000..cd49c8ce2059
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
@@ -0,0 +1,2 @@
+# Whole machine hangs
+kms_cursor_legacy@all-pipes-torture-move
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
new file mode 100644
index 000000000000..0b159b6469c8
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
@@ -0,0 +1,22 @@
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane@plane-position-covered,Fail
+kms_plane@plane-position-hole,Fail
+kms_plane@plane-position-hole-dpms,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
+kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
+kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
+kms_rmfb@close-fd,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
new file mode 100644
index 000000000000..6022edace190
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
@@ -0,0 +1,14 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@pipe-.*
+
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-A-tiling-none
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
new file mode 100644
index 000000000000..40916ddff81f
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
@@ -0,0 +1,18 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Test incorrectly assumes that CTM support implies gamma/degamma
+# LUT support.  None of the subtests handle the case of only having
+# CTM support
+kms_color.*
+
+# 4k@60 is not supported on this hw, but driver doesn't handle it
+# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
+kms_bw@linear-tiling-.*-displays-3840x2160p
+
+# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
+kms_bw@linear-tiling-2.*
+kms_bw@linear-tiling-3.*
+kms_bw@linear-tiling-4.*
+kms_bw@linear-tiling-5.*
+kms_bw@linear-tiling-6.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
new file mode 100644
index 000000000000..8133927a44df
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
@@ -0,0 +1,44 @@
+kms_color@pipe-A-ctm-0-25,Fail
+kms_color@pipe-A-ctm-0-5,Fail
+kms_color@pipe-A-ctm-0-75,Fail
+kms_color@pipe-A-ctm-blue-to-red,Fail
+kms_color@pipe-A-ctm-green-to-red,Fail
+kms_color@pipe-A-ctm-max,Fail
+kms_color@pipe-A-ctm-negative,Fail
+kms_color@pipe-A-ctm-red-to-blue,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
+kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
+kms_cursor_legacy@cursor-vs-flip-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@flip-vs-cursor-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
+kms_cursor_legacy@flip-vs-cursor-legacy,Fail
+kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
+kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
+kms_plane_scaling@invalid-num-scalers,Fail
+kms_plane_scaling@planes-downscale-factor-0-25,Fail
+kms_plane_scaling@scaler-with-clipping-clamping,Timeout
+kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
+kms_rmfb@close-fd,Fail
diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
new file mode 100644
index 000000000000..da4406b74828
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
@@ -0,0 +1,33 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@pipe-A-cursor-alpha-opaque
+kms_cursor_crc@pipe-A-cursor-dpms
+kms_cursor_crc@pipe-A-cursor-size-change
+kms_cursor_legacy@flip-vs-cursor-toggle
+kms_cursor_legacy@pipe-A-forked-bo
+kms_cursor_legacy@pipe-A-forked-move
+kms_cursor_crc@pipe-A-cursor-32x10-offscreen
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen
+kms_cursor_crc@pipe-A-cursor-32x10-random
+kms_cursor_crc@pipe-A-cursor-32x10-sliding
+kms_cursor_crc@pipe-A-cursor-32x32-offscreen
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen
+kms_cursor_crc@pipe-A-cursor-32x32-random
+kms_cursor_crc@pipe-A-cursor-32x32-sliding
+kms_cursor_crc@pipe-A-cursor-64x21-offscreen
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen
+kms_cursor_crc@pipe-A-cursor-64x21-random
+kms_cursor_crc@pipe-A-cursor-64x21-sliding
+kms_cursor_crc@pipe-A-cursor-64x64-offscreen
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen
+kms_cursor_crc@pipe-A-cursor-64x64-random
+kms_cursor_crc@pipe-A-cursor-64x64-sliding
+kms_cursor_crc@pipe-A-cursor-dpms
+kms_cursor_crc@pipe-A-cursor-suspend
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+
+kms_flip@dpms-vs-vblank-race-interruptible
diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
new file mode 100644
index 000000000000..42675f1c6d76
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
@@ -0,0 +1,2 @@
+# Hangs machine
+kms_bw.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
new file mode 100644
index 000000000000..e2a264647a58
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
@@ -0,0 +1,75 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-A-cursor-dpms,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-primary-size-128,Fail
+kms_plane_cursor@pipe-A-primary-size-256,Fail
+kms_plane_cursor@pipe-A-primary-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_prime@basic-crc,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
new file mode 100644
index 000000000000..447fb26cc344
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
@@ -0,0 +1,5 @@
+kms_cursor_crc@pipe-B-cursor-dpms
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
new file mode 100644
index 000000000000..26f9a03fc074
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
@@ -0,0 +1,46 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
+
+# Started hanging the machine on Linux 5.19-rc2:
+#
+# [IGT] kms_plane_lowres: executing
+# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
+# [IGT] kms_plane_lowres: exiting, ret=77
+# Console: switching to colour frame buffer device 170x48
+# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
+# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
+# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
+# 8<--- cut here ---
+# Unable to handle kernel paging request at virtual address 7812078e
+# [7812078e] *pgd=00000000
+# Internal error: Oops: 5 [#1] SMP ARM
+# Modules linked in:
+# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
+# Hardware name: Rockchip (Device Tree)
+# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
+#  spin_dump from do_raw_spin_lock+0xa4/0xe8
+#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
+#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
+#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
+#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
+#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
+#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
+#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
+#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
+#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
+#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
+#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
+#  drm_client_dev_restore from drm_release+0xf4/0x114
+#  drm_release from __fput+0x74/0x240
+#  __fput from task_work_run+0x84/0xb4
+#  task_work_run from do_exit+0x34c/0xa20
+#  do_exit from do_group_exit+0x34/0x98
+#  do_group_exit from __wake_up_parent+0x0/0x18
+# Code: e595c008 12843d19 03e00000 03093168 (15940508) 
+# ---[ end trace 0000000000000000 ]---
+# note: kms_plane_lowre[482] exited with preempt_count 1
+# Fixing recursive fault but reboot is needed!
+kms_plane_lowres@pipe-F-tiling-y
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
new file mode 100644
index 000000000000..0f8afa7e2adc
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
@@ -0,0 +1,86 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_color@pipe-B-legacy-gamma,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
+kms_cursor_crc@pipe-A-cursor-dpms,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-B-cursor-dpms,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane@plane-panning-bottom-right,Fail
+kms_plane@plane-panning-top-left,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
+kms_prime@basic-crc,Fail
+kms_rmfb@close-fd,Fail
+kms_universal_plane@universal-plane-pipe-B-functional,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-B-query-forked,Fail
+kms_vblank@pipe-B-query-idle,Fail
+kms_vblank@pipe-B-wait-busy,Fail
+kms_vblank@pipe-B-wait-forked,Fail
+kms_vblank@pipe-B-wait-forked-busy,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
new file mode 100644
index 000000000000..b4c37a4315e4
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
@@ -0,0 +1,25 @@
+kms_cursor_crc@pipe-A-cursor-64x21-offscreen
+kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
+kms_cursor_crc@pipe-B-cursor-32x10-offscreen
+kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
+kms_cursor_crc@pipe-B-cursor-32x32-offscreen
+kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
+kms_cursor_crc@pipe-B-cursor-64x64-offscreen
+kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_flip@flip-vs-expired-vblank
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
+kms_plane_multiple@atomic-pipe-A-tiling-none
+kms_plane@plane-position-hole
+kms_sequence@get-forked
+kms_sequence@get-forked-busy
+kms_setmode@basic
+kms_vblank@pipe-A-accuracy-idle
+kms_vblank@pipe-A-query-busy
+kms_vblank@pipe-A-query-forked-busy
+kms_vblank@pipe-A-wait-idle
+kms_vblank@pipe-B-accuracy-idle
+kms_vblank@pipe-B-query-busy
+kms_vblank@pipe-B-query-forked-busy
+kms_vblank@pipe-B-wait-idle
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
new file mode 100644
index 000000000000..10c3d81a919a
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
@@ -0,0 +1,5 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
new file mode 100644
index 000000000000..2178bdd064dd
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
@@ -0,0 +1,38 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_addfb_basic@bad-pitch-65536,Fail
+kms_addfb_basic@bo-too-small,Fail
+kms_addfb_basic@size-max,Fail
+kms_addfb_basic@too-high,Fail
+kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_vblank@crtc-id,Fail
+kms_vblank@invalid,Fail
+kms_vblank@pipe-A-accuracy-idle,Fail
+kms_vblank@pipe-A-query-busy,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-forked-busy,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-ts-continuation-idle,Fail
+kms_vblank@pipe-A-ts-continuation-modeset,Fail
+kms_vblank@pipe-A-ts-continuation-suspend,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-A-wait-idle,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
new file mode 100644
index 000000000000..78be18174012
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
@@ -0,0 +1,6 @@
+# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
+# When run in a particular order with other tests
+kms_cursor_legacy.*
+
+# Job just hangs without any output
+kms_flip@flip-vs-suspend.*
\ No newline at end of file
-- 
2.31.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH v5] drm: Add initial ci/ subdirectory
  2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
@ 2022-07-29 15:43     ` Rob Clark
  2022-08-10 18:24     ` Rodrigo Siqueira Jordao
  2022-08-16 15:37     ` [PATCH v6] " Tomeu Vizoso
  2 siblings, 0 replies; 13+ messages in thread
From: Rob Clark @ 2022-07-29 15:43 UTC (permalink / raw)
  To: Tomeu Vizoso
  Cc: David Airlie, Jonathan Corbet, Carlo Caione, Kevin Hilman,
	Heiko Stuebner, Matthias Brugger, Rob Clark, linux-doc,
	Neil Armstrong, linux-kernel, dri-devel, linux-rockchip,
	linux-mediatek, linux-amlogic, kernel, linux-arm-kernel

On Tue, Jul 26, 2022 at 11:17 AM Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
>
> And use it to store expectations about what the DRM drivers are
> supposed to pass in the IGT test suite.
>
> Also include a configuration file that points to the out-of-tree CI
> scripts.
>
> By storing the test expectations along the code we can make sure both
> stay in sync with each other, and so we can know when a code change
> breaks those expectations.
>
> This will allow all contributors to drm to reuse the infrastructure
> already in gitlab.freedesktop.org to test the driver on several
> generations of the hardware.
>
> v2:
>   - Fix names of result expectation files to match SoC
>   - Don't execute tests that are going to skip on all boards
>
> v3:
>   - Remove tracking of dmesg output during test execution
>
> v4:
>   - Move up to drivers/gpu/drm
>   - Add support for a bunch of other drivers
>   - Explain how to incorporate fixes for CI from a
>     ${TARGET_BRANCH}-external-fixes branch
>   - Remove tests that pass from expected results file, to reduce the
>     size of in-tree files
>   - Add docs about how to deal with outages in automated testing labs
>   - Specify the exact SHA of the CI scripts to be used
>
> v5:
>   - Remove unneeded skips from Meson expectations file
>   - Use a more advanced runner that detects flakes automatically
>   - Use a more succint format for the expectations
>   - Run many more tests (and use sharding to finish in time)
>   - Use skip lists to avoid hanging machines
>   - Add some build testing
>   - Build IGT in each pipeline for faster uprevs
>   - List failures in the GitLab UI
>
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>

This looks good to me

Reviewed-by: Rob Clark <robdclark@gmail.com>


> ---
>  Documentation/gpu/automated_testing.rst       | 84 ++++++++++++++++++
>  drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 13 +++
>  drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
>  drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
>  drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
>  drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
>  drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
>  drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
>  drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
>  drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
>  drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
>  drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
>  drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
>  drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
>  drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
>  drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
>  drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
>  drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
>  drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
>  drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
>  drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
>  drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
>  drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
>  drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
>  drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
>  drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
>  drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
>  drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
>  drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
>  drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
>  drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
>  drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
>  drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
>  drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
>  drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
>  drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
>  drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
>  drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
>  drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  5 ++
>  drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
>  drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
>  drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 25 ++++++
>  drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
>  drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
>  drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
>  drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
>  46 files changed, 882 insertions(+)
>  create mode 100644 Documentation/gpu/automated_testing.rst
>  create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>  create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
>
> diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
> new file mode 100644
> index 000000000000..61003ecc7b6e
> --- /dev/null
> +++ b/Documentation/gpu/automated_testing.rst
> @@ -0,0 +1,84 @@
> +.. SPDX-License-Identifier: GPL-2.0+
> +
> +=========================================
> +Automated testing of the DRM subsystem
> +=========================================
> +
> +
> +Introduction
> +============
> +
> +Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
> +
> +Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
> +
> +There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
> +
> +
> +Relevant files
> +==============
> +
> +drivers/gpu/drm/ci/gitlab-ci.yml
> +--------------------------------
> +
> +Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
> +------------------------------------------
> +
> +Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.
> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
> +----------------------------------------
> +
> +Specifies the expected results of running this specific kernel revision on a given hardware configuration.
> +
> +
> +How to enable automated testing on your tree
> +============================================
> +
> +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
> +
> +2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
> +
> +3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
> +
> +4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
> +
> +
> +How to update test expectations
> +===============================
> +
> +If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
> +
> +If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
> +
> +
> +How to expand coverage
> +======================
> +
> +If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
> +
> +If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
> +
> +
> +How to test your changes to the scripts
> +==========================================
> +
> +For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
> +
> +
> +How to incorporate external fixes in your testing
> +=================================================
> +
> +Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
> +
> +If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
> +
> +
> +How to deal with automated testing labs that may be down
> +========================================================
> +
> +If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> new file mode 100644
> index 000000000000..e721568b6ec3
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> @@ -0,0 +1,13 @@
> +kms_addfb_basic@bad-pitch-65536,Fail
> +kms_addfb_basic@bo-too-small,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_hdr@bpc-switch,Fail
> +kms_hdr@bpc-switch-dpms,Fail
> +kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
> +kms_rotation_crc@primary-rotation-180,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> new file mode 100644
> index 000000000000..a2cbc339838c
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> @@ -0,0 +1,20 @@
> +kms_addfb_basic@too-high
> +kms_atomic_transition@plane-all-modeset-transition-internal-panels
> +kms_atomic_transition@plane-all-transition
> +kms_atomic_transition@plane-use-after-nonblocking-unbind
> +kms_bw@linear-tiling-1-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> +kms_plane_multiple@atomic-pipe-B-tiling-none
> +kms_plane@pixel-format
> +kms_plane_scaling@downscale-with-rotation-factor-0-5
> +kms_universal_plane@disable-primary-vs-flip-pipe-A
> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
> new file mode 100644
> index 000000000000..8898bbe0f180
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/gitlab-ci.yml
> @@ -0,0 +1,13 @@
> +variables:
> +  # Change this to use your fork of drm-ci
> +  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
> +  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha bccff240ca042820484096f0782e8f060bd83bca
> +
> +  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
> +  TARGET_BRANCH: drm-next
> +
> +include:
> +  - project: *drm-ci-project-path
> +    ref: *drm-ci-commit-sha
> +    file:
> +      - '.gitlab-ci.yml'
> diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> new file mode 100644
> index 000000000000..67d78f10b671
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> @@ -0,0 +1,32 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_sysfs_edid_timing
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
> new file mode 100644
> index 000000000000..a077f29d5cba
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
> @@ -0,0 +1,29 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> +kms_sysfs_edid_timing,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> new file mode 100644
> index 000000000000..618bbe6bb793
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> @@ -0,0 +1 @@
> +kms_frontbuffer_tracking@fbc-tiling-linear
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> new file mode 100644
> index 000000000000..a214419c6dc5
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> @@ -0,0 +1,36 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip@flip-vs-suspend
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_hdr@bpc-switch-suspend
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_psr2_su@page_flip-NV12
> +kms_psr2_su@page_flip-P010
> +kms_setmode@basic
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> new file mode 100644
> index 000000000000..aedb3d2ef885
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> @@ -0,0 +1,40 @@
> +kms_bw@linear-tiling-1-displays-3840x2160p
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip@blocking-wf_vblank
> +kms_flip@wf_vblank-ts-check
> +kms_flip@wf_vblank-ts-check-interruptible
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_frontbuffer_tracking@fbc-tiling-linear
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_rotation_crc@multiplane-rotation
> +kms_rotation_crc@multiplane-rotation-cropping-bottom
> +kms_rotation_crc@multiplane-rotation-cropping-top
> +kms_setmode@basic
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> new file mode 100644
> index 000000000000..f81f065e27e8
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> @@ -0,0 +1,8 @@
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> new file mode 100644
> index 000000000000..f82017654cf5
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> @@ -0,0 +1,24 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_sysfs_edid_timing
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> new file mode 100644
> index 000000000000..270644ce220c
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> @@ -0,0 +1,19 @@
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-5-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-5-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-5-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_rotation_crc@bad-pixel-format,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> new file mode 100644
> index 000000000000..15fba77ff19d
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> @@ -0,0 +1,6 @@
> +kms_draw_crc@draw-method-rgb565-blt-untiled
> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
> +kms_flip@blocking-absolute-wf_vblank
> +kms_flip@bo-too-big-interruptible
> +kms_flip@flip-vs-rmfb-interruptible
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> new file mode 100644
> index 000000000000..14f33c7ba03b
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> @@ -0,0 +1,8 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# GPU hangs, then the whole machine
> +gem_eio.*
> +
> +# Whole machine hangs
> +kms_flip@absolute-wf_vblank@a-edp1
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
> new file mode 100644
> index 000000000000..dbfedfa5d63f
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
> @@ -0,0 +1,30 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_frontbuffer_tracking@fbc-tiling-linear,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> +kms_sysfs_edid_timing,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> new file mode 100644
> index 000000000000..6bcac5b84fbb
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> @@ -0,0 +1 @@
> +kms_flip@flip-vs-suspend
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> new file mode 100644
> index 000000000000..7f74fb9afc99
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> @@ -0,0 +1,29 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> +kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
> +kms_force_connector_basic@force-connector-state,Fail
> +kms_force_connector_basic@force-edid,Fail
> +kms_force_connector_basic@force-load-detect,Fail
> +kms_force_connector_basic@prune-stale-modes,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_plane_scaling@planes-upscale-20x20,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@plane-properties-atomic,Fail
> +kms_properties@plane-properties-legacy,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> new file mode 100644
> index 000000000000..6ff81d00e84e
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> @@ -0,0 +1,10 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> new file mode 100644
> index 000000000000..208890b79eb0
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> @@ -0,0 +1,14 @@
> +core_setmaster_vs_auth
> +kms_bw@linear-tiling-1-displays-1920x1080p
> +kms_bw@linear-tiling-1-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_cursor_legacy@cursor-vs-flip-atomic
> +kms_plane_scaling@invalid-num-scalers
> +kms_plane_scaling@planes-upscale-20x20
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
> +kms_plane_scaling@upscale-with-modifier-20x20
> +kms_plane_scaling@upscale-with-pixel-format-20x20
> +kms_prop_blob@invalid-set-prop-any
> +kms_properties@get_properties-sanity-atomic
> +kms_properties@plane-properties-atomic
> +kms_properties@plane-properties-legacy
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> new file mode 100644
> index 000000000000..345e1426b3a3
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> @@ -0,0 +1,5 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_properties@connector-properties-atomic,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@get_properties-sanity-non-atomic,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> new file mode 100644
> index 000000000000..16c2dfab3f16
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> @@ -0,0 +1,15 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_cursor_legacy@all-pipes-forked-move,Fail
> +kms_cursor_legacy@all-pipes-torture-bo,Fail
> +kms_cursor_legacy@all-pipes-forked-bo,Fail
> +kms_cursor_legacy@all-pipes-single-bo,Fail
> +kms_cursor_legacy@all-pipes-single-move,Fail
> +kms_cursor_legacy@all-pipes-torture-move,Fail
> +kms_cursor_legacy@pipe-A-forked-bo,Fail
> +kms_cursor_legacy@pipe-A-forked-move,Fail
> +kms_cursor_legacy@pipe-A-single-bo,Fail
> +kms_cursor_legacy@pipe-A-single-move,Fail
> +kms_cursor_legacy@pipe-A-torture-bo,Fail
> +kms_cursor_legacy@pipe-A-torture-move,Fail
> +kms_hdmi_inject@inject-4k,Fail
> diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> new file mode 100644
> index 000000000000..88a1fc0a3b0d
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> @@ -0,0 +1,2 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> new file mode 100644
> index 000000000000..cd49c8ce2059
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> @@ -0,0 +1,2 @@
> +# Whole machine hangs
> +kms_cursor_legacy@all-pipes-torture-move
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> new file mode 100644
> index 000000000000..0b159b6469c8
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> @@ -0,0 +1,22 @@
> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> +kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
> +kms_plane@pixel-format,Fail
> +kms_plane@pixel-format-source-clamping,Fail
> +kms_plane@plane-position-covered,Fail
> +kms_plane@plane-position-hole,Fail
> +kms_plane@plane-position-hole-dpms,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
> +kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
> +kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
> +kms_rmfb@close-fd,Fail
> +kms_rotation_crc@primary-rotation-180,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> new file mode 100644
> index 000000000000..6022edace190
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> @@ -0,0 +1,14 @@
> +# Test ends up reading CRC from frame before cursor update
> +# sometimes.. tbd if this is a kernel CRC bug or a test
> +# bug
> +kms_cursor_crc@pipe-.*
> +
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> +kms_plane_multiple@atomic-pipe-A-tiling-none
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> new file mode 100644
> index 000000000000..40916ddff81f
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> @@ -0,0 +1,18 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Test incorrectly assumes that CTM support implies gamma/degamma
> +# LUT support.  None of the subtests handle the case of only having
> +# CTM support
> +kms_color.*
> +
> +# 4k@60 is not supported on this hw, but driver doesn't handle it
> +# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
> +kms_bw@linear-tiling-.*-displays-3840x2160p
> +
> +# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
> +kms_bw@linear-tiling-2.*
> +kms_bw@linear-tiling-3.*
> +kms_bw@linear-tiling-4.*
> +kms_bw@linear-tiling-5.*
> +kms_bw@linear-tiling-6.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> new file mode 100644
> index 000000000000..8133927a44df
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> @@ -0,0 +1,44 @@
> +kms_color@pipe-A-ctm-0-25,Fail
> +kms_color@pipe-A-ctm-0-5,Fail
> +kms_color@pipe-A-ctm-0-75,Fail
> +kms_color@pipe-A-ctm-blue-to-red,Fail
> +kms_color@pipe-A-ctm-green-to-red,Fail
> +kms_color@pipe-A-ctm-max,Fail
> +kms_color@pipe-A-ctm-negative,Fail
> +kms_color@pipe-A-ctm-red-to-blue,Fail
> +kms_color@pipe-A-legacy-gamma,Fail
> +kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
> +kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
> +kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
> +kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
> +kms_cursor_legacy@cursor-vs-flip-atomic,Fail
> +kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
> +kms_cursor_legacy@cursor-vs-flip-legacy,Fail
> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> +kms_cursor_legacy@flip-vs-cursor-atomic,Fail
> +kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
> +kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
> +kms_cursor_legacy@flip-vs-cursor-legacy,Fail
> +kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
> +kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
> +kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
> +kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
> +kms_plane@pixel-format,Fail
> +kms_plane@pixel-format-source-clamping,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
> +kms_plane_scaling@invalid-num-scalers,Fail
> +kms_plane_scaling@planes-downscale-factor-0-25,Fail
> +kms_plane_scaling@scaler-with-clipping-clamping,Timeout
> +kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
> +kms_rmfb@close-fd,Fail
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> new file mode 100644
> index 000000000000..da4406b74828
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> @@ -0,0 +1,33 @@
> +# Test ends up reading CRC from frame before cursor update
> +# sometimes.. tbd if this is a kernel CRC bug or a test
> +# bug
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque
> +kms_cursor_crc@pipe-A-cursor-dpms
> +kms_cursor_crc@pipe-A-cursor-size-change
> +kms_cursor_legacy@flip-vs-cursor-toggle
> +kms_cursor_legacy@pipe-A-forked-bo
> +kms_cursor_legacy@pipe-A-forked-move
> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen
> +kms_cursor_crc@pipe-A-cursor-32x10-random
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding
> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen
> +kms_cursor_crc@pipe-A-cursor-32x32-random
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding
> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-random
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding
> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen
> +kms_cursor_crc@pipe-A-cursor-64x64-random
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding
> +kms_cursor_crc@pipe-A-cursor-dpms
> +kms_cursor_crc@pipe-A-cursor-suspend
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +
> +kms_flip@dpms-vs-vblank-race-interruptible
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> new file mode 100644
> index 000000000000..42675f1c6d76
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> @@ -0,0 +1,2 @@
> +# Hangs machine
> +kms_bw.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> new file mode 100644
> index 000000000000..e2a264647a58
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> @@ -0,0 +1,75 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_force_connector_basic@force-connector-state,Fail
> +kms_force_connector_basic@force-edid,Fail
> +kms_force_connector_basic@force-load-detect,Fail
> +kms_force_connector_basic@prune-stale-modes,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> +kms_plane_cursor@pipe-A-primary-size-128,Fail
> +kms_plane_cursor@pipe-A-primary-size-256,Fail
> +kms_plane_cursor@pipe-A-primary-size-64,Fail
> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> +kms_plane_cursor@pipe-B-primary-size-128,Fail
> +kms_plane_cursor@pipe-B-primary-size-256,Fail
> +kms_plane_cursor@pipe-B-primary-size-64,Fail
> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_prime@basic-crc,Fail
> +kms_properties@connector-properties-atomic,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@get_properties-sanity-non-atomic,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> new file mode 100644
> index 000000000000..447fb26cc344
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> @@ -0,0 +1,5 @@
> +kms_cursor_crc@pipe-B-cursor-dpms
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> new file mode 100644
> index 000000000000..26f9a03fc074
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> @@ -0,0 +1,46 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Too unstable, machine ends up hanging after lots of Oopses
> +kms_cursor_legacy.*
> +
> +# Started hanging the machine on Linux 5.19-rc2:
> +#
> +# [IGT] kms_plane_lowres: executing
> +# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
> +# [IGT] kms_plane_lowres: exiting, ret=77
> +# Console: switching to colour frame buffer device 170x48
> +# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
> +# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
> +# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
> +# 8<--- cut here ---
> +# Unable to handle kernel paging request at virtual address 7812078e
> +# [7812078e] *pgd=00000000
> +# Internal error: Oops: 5 [#1] SMP ARM
> +# Modules linked in:
> +# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
> +# Hardware name: Rockchip (Device Tree)
> +# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
> +#  spin_dump from do_raw_spin_lock+0xa4/0xe8
> +#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
> +#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
> +#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
> +#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
> +#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
> +#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
> +#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
> +#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
> +#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
> +#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
> +#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
> +#  drm_client_dev_restore from drm_release+0xf4/0x114
> +#  drm_release from __fput+0x74/0x240
> +#  __fput from task_work_run+0x84/0xb4
> +#  task_work_run from do_exit+0x34c/0xa20
> +#  do_exit from do_group_exit+0x34/0x98
> +#  do_group_exit from __wake_up_parent+0x0/0x18
> +# Code: e595c008 12843d19 03e00000 03093168 (15940508)
> +# ---[ end trace 0000000000000000 ]---
> +# note: kms_plane_lowre[482] exited with preempt_count 1
> +# Fixing recursive fault but reboot is needed!
> +kms_plane_lowres@pipe-F-tiling-y
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> new file mode 100644
> index 000000000000..0f8afa7e2adc
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> @@ -0,0 +1,86 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_color@pipe-A-legacy-gamma,Fail
> +kms_color@pipe-B-legacy-gamma,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-B-cursor-dpms,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> +kms_plane@plane-panning-bottom-right,Fail
> +kms_plane@plane-panning-top-left,Fail
> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> +kms_plane_cursor@pipe-B-primary-size-128,Fail
> +kms_plane_cursor@pipe-B-primary-size-256,Fail
> +kms_plane_cursor@pipe-B-primary-size-64,Fail
> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> +kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
> +kms_prime@basic-crc,Fail
> +kms_rmfb@close-fd,Fail
> +kms_universal_plane@universal-plane-pipe-B-functional,Fail
> +kms_vblank@pipe-A-query-forked,Fail
> +kms_vblank@pipe-A-query-idle,Fail
> +kms_vblank@pipe-A-wait-busy,Fail
> +kms_vblank@pipe-A-wait-forked,Fail
> +kms_vblank@pipe-A-wait-forked-busy,Fail
> +kms_vblank@pipe-B-query-forked,Fail
> +kms_vblank@pipe-B-query-idle,Fail
> +kms_vblank@pipe-B-wait-busy,Fail
> +kms_vblank@pipe-B-wait-forked,Fail
> +kms_vblank@pipe-B-wait-forked-busy,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> new file mode 100644
> index 000000000000..b4c37a4315e4
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> @@ -0,0 +1,25 @@
> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-32x10-offscreen
> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen
> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen
> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_flip@flip-vs-expired-vblank
> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
> +kms_plane_multiple@atomic-pipe-A-tiling-none
> +kms_plane@plane-position-hole
> +kms_sequence@get-forked
> +kms_sequence@get-forked-busy
> +kms_setmode@basic
> +kms_vblank@pipe-A-accuracy-idle
> +kms_vblank@pipe-A-query-busy
> +kms_vblank@pipe-A-query-forked-busy
> +kms_vblank@pipe-A-wait-idle
> +kms_vblank@pipe-B-accuracy-idle
> +kms_vblank@pipe-B-query-busy
> +kms_vblank@pipe-B-query-forked-busy
> +kms_vblank@pipe-B-wait-idle
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> new file mode 100644
> index 000000000000..10c3d81a919a
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> @@ -0,0 +1,5 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Too unstable, machine ends up hanging after lots of Oopses
> +kms_cursor_legacy.*
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> new file mode 100644
> index 000000000000..2178bdd064dd
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> @@ -0,0 +1,38 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_addfb_basic@bad-pitch-65536,Fail
> +kms_addfb_basic@bo-too-small,Fail
> +kms_addfb_basic@size-max,Fail
> +kms_addfb_basic@too-high,Fail
> +kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_vblank@crtc-id,Fail
> +kms_vblank@invalid,Fail
> +kms_vblank@pipe-A-accuracy-idle,Fail
> +kms_vblank@pipe-A-query-busy,Fail
> +kms_vblank@pipe-A-query-forked,Fail
> +kms_vblank@pipe-A-query-forked-busy,Fail
> +kms_vblank@pipe-A-query-idle,Fail
> +kms_vblank@pipe-A-ts-continuation-idle,Fail
> +kms_vblank@pipe-A-ts-continuation-modeset,Fail
> +kms_vblank@pipe-A-ts-continuation-suspend,Fail
> +kms_vblank@pipe-A-wait-busy,Fail
> +kms_vblank@pipe-A-wait-forked,Fail
> +kms_vblank@pipe-A-wait-forked-busy,Fail
> +kms_vblank@pipe-A-wait-idle,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> new file mode 100644
> index 000000000000..78be18174012
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> @@ -0,0 +1,6 @@
> +# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
> +# When run in a particular order with other tests
> +kms_cursor_legacy.*
> +
> +# Job just hangs without any output
> +kms_flip@flip-vs-suspend.*
> \ No newline at end of file
> --
> 2.31.1
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v5] drm: Add initial ci/ subdirectory
  2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
  2022-07-29 15:43     ` Rob Clark
@ 2022-08-10 18:24     ` Rodrigo Siqueira Jordao
  2022-08-10 22:08       ` Rob Clark
  2022-08-11 10:10       ` Tomeu Vizoso
  2022-08-16 15:37     ` [PATCH v6] " Tomeu Vizoso
  2 siblings, 2 replies; 13+ messages in thread
From: Rodrigo Siqueira Jordao @ 2022-08-10 18:24 UTC (permalink / raw)
  To: Tomeu Vizoso, Nicholas Choi, Sun peng Li, Chiu, Harrison,
	Wentland, Harry
  Cc: linux-doc, Neil Armstrong, linux-kernel, dri-devel,
	linux-rockchip, linux-mediatek, linux-amlogic, kernel,
	linux-arm-kernel, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger, Rob Clark

Hi Tomeu,

First of all, nice patch! I just saw it, and I have some basic questions 
(I don't understand many of these CI details). I also CC some CI folks 
from the display team at AMD.

On 2022-07-26 14:16, Tomeu Vizoso wrote:
> And use it to store expectations about what the DRM drivers are
> supposed to pass in the IGT test suite.
> 
> Also include a configuration file that points to the out-of-tree CI
> scripts.
> 
> By storing the test expectations along the code we can make sure both
> stay in sync with each other, and so we can know when a code change
> breaks those expectations.
> 
> This will allow all contributors to drm to reuse the infrastructure
> already in gitlab.freedesktop.org to test the driver on several
> generations of the hardware.
> 
> v2:
>    - Fix names of result expectation files to match SoC
>    - Don't execute tests that are going to skip on all boards
> 
> v3:
>    - Remove tracking of dmesg output during test execution
> 
> v4:
>    - Move up to drivers/gpu/drm
>    - Add support for a bunch of other drivers
>    - Explain how to incorporate fixes for CI from a
>      ${TARGET_BRANCH}-external-fixes branch
>    - Remove tests that pass from expected results file, to reduce the
>      size of in-tree files
>    - Add docs about how to deal with outages in automated testing labs
>    - Specify the exact SHA of the CI scripts to be used
> 
> v5:
>    - Remove unneeded skips from Meson expectations file
>    - Use a more advanced runner that detects flakes automatically
>    - Use a more succint format for the expectations
>    - Run many more tests (and use sharding to finish in time)
>    - Use skip lists to avoid hanging machines
>    - Add some build testing
>    - Build IGT in each pipeline for faster uprevs
>    - List failures in the GitLab UI
> 
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
> ---
>   Documentation/gpu/automated_testing.rst       | 84 ++++++++++++++++++
>   drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 13 +++
>   drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
>   drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
>   drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
>   drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
>   drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
>   drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
>   drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
>   drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
>   drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
>   drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
>   drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
>   drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
>   drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
>   drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
>   drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
>   drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
>   drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
>   drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
>   drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
>   drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
>   drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
>   drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
>   drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
>   drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
>   drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
>   drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
>   drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
>   drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
>   drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
>   drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
>   drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
>   drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
>   drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
>   drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
>   drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
>   drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
>   drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  5 ++
>   drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
>   drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
>   drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 25 ++++++
>   drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
>   drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
>   drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
>   drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
>   46 files changed, 882 insertions(+)
>   create mode 100644 Documentation/gpu/automated_testing.rst
>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>   create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> 
> diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
> new file mode 100644
> index 000000000000..61003ecc7b6e
> --- /dev/null
> +++ b/Documentation/gpu/automated_testing.rst
> @@ -0,0 +1,84 @@
> +.. SPDX-License-Identifier: GPL-2.0+
> +
> +=========================================
> +Automated testing of the DRM subsystem
> +=========================================
> +
> +
> +Introduction
> +============
> +
> +Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
> +
> +Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
> +
> +There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
> +
> +
> +Relevant files
> +==============
> +
> +drivers/gpu/drm/ci/gitlab-ci.yml
> +--------------------------------
> +
> +Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
> +------------------------------------------
> +
> +Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.

The IGT repo has testlists per vendor (e.g., 
https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/tree/master/tests/intel-ci). 
What is the advantage of keeping a similar IGT list here? Should we 
maintain that list in both projects?

> +
> +
> +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
> +----------------------------------------
> +
> +Specifies the expected results of running this specific kernel revision on a given hardware configuration.
> +
> +
> +How to enable automated testing on your tree
> +============================================
> +
> +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
> +
> +2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
> +
> +3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
> +
> +4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
> +
> +
> +How to update test expectations
> +===============================
> +
> +If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
> +
> +If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
> +
> +
> +How to expand coverage
> +======================
> +
> +If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
> +
> +If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.

Usually, we run our IGT test on the latest code from the IGT repo. Is it 
possible to configure this behavior?

> +
> +
> +How to test your changes to the scripts
> +==========================================
> +
> +For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
> +
> +
> +How to incorporate external fixes in your testing
> +=================================================
> +
> +Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
> +
> +If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
> +
> +
> +How to deal with automated testing labs that may be down
> +========================================================
> +
> +If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.


This part is the one that I really want to understand. How can we plug 
our farm? Do you have a guideline? I want to know if we can run a prototype.


> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> new file mode 100644
> index 000000000000..e721568b6ec3
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> @@ -0,0 +1,13 @@
> +kms_addfb_basic@bad-pitch-65536,Fail
> +kms_addfb_basic@bo-too-small,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_hdr@bpc-switch,Fail
> +kms_hdr@bpc-switch-dpms,Fail
> +kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
> +kms_rotation_crc@primary-rotation-180,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> new file mode 100644
> index 000000000000..a2cbc339838c
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> @@ -0,0 +1,20 @@
> +kms_addfb_basic@too-high
> +kms_atomic_transition@plane-all-modeset-transition-internal-panels
> +kms_atomic_transition@plane-all-transition
> +kms_atomic_transition@plane-use-after-nonblocking-unbind
> +kms_bw@linear-tiling-1-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> +kms_plane_multiple@atomic-pipe-B-tiling-none
> +kms_plane@pixel-format
> +kms_plane_scaling@downscale-with-rotation-factor-0-5
> +kms_universal_plane@disable-primary-vs-flip-pipe-A

We have multiple test lists for different ASICs, and we are working to 
have one unified list. I guess we just need to that list here?

Maybe Nicholas/Herrison can provide a complete list here.

Thanks
Siqueira


> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
> new file mode 100644
> index 000000000000..8898bbe0f180
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/gitlab-ci.yml
> @@ -0,0 +1,13 @@
> +variables:
> +  # Change this to use your fork of drm-ci
> +  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
> +  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha bccff240ca042820484096f0782e8f060bd83bca
> +
> +  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
> +  TARGET_BRANCH: drm-next
> +
> +include:
> +  - project: *drm-ci-project-path
> +    ref: *drm-ci-commit-sha
> +    file:
> +      - '.gitlab-ci.yml'
> diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> new file mode 100644
> index 000000000000..67d78f10b671
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> @@ -0,0 +1,32 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_sysfs_edid_timing
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
> new file mode 100644
> index 000000000000..a077f29d5cba
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
> @@ -0,0 +1,29 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> +kms_sysfs_edid_timing,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> new file mode 100644
> index 000000000000..618bbe6bb793
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> @@ -0,0 +1 @@
> +kms_frontbuffer_tracking@fbc-tiling-linear
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> new file mode 100644
> index 000000000000..a214419c6dc5
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> @@ -0,0 +1,36 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip@flip-vs-suspend
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_hdr@bpc-switch-suspend
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_psr2_su@page_flip-NV12
> +kms_psr2_su@page_flip-P010
> +kms_setmode@basic
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> new file mode 100644
> index 000000000000..aedb3d2ef885
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> @@ -0,0 +1,40 @@
> +kms_bw@linear-tiling-1-displays-3840x2160p
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-2560x1440p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-2560x1440p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip@blocking-wf_vblank
> +kms_flip@wf_vblank-ts-check
> +kms_flip@wf_vblank-ts-check-interruptible
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> +kms_frontbuffer_tracking@fbc-tiling-linear
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> +kms_rotation_crc@multiplane-rotation
> +kms_rotation_crc@multiplane-rotation-cropping-bottom
> +kms_rotation_crc@multiplane-rotation-cropping-top
> +kms_setmode@basic
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> new file mode 100644
> index 000000000000..f81f065e27e8
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> @@ -0,0 +1,8 @@
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> new file mode 100644
> index 000000000000..f82017654cf5
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> @@ -0,0 +1,24 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p
> +kms_bw@linear-tiling-2-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_bw@linear-tiling-3-displays-2560x1440p
> +kms_bw@linear-tiling-3-displays-3840x2160p
> +kms_bw@linear-tiling-4-displays-1920x1080p
> +kms_bw@linear-tiling-4-displays-3840x2160p
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> +kms_plane_alpha_blend@pipe-A-alpha-basic
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-alpha-basic
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> +kms_plane_alpha_blend@pipe-C-alpha-basic
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> +kms_sysfs_edid_timing
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> new file mode 100644
> index 000000000000..e2c538a0f954
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> @@ -0,0 +1,2 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> new file mode 100644
> index 000000000000..270644ce220c
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> @@ -0,0 +1,19 @@
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-5-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-5-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-5-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_rotation_crc@bad-pixel-format,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> new file mode 100644
> index 000000000000..15fba77ff19d
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> @@ -0,0 +1,6 @@
> +kms_draw_crc@draw-method-rgb565-blt-untiled
> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
> +kms_flip@blocking-absolute-wf_vblank
> +kms_flip@bo-too-big-interruptible
> +kms_flip@flip-vs-rmfb-interruptible
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> new file mode 100644
> index 000000000000..14f33c7ba03b
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> @@ -0,0 +1,8 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# GPU hangs, then the whole machine
> +gem_eio.*
> +
> +# Whole machine hangs
> +kms_flip@absolute-wf_vblank@a-edp1
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
> new file mode 100644
> index 000000000000..dbfedfa5d63f
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
> @@ -0,0 +1,30 @@
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> +kms_frontbuffer_tracking@fbc-tiling-linear,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> +kms_sysfs_edid_timing,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> new file mode 100644
> index 000000000000..6bcac5b84fbb
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> @@ -0,0 +1 @@
> +kms_flip@flip-vs-suspend
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> new file mode 100644
> index 000000000000..7f74fb9afc99
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> @@ -0,0 +1,29 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> +kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
> +kms_force_connector_basic@force-connector-state,Fail
> +kms_force_connector_basic@force-edid,Fail
> +kms_force_connector_basic@force-load-detect,Fail
> +kms_force_connector_basic@prune-stale-modes,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_plane_scaling@planes-upscale-20x20,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@plane-properties-atomic,Fail
> +kms_properties@plane-properties-legacy,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> new file mode 100644
> index 000000000000..6ff81d00e84e
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> @@ -0,0 +1,10 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> new file mode 100644
> index 000000000000..208890b79eb0
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> @@ -0,0 +1,14 @@
> +core_setmaster_vs_auth
> +kms_bw@linear-tiling-1-displays-1920x1080p
> +kms_bw@linear-tiling-1-displays-3840x2160p
> +kms_bw@linear-tiling-3-displays-1920x1080p
> +kms_cursor_legacy@cursor-vs-flip-atomic
> +kms_plane_scaling@invalid-num-scalers
> +kms_plane_scaling@planes-upscale-20x20
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
> +kms_plane_scaling@upscale-with-modifier-20x20
> +kms_plane_scaling@upscale-with-pixel-format-20x20
> +kms_prop_blob@invalid-set-prop-any
> +kms_properties@get_properties-sanity-atomic
> +kms_properties@plane-properties-atomic
> +kms_properties@plane-properties-legacy
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> new file mode 100644
> index 000000000000..345e1426b3a3
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> @@ -0,0 +1,5 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_properties@connector-properties-atomic,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@get_properties-sanity-non-atomic,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> new file mode 100644
> index 000000000000..16c2dfab3f16
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> @@ -0,0 +1,15 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_cursor_legacy@all-pipes-forked-move,Fail
> +kms_cursor_legacy@all-pipes-torture-bo,Fail
> +kms_cursor_legacy@all-pipes-forked-bo,Fail
> +kms_cursor_legacy@all-pipes-single-bo,Fail
> +kms_cursor_legacy@all-pipes-single-move,Fail
> +kms_cursor_legacy@all-pipes-torture-move,Fail
> +kms_cursor_legacy@pipe-A-forked-bo,Fail
> +kms_cursor_legacy@pipe-A-forked-move,Fail
> +kms_cursor_legacy@pipe-A-single-bo,Fail
> +kms_cursor_legacy@pipe-A-single-move,Fail
> +kms_cursor_legacy@pipe-A-torture-bo,Fail
> +kms_cursor_legacy@pipe-A-torture-move,Fail
> +kms_hdmi_inject@inject-4k,Fail
> diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> new file mode 100644
> index 000000000000..88a1fc0a3b0d
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> @@ -0,0 +1,2 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> new file mode 100644
> index 000000000000..b63329d06767
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> @@ -0,0 +1,4 @@
> +kms_force_connector_basic@force-connector-state
> +kms_force_connector_basic@force-edid
> +kms_force_connector_basic@force-load-detect
> +kms_force_connector_basic@prune-stale-modes
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> new file mode 100644
> index 000000000000..cd49c8ce2059
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> @@ -0,0 +1,2 @@
> +# Whole machine hangs
> +kms_cursor_legacy@all-pipes-torture-move
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> new file mode 100644
> index 000000000000..0b159b6469c8
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> @@ -0,0 +1,22 @@
> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> +kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
> +kms_plane@pixel-format,Fail
> +kms_plane@pixel-format-source-clamping,Fail
> +kms_plane@plane-position-covered,Fail
> +kms_plane@plane-position-hole,Fail
> +kms_plane@plane-position-hole-dpms,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> +kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
> +kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
> +kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
> +kms_rmfb@close-fd,Fail
> +kms_rotation_crc@primary-rotation-180,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> +kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> new file mode 100644
> index 000000000000..6022edace190
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> @@ -0,0 +1,14 @@
> +# Test ends up reading CRC from frame before cursor update
> +# sometimes.. tbd if this is a kernel CRC bug or a test
> +# bug
> +kms_cursor_crc@pipe-.*
> +
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> +kms_plane_multiple@atomic-pipe-A-tiling-none
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> new file mode 100644
> index 000000000000..40916ddff81f
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> @@ -0,0 +1,18 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Test incorrectly assumes that CTM support implies gamma/degamma
> +# LUT support.  None of the subtests handle the case of only having
> +# CTM support
> +kms_color.*
> +
> +# 4k@60 is not supported on this hw, but driver doesn't handle it
> +# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
> +kms_bw@linear-tiling-.*-displays-3840x2160p
> +
> +# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
> +kms_bw@linear-tiling-2.*
> +kms_bw@linear-tiling-3.*
> +kms_bw@linear-tiling-4.*
> +kms_bw@linear-tiling-5.*
> +kms_bw@linear-tiling-6.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> new file mode 100644
> index 000000000000..8133927a44df
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> @@ -0,0 +1,44 @@
> +kms_color@pipe-A-ctm-0-25,Fail
> +kms_color@pipe-A-ctm-0-5,Fail
> +kms_color@pipe-A-ctm-0-75,Fail
> +kms_color@pipe-A-ctm-blue-to-red,Fail
> +kms_color@pipe-A-ctm-green-to-red,Fail
> +kms_color@pipe-A-ctm-max,Fail
> +kms_color@pipe-A-ctm-negative,Fail
> +kms_color@pipe-A-ctm-red-to-blue,Fail
> +kms_color@pipe-A-legacy-gamma,Fail
> +kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
> +kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
> +kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
> +kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
> +kms_cursor_legacy@cursor-vs-flip-atomic,Fail
> +kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
> +kms_cursor_legacy@cursor-vs-flip-legacy,Fail
> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> +kms_cursor_legacy@flip-vs-cursor-atomic,Fail
> +kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
> +kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
> +kms_cursor_legacy@flip-vs-cursor-legacy,Fail
> +kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
> +kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
> +kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
> +kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
> +kms_plane@pixel-format,Fail
> +kms_plane@pixel-format-source-clamping,Fail
> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
> +kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
> +kms_plane_scaling@invalid-num-scalers,Fail
> +kms_plane_scaling@planes-downscale-factor-0-25,Fail
> +kms_plane_scaling@scaler-with-clipping-clamping,Timeout
> +kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
> +kms_rmfb@close-fd,Fail
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> new file mode 100644
> index 000000000000..da4406b74828
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> @@ -0,0 +1,33 @@
> +# Test ends up reading CRC from frame before cursor update
> +# sometimes.. tbd if this is a kernel CRC bug or a test
> +# bug
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque
> +kms_cursor_crc@pipe-A-cursor-dpms
> +kms_cursor_crc@pipe-A-cursor-size-change
> +kms_cursor_legacy@flip-vs-cursor-toggle
> +kms_cursor_legacy@pipe-A-forked-bo
> +kms_cursor_legacy@pipe-A-forked-move
> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen
> +kms_cursor_crc@pipe-A-cursor-32x10-random
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding
> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen
> +kms_cursor_crc@pipe-A-cursor-32x32-random
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding
> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-random
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding
> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen
> +kms_cursor_crc@pipe-A-cursor-64x64-random
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding
> +kms_cursor_crc@pipe-A-cursor-dpms
> +kms_cursor_crc@pipe-A-cursor-suspend
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +
> +kms_flip@dpms-vs-vblank-race-interruptible
> diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> new file mode 100644
> index 000000000000..42675f1c6d76
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> @@ -0,0 +1,2 @@
> +# Hangs machine
> +kms_bw.*
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> new file mode 100644
> index 000000000000..e2a264647a58
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> @@ -0,0 +1,75 @@
> +kms_3d,Fail
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_force_connector_basic@force-connector-state,Fail
> +kms_force_connector_basic@force-edid,Fail
> +kms_force_connector_basic@force-load-detect,Fail
> +kms_force_connector_basic@prune-stale-modes,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> +kms_plane_cursor@pipe-A-primary-size-128,Fail
> +kms_plane_cursor@pipe-A-primary-size-256,Fail
> +kms_plane_cursor@pipe-A-primary-size-64,Fail
> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> +kms_plane_cursor@pipe-B-primary-size-128,Fail
> +kms_plane_cursor@pipe-B-primary-size-256,Fail
> +kms_plane_cursor@pipe-B-primary-size-64,Fail
> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_prime@basic-crc,Fail
> +kms_properties@connector-properties-atomic,Fail
> +kms_properties@get_properties-sanity-atomic,Fail
> +kms_properties@get_properties-sanity-non-atomic,Fail
> +kms_rmfb@close-fd,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> new file mode 100644
> index 000000000000..447fb26cc344
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> @@ -0,0 +1,5 @@
> +kms_cursor_crc@pipe-B-cursor-dpms
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> new file mode 100644
> index 000000000000..26f9a03fc074
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> @@ -0,0 +1,46 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Too unstable, machine ends up hanging after lots of Oopses
> +kms_cursor_legacy.*
> +
> +# Started hanging the machine on Linux 5.19-rc2:
> +#
> +# [IGT] kms_plane_lowres: executing
> +# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
> +# [IGT] kms_plane_lowres: exiting, ret=77
> +# Console: switching to colour frame buffer device 170x48
> +# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
> +# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
> +# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
> +# 8<--- cut here ---
> +# Unable to handle kernel paging request at virtual address 7812078e
> +# [7812078e] *pgd=00000000
> +# Internal error: Oops: 5 [#1] SMP ARM
> +# Modules linked in:
> +# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
> +# Hardware name: Rockchip (Device Tree)
> +# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
> +#  spin_dump from do_raw_spin_lock+0xa4/0xe8
> +#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
> +#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
> +#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
> +#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
> +#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
> +#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
> +#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
> +#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
> +#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
> +#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
> +#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
> +#  drm_client_dev_restore from drm_release+0xf4/0x114
> +#  drm_release from __fput+0x74/0x240
> +#  __fput from task_work_run+0x84/0xb4
> +#  task_work_run from do_exit+0x34c/0xa20
> +#  do_exit from do_group_exit+0x34/0x98
> +#  do_group_exit from __wake_up_parent+0x0/0x18
> +# Code: e595c008 12843d19 03e00000 03093168 (15940508)
> +# ---[ end trace 0000000000000000 ]---
> +# note: kms_plane_lowre[482] exited with preempt_count 1
> +# Fixing recursive fault but reboot is needed!
> +kms_plane_lowres@pipe-F-tiling-y
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> new file mode 100644
> index 000000000000..0f8afa7e2adc
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> @@ -0,0 +1,86 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_color@pipe-A-legacy-gamma,Fail
> +kms_color@pipe-B-legacy-gamma,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> +kms_cursor_crc@pipe-B-cursor-dpms,Fail
> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
> +kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
> +kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A,Fail
> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B,Fail
> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> +kms_plane@plane-panning-bottom-right,Fail
> +kms_plane@plane-panning-top-left,Fail
> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> +kms_plane_cursor@pipe-B-primary-size-128,Fail
> +kms_plane_cursor@pipe-B-primary-size-256,Fail
> +kms_plane_cursor@pipe-B-primary-size-64,Fail
> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> +kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
> +kms_prime@basic-crc,Fail
> +kms_rmfb@close-fd,Fail
> +kms_universal_plane@universal-plane-pipe-B-functional,Fail
> +kms_vblank@pipe-A-query-forked,Fail
> +kms_vblank@pipe-A-query-idle,Fail
> +kms_vblank@pipe-A-wait-busy,Fail
> +kms_vblank@pipe-A-wait-forked,Fail
> +kms_vblank@pipe-A-wait-forked-busy,Fail
> +kms_vblank@pipe-B-query-forked,Fail
> +kms_vblank@pipe-B-query-idle,Fail
> +kms_vblank@pipe-B-wait-busy,Fail
> +kms_vblank@pipe-B-wait-forked,Fail
> +kms_vblank@pipe-B-wait-forked-busy,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> new file mode 100644
> index 000000000000..b4c37a4315e4
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> @@ -0,0 +1,25 @@
> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> +kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-32x10-offscreen
> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen
> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen
> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> +kms_flip@flip-vs-expired-vblank
> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
> +kms_plane_multiple@atomic-pipe-A-tiling-none
> +kms_plane@plane-position-hole
> +kms_sequence@get-forked
> +kms_sequence@get-forked-busy
> +kms_setmode@basic
> +kms_vblank@pipe-A-accuracy-idle
> +kms_vblank@pipe-A-query-busy
> +kms_vblank@pipe-A-query-forked-busy
> +kms_vblank@pipe-A-wait-idle
> +kms_vblank@pipe-B-accuracy-idle
> +kms_vblank@pipe-B-query-busy
> +kms_vblank@pipe-B-query-forked-busy
> +kms_vblank@pipe-B-wait-idle
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> new file mode 100644
> index 000000000000..10c3d81a919a
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> @@ -0,0 +1,5 @@
> +# Suspend to RAM seems to be broken on this machine
> +.*suspend.*
> +
> +# Too unstable, machine ends up hanging after lots of Oopses
> +kms_cursor_legacy.*
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> new file mode 100644
> index 000000000000..2178bdd064dd
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> @@ -0,0 +1,38 @@
> +kms_addfb_basic@addfb25-bad-modifier,Fail
> +kms_addfb_basic@bad-pitch-65536,Fail
> +kms_addfb_basic@bo-too-small,Fail
> +kms_addfb_basic@size-max,Fail
> +kms_addfb_basic@too-high,Fail
> +kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> +kms_invalid_mode@int-max-clock,Fail
> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> +kms_vblank@crtc-id,Fail
> +kms_vblank@invalid,Fail
> +kms_vblank@pipe-A-accuracy-idle,Fail
> +kms_vblank@pipe-A-query-busy,Fail
> +kms_vblank@pipe-A-query-forked,Fail
> +kms_vblank@pipe-A-query-forked-busy,Fail
> +kms_vblank@pipe-A-query-idle,Fail
> +kms_vblank@pipe-A-ts-continuation-idle,Fail
> +kms_vblank@pipe-A-ts-continuation-modeset,Fail
> +kms_vblank@pipe-A-ts-continuation-suspend,Fail
> +kms_vblank@pipe-A-wait-busy,Fail
> +kms_vblank@pipe-A-wait-forked,Fail
> +kms_vblank@pipe-A-wait-forked-busy,Fail
> +kms_vblank@pipe-A-wait-idle,Fail
> \ No newline at end of file
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> new file mode 100644
> index 000000000000..78be18174012
> --- /dev/null
> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> @@ -0,0 +1,6 @@
> +# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
> +# When run in a particular order with other tests
> +kms_cursor_legacy.*
> +
> +# Job just hangs without any output
> +kms_flip@flip-vs-suspend.*
> \ No newline at end of file


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v5] drm: Add initial ci/ subdirectory
  2022-08-10 18:24     ` Rodrigo Siqueira Jordao
@ 2022-08-10 22:08       ` Rob Clark
  2022-08-11 10:10       ` Tomeu Vizoso
  1 sibling, 0 replies; 13+ messages in thread
From: Rob Clark @ 2022-08-10 22:08 UTC (permalink / raw)
  To: Rodrigo Siqueira Jordao
  Cc: Tomeu Vizoso, Nicholas Choi, Sun peng Li, Chiu, Harrison,
	Wentland, Harry, Rob Clark, Jonathan Corbet, Neil Armstrong,
	David Airlie, Kevin Hilman, linux-doc, linux-kernel, dri-devel,
	linux-rockchip, linux-mediatek, Carlo Caione, linux-amlogic,
	kernel, linux-arm-kernel, Matthias Brugger

On Wed, Aug 10, 2022 at 11:25 AM Rodrigo Siqueira Jordao
<Rodrigo.Siqueira@amd.com> wrote:
>
> Hi Tomeu,
>
> First of all, nice patch! I just saw it, and I have some basic questions
> (I don't understand many of these CI details). I also CC some CI folks
> from the display team at AMD.
>
> On 2022-07-26 14:16, Tomeu Vizoso wrote:
> > And use it to store expectations about what the DRM drivers are
> > supposed to pass in the IGT test suite.
> >
> > Also include a configuration file that points to the out-of-tree CI
> > scripts.
> >
> > By storing the test expectations along the code we can make sure both
> > stay in sync with each other, and so we can know when a code change
> > breaks those expectations.
> >
> > This will allow all contributors to drm to reuse the infrastructure
> > already in gitlab.freedesktop.org to test the driver on several
> > generations of the hardware.
> >
> > v2:
> >    - Fix names of result expectation files to match SoC
> >    - Don't execute tests that are going to skip on all boards
> >
> > v3:
> >    - Remove tracking of dmesg output during test execution
> >
> > v4:
> >    - Move up to drivers/gpu/drm
> >    - Add support for a bunch of other drivers
> >    - Explain how to incorporate fixes for CI from a
> >      ${TARGET_BRANCH}-external-fixes branch
> >    - Remove tests that pass from expected results file, to reduce the
> >      size of in-tree files
> >    - Add docs about how to deal with outages in automated testing labs
> >    - Specify the exact SHA of the CI scripts to be used
> >
> > v5:
> >    - Remove unneeded skips from Meson expectations file
> >    - Use a more advanced runner that detects flakes automatically
> >    - Use a more succint format for the expectations
> >    - Run many more tests (and use sharding to finish in time)
> >    - Use skip lists to avoid hanging machines
> >    - Add some build testing
> >    - Build IGT in each pipeline for faster uprevs
> >    - List failures in the GitLab UI
> >
> > Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
> > Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
> > ---
> >   Documentation/gpu/automated_testing.rst       | 84 ++++++++++++++++++
> >   drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 13 +++
> >   drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
> >   drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
> >   drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
> >   drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
> >   drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
> >   drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
> >   drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
> >   drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
> >   drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
> >   drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
> >   drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
> >   drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
> >   drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
> >   drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
> >   drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
> >   drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
> >   drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
> >   drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
> >   drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
> >   drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
> >   drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
> >   drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
> >   drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
> >   drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
> >   drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
> >   drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
> >   drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
> >   drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
> >   drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
> >   drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
> >   drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
> >   drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
> >   drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
> >   drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
> >   drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
> >   drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
> >   drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  5 ++
> >   drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
> >   drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
> >   drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 25 ++++++
> >   drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
> >   drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
> >   drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
> >   drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
> >   46 files changed, 882 insertions(+)
> >   create mode 100644 Documentation/gpu/automated_testing.rst
> >   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
> >   create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> >   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> >   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
> >   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> >
> > diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
> > new file mode 100644
> > index 000000000000..61003ecc7b6e
> > --- /dev/null
> > +++ b/Documentation/gpu/automated_testing.rst
> > @@ -0,0 +1,84 @@
> > +.. SPDX-License-Identifier: GPL-2.0+
> > +
> > +=========================================
> > +Automated testing of the DRM subsystem
> > +=========================================
> > +
> > +
> > +Introduction
> > +============
> > +
> > +Making sure that changes to the core or drivers don't introduce regressions can be very time consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
> > +
> > +Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers finding them and then reporting back.
> > +
> > +There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
> > +
> > +
> > +Relevant files
> > +==============
> > +
> > +drivers/gpu/drm/ci/gitlab-ci.yml
> > +--------------------------------
> > +
> > +Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
> > +
> > +
> > +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
> > +------------------------------------------
> > +
> > +Specifies the tests that the current code is expected to be able to reliably run for ${DRIVER_NAME}. These tests are expected to not hang the DUT (Device Under Testing) when running on the revision they belong to, and to give consistent results.
>
> The IGT repo has testlists per vendor (e.g.,
> https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/tree/master/tests/intel-ci).
> What is the advantage of keeping a similar IGT list here? Should we
> maintain that list in both projects?
>
> > +
> > +
> > +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
> > +----------------------------------------
> > +
> > +Specifies the expected results of running this specific kernel revision on a given hardware configuration.
> > +
> > +
> > +How to enable automated testing on your tree
> > +============================================
> > +
> > +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
> > +
> > +2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
> > +
> > +3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
> > +
> > +4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
> > +
> > +
> > +How to update test expectations
> > +===============================
> > +
> > +If your changes to the code fix any tests, you will have to update one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test platforms affected by the change.
> > +
> > +If you have run a pipeline and it failed because of a mismatched test result, you will have been offered a patch at the end of the run that you can apply after reviewing it.
> > +
> > +
> > +How to expand coverage
> > +======================
> > +
> > +If your code changes makes it possible to run more tests (by solving reliability issues, for example), more tests can be added to this list, and then the expected results updated with the patch file that will be printed at the end of each job.
> > +
> > +If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
>
> Usually, we run our IGT test on the latest code from the IGT repo. Is it
> possible to configure this behavior?

That is *probably* not a good idea, in case test results change as a
result of IGT version (ie. IGT test fix causes something to change
fail->pass, new test introduced that fails on some devices, etc)

> > +
> > +
> > +How to test your changes to the scripts
> > +==========================================
> > +
> > +For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
> > +
> > +
> > +How to incorporate external fixes in your testing
> > +=================================================
> > +
> > +Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
> > +
> > +If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
> > +
> > +
> > +How to deal with automated testing labs that may be down
> > +========================================================
> > +
> > +If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
>
>
> This part is the one that I really want to understand. How can we plug
> our farm? Do you have a guideline? I want to know if we can run a prototype.

There are some docs in mesa: https://docs.mesa3d.org/ci/index.html

BR,
-R

>
> > diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> > new file mode 100644
> > index 000000000000..e721568b6ec3
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
> > @@ -0,0 +1,13 @@
> > +kms_addfb_basic@bad-pitch-65536,Fail
> > +kms_addfb_basic@bo-too-small,Fail
> > +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> > +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> > +kms_hdr@bpc-switch,Fail
> > +kms_hdr@bpc-switch-dpms,Fail
> > +kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
> > +kms_rotation_crc@primary-rotation-180,Fail
> > +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> > new file mode 100644
> > index 000000000000..a2cbc339838c
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
> > @@ -0,0 +1,20 @@
> > +kms_addfb_basic@too-high
> > +kms_atomic_transition@plane-all-modeset-transition-internal-panels
> > +kms_atomic_transition@plane-all-transition
> > +kms_atomic_transition@plane-use-after-nonblocking-unbind
> > +kms_bw@linear-tiling-1-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-3840x2160p
> > +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> > +kms_plane_multiple@atomic-pipe-B-tiling-none
> > +kms_plane@pixel-format
> > +kms_plane_scaling@downscale-with-rotation-factor-0-5
> > +kms_universal_plane@disable-primary-vs-flip-pipe-A
>
> We have multiple test lists for different ASICs, and we are working to
> have one unified list. I guess we just need to that list here?
>
> Maybe Nicholas/Herrison can provide a complete list here.
>
> Thanks
> Siqueira
>
>
> > diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> > new file mode 100644
> > index 000000000000..e2c538a0f954
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
> > new file mode 100644
> > index 000000000000..8898bbe0f180
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/gitlab-ci.yml
> > @@ -0,0 +1,13 @@
> > +variables:
> > +  # Change this to use your fork of drm-ci
> > +  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
> > +  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha bccff240ca042820484096f0782e8f060bd83bca
> > +
> > +  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
> > +  TARGET_BRANCH: drm-next
> > +
> > +include:
> > +  - project: *drm-ci-project-path
> > +    ref: *drm-ci-commit-sha
> > +    file:
> > +      - '.gitlab-ci.yml'
> > diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> > new file mode 100644
> > index 000000000000..67d78f10b671
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
> > @@ -0,0 +1,32 @@
> > +kms_bw@linear-tiling-2-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-2560x1440p
> > +kms_bw@linear-tiling-2-displays-3840x2160p
> > +kms_bw@linear-tiling-3-displays-1920x1080p
> > +kms_bw@linear-tiling-3-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-3840x2160p
> > +kms_bw@linear-tiling-4-displays-1920x1080p
> > +kms_bw@linear-tiling-4-displays-2560x1440p
> > +kms_bw@linear-tiling-4-displays-3840x2160p
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> > +kms_plane_alpha_blend@pipe-A-alpha-basic
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-B-alpha-basic
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-C-alpha-basic
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> > +kms_sysfs_edid_timing
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
> > new file mode 100644
> > index 000000000000..e2c538a0f954
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
> > new file mode 100644
> > index 000000000000..a077f29d5cba
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
> > @@ -0,0 +1,29 @@
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> > +kms_sysfs_edid_timing,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> > new file mode 100644
> > index 000000000000..618bbe6bb793
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
> > @@ -0,0 +1 @@
> > +kms_frontbuffer_tracking@fbc-tiling-linear
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
> > new file mode 100644
> > index 000000000000..e2c538a0f954
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> > new file mode 100644
> > index 000000000000..a214419c6dc5
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
> > @@ -0,0 +1,36 @@
> > +kms_bw@linear-tiling-2-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-2560x1440p
> > +kms_bw@linear-tiling-2-displays-3840x2160p
> > +kms_bw@linear-tiling-3-displays-1920x1080p
> > +kms_bw@linear-tiling-3-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-3840x2160p
> > +kms_bw@linear-tiling-4-displays-1920x1080p
> > +kms_bw@linear-tiling-4-displays-2560x1440p
> > +kms_bw@linear-tiling-4-displays-3840x2160p
> > +kms_flip@flip-vs-suspend
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> > +kms_hdr@bpc-switch-suspend
> > +kms_plane_alpha_blend@pipe-A-alpha-basic
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-B-alpha-basic
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-C-alpha-basic
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> > +kms_psr2_su@page_flip-NV12
> > +kms_psr2_su@page_flip-P010
> > +kms_setmode@basic
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> > new file mode 100644
> > index 000000000000..aedb3d2ef885
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
> > @@ -0,0 +1,40 @@
> > +kms_bw@linear-tiling-1-displays-3840x2160p
> > +kms_bw@linear-tiling-2-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-2560x1440p
> > +kms_bw@linear-tiling-2-displays-3840x2160p
> > +kms_bw@linear-tiling-3-displays-1920x1080p
> > +kms_bw@linear-tiling-3-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-3840x2160p
> > +kms_bw@linear-tiling-4-displays-1920x1080p
> > +kms_bw@linear-tiling-4-displays-2560x1440p
> > +kms_bw@linear-tiling-4-displays-3840x2160p
> > +kms_flip@blocking-wf_vblank
> > +kms_flip@wf_vblank-ts-check
> > +kms_flip@wf_vblank-ts-check-interruptible
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
> > +kms_frontbuffer_tracking@fbc-tiling-linear
> > +kms_plane_alpha_blend@pipe-A-alpha-basic
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-B-alpha-basic
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-C-alpha-basic
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max
> > +kms_rotation_crc@multiplane-rotation
> > +kms_rotation_crc@multiplane-rotation-cropping-bottom
> > +kms_rotation_crc@multiplane-rotation-cropping-top
> > +kms_setmode@basic
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
> > new file mode 100644
> > index 000000000000..e2c538a0f954
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> > new file mode 100644
> > index 000000000000..f81f065e27e8
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
> > @@ -0,0 +1,8 @@
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> > new file mode 100644
> > index 000000000000..f82017654cf5
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
> > @@ -0,0 +1,24 @@
> > +kms_bw@linear-tiling-2-displays-1920x1080p
> > +kms_bw@linear-tiling-2-displays-3840x2160p
> > +kms_bw@linear-tiling-3-displays-1920x1080p
> > +kms_bw@linear-tiling-3-displays-2560x1440p
> > +kms_bw@linear-tiling-3-displays-3840x2160p
> > +kms_bw@linear-tiling-4-displays-1920x1080p
> > +kms_bw@linear-tiling-4-displays-3840x2160p
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
> > +kms_plane_alpha_blend@pipe-A-alpha-basic
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-B-alpha-basic
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max
> > +kms_plane_alpha_blend@pipe-C-alpha-basic
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
> > +kms_sysfs_edid_timing
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> > new file mode 100644
> > index 000000000000..e2c538a0f954
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> > new file mode 100644
> > index 000000000000..270644ce220c
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
> > @@ -0,0 +1,19 @@
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-5-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-5-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-5-displays-3840x2160p,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> > +kms_rotation_crc@bad-pixel-format,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> > new file mode 100644
> > index 000000000000..15fba77ff19d
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
> > @@ -0,0 +1,6 @@
> > +kms_draw_crc@draw-method-rgb565-blt-untiled
> > +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
> > +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
> > +kms_flip@blocking-absolute-wf_vblank
> > +kms_flip@bo-too-big-interruptible
> > +kms_flip@flip-vs-rmfb-interruptible
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> > new file mode 100644
> > index 000000000000..14f33c7ba03b
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
> > @@ -0,0 +1,8 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > +
> > +# GPU hangs, then the whole machine
> > +gem_eio.*
> > +
> > +# Whole machine hangs
> > +kms_flip@absolute-wf_vblank@a-edp1
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
> > new file mode 100644
> > index 000000000000..dbfedfa5d63f
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
> > @@ -0,0 +1,30 @@
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
> > +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
> > +kms_frontbuffer_tracking@fbc-tiling-linear,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
> > +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
> > +kms_sysfs_edid_timing,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> > new file mode 100644
> > index 000000000000..6bcac5b84fbb
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
> > @@ -0,0 +1 @@
> > +kms_flip@flip-vs-suspend
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> > new file mode 100644
> > index 000000000000..7f74fb9afc99
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
> > @@ -0,0 +1,29 @@
> > +kms_3d,Fail
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> > +kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
> > +kms_force_connector_basic@force-connector-state,Fail
> > +kms_force_connector_basic@force-edid,Fail
> > +kms_force_connector_basic@force-load-detect,Fail
> > +kms_force_connector_basic@prune-stale-modes,Fail
> > +kms_invalid_mode@int-max-clock,Fail
> > +kms_plane_scaling@planes-upscale-20x20,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> > +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> > +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> > +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> > +kms_properties@get_properties-sanity-atomic,Fail
> > +kms_properties@plane-properties-atomic,Fail
> > +kms_properties@plane-properties-legacy,Fail
> > +kms_rmfb@close-fd,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> > new file mode 100644
> > index 000000000000..6ff81d00e84e
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
> > @@ -0,0 +1,10 @@
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
> > +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> > +kms_rmfb@close-fd,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> > new file mode 100644
> > index 000000000000..208890b79eb0
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
> > @@ -0,0 +1,14 @@
> > +core_setmaster_vs_auth
> > +kms_bw@linear-tiling-1-displays-1920x1080p
> > +kms_bw@linear-tiling-1-displays-3840x2160p
> > +kms_bw@linear-tiling-3-displays-1920x1080p
> > +kms_cursor_legacy@cursor-vs-flip-atomic
> > +kms_plane_scaling@invalid-num-scalers
> > +kms_plane_scaling@planes-upscale-20x20
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
> > +kms_plane_scaling@upscale-with-modifier-20x20
> > +kms_plane_scaling@upscale-with-pixel-format-20x20
> > +kms_prop_blob@invalid-set-prop-any
> > +kms_properties@get_properties-sanity-atomic
> > +kms_properties@plane-properties-atomic
> > +kms_properties@plane-properties-legacy
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> > new file mode 100644
> > index 000000000000..345e1426b3a3
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
> > @@ -0,0 +1,5 @@
> > +kms_3d,Fail
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_properties@connector-properties-atomic,Fail
> > +kms_properties@get_properties-sanity-atomic,Fail
> > +kms_properties@get_properties-sanity-non-atomic,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> > new file mode 100644
> > index 000000000000..b63329d06767
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
> > @@ -0,0 +1,4 @@
> > +kms_force_connector_basic@force-connector-state
> > +kms_force_connector_basic@force-edid
> > +kms_force_connector_basic@force-load-detect
> > +kms_force_connector_basic@prune-stale-modes
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> > new file mode 100644
> > index 000000000000..16c2dfab3f16
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
> > @@ -0,0 +1,15 @@
> > +kms_3d,Fail
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_cursor_legacy@all-pipes-forked-move,Fail
> > +kms_cursor_legacy@all-pipes-torture-bo,Fail
> > +kms_cursor_legacy@all-pipes-forked-bo,Fail
> > +kms_cursor_legacy@all-pipes-single-bo,Fail
> > +kms_cursor_legacy@all-pipes-single-move,Fail
> > +kms_cursor_legacy@all-pipes-torture-move,Fail
> > +kms_cursor_legacy@pipe-A-forked-bo,Fail
> > +kms_cursor_legacy@pipe-A-forked-move,Fail
> > +kms_cursor_legacy@pipe-A-single-bo,Fail
> > +kms_cursor_legacy@pipe-A-single-move,Fail
> > +kms_cursor_legacy@pipe-A-torture-bo,Fail
> > +kms_cursor_legacy@pipe-A-torture-move,Fail
> > +kms_hdmi_inject@inject-4k,Fail
> > diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> > new file mode 100644
> > index 000000000000..b63329d06767
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
> > @@ -0,0 +1,4 @@
> > +kms_force_connector_basic@force-connector-state
> > +kms_force_connector_basic@force-edid
> > +kms_force_connector_basic@force-load-detect
> > +kms_force_connector_basic@prune-stale-modes
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> > new file mode 100644
> > index 000000000000..88a1fc0a3b0d
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
> > @@ -0,0 +1,2 @@
> > +kms_3d,Fail
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> > new file mode 100644
> > index 000000000000..b63329d06767
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
> > @@ -0,0 +1,4 @@
> > +kms_force_connector_basic@force-connector-state
> > +kms_force_connector_basic@force-edid
> > +kms_force_connector_basic@force-load-detect
> > +kms_force_connector_basic@prune-stale-modes
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> > new file mode 100644
> > index 000000000000..cd49c8ce2059
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Whole machine hangs
> > +kms_cursor_legacy@all-pipes-torture-move
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> > new file mode 100644
> > index 000000000000..0b159b6469c8
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
> > @@ -0,0 +1,22 @@
> > +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> > +kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
> > +kms_plane@pixel-format,Fail
> > +kms_plane@pixel-format-source-clamping,Fail
> > +kms_plane@plane-position-covered,Fail
> > +kms_plane@plane-position-hole,Fail
> > +kms_plane@plane-position-hole-dpms,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> > +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> > +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
> > +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
> > +kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
> > +kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
> > +kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
> > +kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
> > +kms_rmfb@close-fd,Fail
> > +kms_rotation_crc@primary-rotation-180,Fail
> > +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
> > +kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
> > diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> > new file mode 100644
> > index 000000000000..6022edace190
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
> > @@ -0,0 +1,14 @@
> > +# Test ends up reading CRC from frame before cursor update
> > +# sometimes.. tbd if this is a kernel CRC bug or a test
> > +# bug
> > +kms_cursor_crc@pipe-.*
> > +
> > +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> > +kms_plane_multiple@atomic-pipe-A-tiling-none
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> > new file mode 100644
> > index 000000000000..40916ddff81f
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
> > @@ -0,0 +1,18 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > +
> > +# Test incorrectly assumes that CTM support implies gamma/degamma
> > +# LUT support.  None of the subtests handle the case of only having
> > +# CTM support
> > +kms_color.*
> > +
> > +# 4k@60 is not supported on this hw, but driver doesn't handle it
> > +# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
> > +kms_bw@linear-tiling-.*-displays-3840x2160p
> > +
> > +# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
> > +kms_bw@linear-tiling-2.*
> > +kms_bw@linear-tiling-3.*
> > +kms_bw@linear-tiling-4.*
> > +kms_bw@linear-tiling-5.*
> > +kms_bw@linear-tiling-6.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> > new file mode 100644
> > index 000000000000..8133927a44df
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
> > @@ -0,0 +1,44 @@
> > +kms_color@pipe-A-ctm-0-25,Fail
> > +kms_color@pipe-A-ctm-0-5,Fail
> > +kms_color@pipe-A-ctm-0-75,Fail
> > +kms_color@pipe-A-ctm-blue-to-red,Fail
> > +kms_color@pipe-A-ctm-green-to-red,Fail
> > +kms_color@pipe-A-ctm-max,Fail
> > +kms_color@pipe-A-ctm-negative,Fail
> > +kms_color@pipe-A-ctm-red-to-blue,Fail
> > +kms_color@pipe-A-legacy-gamma,Fail
> > +kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
> > +kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
> > +kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
> > +kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
> > +kms_cursor_legacy@cursor-vs-flip-atomic,Fail
> > +kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
> > +kms_cursor_legacy@cursor-vs-flip-legacy,Fail
> > +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
> > +kms_cursor_legacy@flip-vs-cursor-atomic,Fail
> > +kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
> > +kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
> > +kms_cursor_legacy@flip-vs-cursor-legacy,Fail
> > +kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
> > +kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
> > +kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
> > +kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
> > +kms_plane@pixel-format,Fail
> > +kms_plane@pixel-format-source-clamping,Fail
> > +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
> > +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
> > +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> > +kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
> > +kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
> > +kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
> > +kms_plane_scaling@invalid-num-scalers,Fail
> > +kms_plane_scaling@planes-downscale-factor-0-25,Fail
> > +kms_plane_scaling@scaler-with-clipping-clamping,Timeout
> > +kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
> > +kms_rmfb@close-fd,Fail
> > diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> > new file mode 100644
> > index 000000000000..da4406b74828
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
> > @@ -0,0 +1,33 @@
> > +# Test ends up reading CRC from frame before cursor update
> > +# sometimes.. tbd if this is a kernel CRC bug or a test
> > +# bug
> > +kms_cursor_crc@pipe-A-cursor-alpha-opaque
> > +kms_cursor_crc@pipe-A-cursor-dpms
> > +kms_cursor_crc@pipe-A-cursor-size-change
> > +kms_cursor_legacy@flip-vs-cursor-toggle
> > +kms_cursor_legacy@pipe-A-forked-bo
> > +kms_cursor_legacy@pipe-A-forked-move
> > +kms_cursor_crc@pipe-A-cursor-32x10-offscreen
> > +kms_cursor_crc@pipe-A-cursor-32x10-onscreen
> > +kms_cursor_crc@pipe-A-cursor-32x10-random
> > +kms_cursor_crc@pipe-A-cursor-32x10-sliding
> > +kms_cursor_crc@pipe-A-cursor-32x32-offscreen
> > +kms_cursor_crc@pipe-A-cursor-32x32-onscreen
> > +kms_cursor_crc@pipe-A-cursor-32x32-random
> > +kms_cursor_crc@pipe-A-cursor-32x32-sliding
> > +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> > +kms_cursor_crc@pipe-A-cursor-64x21-onscreen
> > +kms_cursor_crc@pipe-A-cursor-64x21-random
> > +kms_cursor_crc@pipe-A-cursor-64x21-sliding
> > +kms_cursor_crc@pipe-A-cursor-64x64-offscreen
> > +kms_cursor_crc@pipe-A-cursor-64x64-onscreen
> > +kms_cursor_crc@pipe-A-cursor-64x64-random
> > +kms_cursor_crc@pipe-A-cursor-64x64-sliding
> > +kms_cursor_crc@pipe-A-cursor-dpms
> > +kms_cursor_crc@pipe-A-cursor-suspend
> > +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-left-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> > +
> > +kms_flip@dpms-vs-vblank-race-interruptible
> > diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> > new file mode 100644
> > index 000000000000..42675f1c6d76
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
> > @@ -0,0 +1,2 @@
> > +# Hangs machine
> > +kms_bw.*
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> > new file mode 100644
> > index 000000000000..e2a264647a58
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
> > @@ -0,0 +1,75 @@
> > +kms_3d,Fail
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> > +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> > +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> > +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> > +kms_force_connector_basic@force-connector-state,Fail
> > +kms_force_connector_basic@force-edid,Fail
> > +kms_force_connector_basic@force-load-detect,Fail
> > +kms_force_connector_basic@prune-stale-modes,Fail
> > +kms_invalid_mode@int-max-clock,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-128,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-256,Fail
> > +kms_plane_cursor@pipe-A-overlay-size-64,Fail
> > +kms_plane_cursor@pipe-A-primary-size-128,Fail
> > +kms_plane_cursor@pipe-A-primary-size-256,Fail
> > +kms_plane_cursor@pipe-A-primary-size-64,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-128,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-256,Fail
> > +kms_plane_cursor@pipe-A-viewport-size-64,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> > +kms_plane_cursor@pipe-B-primary-size-128,Fail
> > +kms_plane_cursor@pipe-B-primary-size-256,Fail
> > +kms_plane_cursor@pipe-B-primary-size-64,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> > +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> > +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> > +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> > +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> > +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> > +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> > +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> > +kms_prime@basic-crc,Fail
> > +kms_properties@connector-properties-atomic,Fail
> > +kms_properties@get_properties-sanity-atomic,Fail
> > +kms_properties@get_properties-sanity-non-atomic,Fail
> > +kms_rmfb@close-fd,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> > new file mode 100644
> > index 000000000000..447fb26cc344
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
> > @@ -0,0 +1,5 @@
> > +kms_cursor_crc@pipe-B-cursor-dpms
> > +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-A-64x64-top-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-top-edge
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> > new file mode 100644
> > index 000000000000..26f9a03fc074
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
> > @@ -0,0 +1,46 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > +
> > +# Too unstable, machine ends up hanging after lots of Oopses
> > +kms_cursor_legacy.*
> > +
> > +# Started hanging the machine on Linux 5.19-rc2:
> > +#
> > +# [IGT] kms_plane_lowres: executing
> > +# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
> > +# [IGT] kms_plane_lowres: exiting, ret=77
> > +# Console: switching to colour frame buffer device 170x48
> > +# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
> > +# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
> > +# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
> > +# 8<--- cut here ---
> > +# Unable to handle kernel paging request at virtual address 7812078e
> > +# [7812078e] *pgd=00000000
> > +# Internal error: Oops: 5 [#1] SMP ARM
> > +# Modules linked in:
> > +# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
> > +# Hardware name: Rockchip (Device Tree)
> > +# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
> > +#  spin_dump from do_raw_spin_lock+0xa4/0xe8
> > +#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
> > +#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
> > +#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
> > +#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
> > +#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
> > +#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
> > +#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
> > +#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
> > +#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
> > +#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
> > +#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
> > +#  drm_client_dev_restore from drm_release+0xf4/0x114
> > +#  drm_release from __fput+0x74/0x240
> > +#  __fput from task_work_run+0x84/0xb4
> > +#  task_work_run from do_exit+0x34c/0xa20
> > +#  do_exit from do_group_exit+0x34/0x98
> > +#  do_group_exit from __wake_up_parent+0x0/0x18
> > +# Code: e595c008 12843d19 03e00000 03093168 (15940508)
> > +# ---[ end trace 0000000000000000 ]---
> > +# note: kms_plane_lowre[482] exited with preempt_count 1
> > +# Fixing recursive fault but reboot is needed!
> > +kms_plane_lowres@pipe-F-tiling-y
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> > new file mode 100644
> > index 000000000000..0f8afa7e2adc
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
> > @@ -0,0 +1,86 @@
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_color@pipe-A-legacy-gamma,Fail
> > +kms_color@pipe-B-legacy-gamma,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
> > +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
> > +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
> > +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
> > +kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
> > +kms_cursor_crc@pipe-A-cursor-dpms,Fail
> > +kms_cursor_crc@pipe-A-cursor-size-change,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
> > +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
> > +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
> > +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
> > +kms_cursor_crc@pipe-B-cursor-dpms,Fail
> > +kms_cursor_crc@pipe-B-cursor-size-change,Fail
> > +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
> > +kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
> > +kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
> > +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
> > +kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
> > +kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
> > +kms_invalid_mode@int-max-clock,Fail
> > +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
> > +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-A,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-B,Fail
> > +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
> > +kms_plane@plane-panning-bottom-right,Fail
> > +kms_plane@plane-panning-top-left,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-128,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-256,Fail
> > +kms_plane_cursor@pipe-B-overlay-size-64,Fail
> > +kms_plane_cursor@pipe-B-primary-size-128,Fail
> > +kms_plane_cursor@pipe-B-primary-size-256,Fail
> > +kms_plane_cursor@pipe-B-primary-size-64,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-128,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-256,Fail
> > +kms_plane_cursor@pipe-B-viewport-size-64,Fail
> > +kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
> > +kms_prime@basic-crc,Fail
> > +kms_rmfb@close-fd,Fail
> > +kms_universal_plane@universal-plane-pipe-B-functional,Fail
> > +kms_vblank@pipe-A-query-forked,Fail
> > +kms_vblank@pipe-A-query-idle,Fail
> > +kms_vblank@pipe-A-wait-busy,Fail
> > +kms_vblank@pipe-A-wait-forked,Fail
> > +kms_vblank@pipe-A-wait-forked-busy,Fail
> > +kms_vblank@pipe-B-query-forked,Fail
> > +kms_vblank@pipe-B-query-idle,Fail
> > +kms_vblank@pipe-B-wait-busy,Fail
> > +kms_vblank@pipe-B-wait-forked,Fail
> > +kms_vblank@pipe-B-wait-forked-busy,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> > new file mode 100644
> > index 000000000000..b4c37a4315e4
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
> > @@ -0,0 +1,25 @@
> > +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
> > +kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
> > +kms_cursor_crc@pipe-B-cursor-32x10-offscreen
> > +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
> > +kms_cursor_crc@pipe-B-cursor-32x32-offscreen
> > +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
> > +kms_cursor_crc@pipe-B-cursor-64x64-offscreen
> > +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
> > +kms_cursor_edge_walk@pipe-A-64x64-right-edge
> > +kms_cursor_edge_walk@pipe-B-64x64-left-edge
> > +kms_flip@flip-vs-expired-vblank
> > +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
> > +kms_plane_multiple@atomic-pipe-A-tiling-none
> > +kms_plane@plane-position-hole
> > +kms_sequence@get-forked
> > +kms_sequence@get-forked-busy
> > +kms_setmode@basic
> > +kms_vblank@pipe-A-accuracy-idle
> > +kms_vblank@pipe-A-query-busy
> > +kms_vblank@pipe-A-query-forked-busy
> > +kms_vblank@pipe-A-wait-idle
> > +kms_vblank@pipe-B-accuracy-idle
> > +kms_vblank@pipe-B-query-busy
> > +kms_vblank@pipe-B-query-forked-busy
> > +kms_vblank@pipe-B-wait-idle
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> > new file mode 100644
> > index 000000000000..10c3d81a919a
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
> > @@ -0,0 +1,5 @@
> > +# Suspend to RAM seems to be broken on this machine
> > +.*suspend.*
> > +
> > +# Too unstable, machine ends up hanging after lots of Oopses
> > +kms_cursor_legacy.*
> > diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> > new file mode 100644
> > index 000000000000..2178bdd064dd
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
> > @@ -0,0 +1,38 @@
> > +kms_addfb_basic@addfb25-bad-modifier,Fail
> > +kms_addfb_basic@bad-pitch-65536,Fail
> > +kms_addfb_basic@bo-too-small,Fail
> > +kms_addfb_basic@size-max,Fail
> > +kms_addfb_basic@too-high,Fail
> > +kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
> > +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
> > +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
> > +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
> > +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
> > +kms_invalid_mode@int-max-clock,Fail
> > +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
> > +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
> > +kms_plane_scaling@planes-upscale-20x20,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
> > +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
> > +kms_plane_scaling@upscale-with-modifier-20x20,Fail
> > +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
> > +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
> > +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
> > +kms_plane_scaling@upscale-with-rotation-20x20,Fail
> > +kms_vblank@crtc-id,Fail
> > +kms_vblank@invalid,Fail
> > +kms_vblank@pipe-A-accuracy-idle,Fail
> > +kms_vblank@pipe-A-query-busy,Fail
> > +kms_vblank@pipe-A-query-forked,Fail
> > +kms_vblank@pipe-A-query-forked-busy,Fail
> > +kms_vblank@pipe-A-query-idle,Fail
> > +kms_vblank@pipe-A-ts-continuation-idle,Fail
> > +kms_vblank@pipe-A-ts-continuation-modeset,Fail
> > +kms_vblank@pipe-A-ts-continuation-suspend,Fail
> > +kms_vblank@pipe-A-wait-busy,Fail
> > +kms_vblank@pipe-A-wait-forked,Fail
> > +kms_vblank@pipe-A-wait-forked-busy,Fail
> > +kms_vblank@pipe-A-wait-idle,Fail
> > \ No newline at end of file
> > diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
> > new file mode 100644
> > index 000000000000..e69de29bb2d1
> > diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> > new file mode 100644
> > index 000000000000..78be18174012
> > --- /dev/null
> > +++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
> > @@ -0,0 +1,6 @@
> > +# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
> > +# When run in a particular order with other tests
> > +kms_cursor_legacy.*
> > +
> > +# Job just hangs without any output
> > +kms_flip@flip-vs-suspend.*
> > \ No newline at end of file
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v5] drm: Add initial ci/ subdirectory
  2022-08-10 18:24     ` Rodrigo Siqueira Jordao
  2022-08-10 22:08       ` Rob Clark
@ 2022-08-11 10:10       ` Tomeu Vizoso
  1 sibling, 0 replies; 13+ messages in thread
From: Tomeu Vizoso @ 2022-08-11 10:10 UTC (permalink / raw)
  To: Rodrigo Siqueira Jordao, Nicholas Choi, Sun peng Li, Chiu,
	Harrison, Wentland, Harry
  Cc: linux-doc, Neil Armstrong, linux-kernel, dri-devel,
	linux-rockchip, linux-mediatek, linux-amlogic, kernel,
	linux-arm-kernel, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger, Rob Clark

On 8/10/22 8:24 PM, Rodrigo Siqueira Jordao wrote:
> Hi Tomeu,
> 
> First of all, nice patch! I just saw it, and I have some basic questions 
> (I don't understand many of these CI details). I also CC some CI folks 
> from the display team at AMD.

Thanks for the feedback!

> On 2022-07-26 14:16, Tomeu Vizoso wrote:
>> And use it to store expectations about what the DRM drivers are
>> supposed to pass in the IGT test suite.
>>
>> Also include a configuration file that points to the out-of-tree CI
>> scripts.
>>
>> By storing the test expectations along the code we can make sure both
>> stay in sync with each other, and so we can know when a code change
>> breaks those expectations.
>>
>> This will allow all contributors to drm to reuse the infrastructure
>> already in gitlab.freedesktop.org to test the driver on several
>> generations of the hardware.
>>
>> v2:
>>    - Fix names of result expectation files to match SoC
>>    - Don't execute tests that are going to skip on all boards
>>
>> v3:
>>    - Remove tracking of dmesg output during test execution
>>
>> v4:
>>    - Move up to drivers/gpu/drm
>>    - Add support for a bunch of other drivers
>>    - Explain how to incorporate fixes for CI from a
>>      ${TARGET_BRANCH}-external-fixes branch
>>    - Remove tests that pass from expected results file, to reduce the
>>      size of in-tree files
>>    - Add docs about how to deal with outages in automated testing labs
>>    - Specify the exact SHA of the CI scripts to be used
>>
>> v5:
>>    - Remove unneeded skips from Meson expectations file
>>    - Use a more advanced runner that detects flakes automatically
>>    - Use a more succint format for the expectations
>>    - Run many more tests (and use sharding to finish in time)
>>    - Use skip lists to avoid hanging machines
>>    - Add some build testing
>>    - Build IGT in each pipeline for faster uprevs
>>    - List failures in the GitLab UI
>>
>> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
>> Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
>> ---
>>   Documentation/gpu/automated_testing.rst       | 84 ++++++++++++++++++
>>   drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 13 +++
>>   drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
>>   drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
>>   drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
>>   drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
>>   drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
>>   drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
>>   drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
>>   drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
>>   drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
>>   drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
>>   drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
>>   drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
>>   drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
>>   drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
>>   drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
>>   drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
>>   drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
>>   drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
>>   drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
>>   drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
>>   drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
>>   drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
>>   drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
>>   drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
>>   drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
>>   drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
>>   drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
>>   drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
>>   drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
>>   drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
>>   drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
>>   drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
>>   drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
>>   drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
>>   drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
>>   drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
>>   drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  5 ++
>>   drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
>>   drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
>>   drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 25 ++++++
>>   drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
>>   drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
>>   drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
>>   drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
>>   46 files changed, 882 insertions(+)
>>   create mode 100644 Documentation/gpu/automated_testing.rst
>>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>>   create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
>>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
>>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
>>   create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
>>
>> diff --git a/Documentation/gpu/automated_testing.rst 
>> b/Documentation/gpu/automated_testing.rst
>> new file mode 100644
>> index 000000000000..61003ecc7b6e
>> --- /dev/null
>> +++ b/Documentation/gpu/automated_testing.rst
>> @@ -0,0 +1,84 @@
>> +.. SPDX-License-Identifier: GPL-2.0+
>> +
>> +=========================================
>> +Automated testing of the DRM subsystem
>> +=========================================
>> +
>> +
>> +Introduction
>> +============
>> +
>> +Making sure that changes to the core or drivers don't introduce 
>> regressions can be very time consuming when lots of different hardware 
>> configurations need to be tested. Moreover, it isn't practical for 
>> each person interested in this testing to have to acquire and maintain 
>> what can be a considerable amount of hardware.
>> +
>> +Also, it is desirable for developers to check for regressions in 
>> their code by themselves, instead of relying on the maintainers 
>> finding them and then reporting back.
>> +
>> +There are facilities in gitlab.freedesktop.org to automatically test 
>> Mesa that can be used as well for testing the DRM subsystem. This 
>> document explains how people interested in testing it can use this 
>> shared infrastructure to save quite some time and effort.
>> +
>> +
>> +Relevant files
>> +==============
>> +
>> +drivers/gpu/drm/ci/gitlab-ci.yml
>> +--------------------------------
>> +
>> +Specifies the specific version of the scripts to be used. GitLab CI 
>> will use the values defined in this file to fetch the right scripts.
>> +
>> +
>> +drivers/gpu/drm/ci/${DRIVER_NAME}.testlist
>> +------------------------------------------
>> +
>> +Specifies the tests that the current code is expected to be able to 
>> reliably run for ${DRIVER_NAME}. These tests are expected to not hang 
>> the DUT (Device Under Testing) when running on the revision they 
>> belong to, and to give consistent results.
> 
> The IGT repo has testlists per vendor (e.g., 
> https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/tree/master/tests/intel-ci). 
> What is the advantage of keeping a similar IGT list here? Should we 
> maintain that list in both projects?

Yes, I think that would be better, but the ones currently there were too 
small or too big to be useful here.

Note that the documentation here is erroneous, the test lists in this 
iteration are in the drm-ci repo, not in the kernel. Will fix this when 
I send my rebased patches.

>> +drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt
>> +----------------------------------------
>> +
>> +Specifies the expected results of running this specific kernel 
>> revision on a given hardware configuration.
>> +
>> +
>> +How to enable automated testing on your tree
>> +============================================
>> +
>> +1. Create a Linux tree in https://gitlab.freedesktop.org/ if you 
>> don't have one yet
>> +
>> +2. In your kernel repo's configuration (eg. 
>> https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change 
>> the CI/CD configuration file from .gitlab-ci.yml to 
>> drivers/gpu/drm/ci/gitlab-ci.yml.
>> +
>> +3. Next time you push to this repository, you will see a CI pipeline 
>> being created (eg. 
>> https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
>> +
>> +4. The various jobs will be run and when the pipeline is finished, 
>> all jobs should be green unless a regression has been found.
>> +
>> +
>> +How to update test expectations
>> +===============================
>> +
>> +If your changes to the code fix any tests, you will have to update 
>> one or more of the files in 
>> drivers/gpu/drm/ci/${DRIVER_NAME}_*_results.txt, for each of the test 
>> platforms affected by the change.
>> +
>> +If you have run a pipeline and it failed because of a mismatched test 
>> result, you will have been offered a patch at the end of the run that 
>> you can apply after reviewing it.
>> +
>> +
>> +How to expand coverage
>> +======================
>> +
>> +If your code changes makes it possible to run more tests (by solving 
>> reliability issues, for example), more tests can be added to this 
>> list, and then the expected results updated with the patch file that 
>> will be printed at the end of each job.
>> +
>> +If there is a need for updating the version of IGT being used (maybe 
>> you have added more tests to it), follow the instructions at 
>> https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
> 
> Usually, we run our IGT test on the latest code from the IGT repo. Is it 
> possible to configure this behavior?

If this was the default behavior, it would create a lot of trouble 
because pipelines for the same code would sometimes pass or fail based 
on whatever had been pushed to the IGT repo. It is really important for 
this to work well for everybody that the pipeline is kept green at all 
times for checked-in code.

But it is already very easy to uprev IGT, it just requires updating a 
commit hash in the drm-ci repo.

>> +How to test your changes to the scripts
>> +==========================================
>> +
>> +For testing changes to the scripts in the drm-ci repo, change the 
>> DRM_CI_PROJECT_PATH variable in drivers/gpu/drm/ci/gitlab-ci.yml to 
>> match your fork of the project (eg. tomeu/drm-ci). This fork needs to 
>> be in https://gitlab.freedesktop.org/.
>> +
>> +
>> +How to incorporate external fixes in your testing
>> +=================================================
>> +
>> +Often, regressions in other trees will prevent testing changes local 
>> to the tree under test. These fixes will be automatically merged in 
>> during the build jobs from a branch in the target tree that is named 
>> as ${TARGET_BRANCH}-external-fixes.
>> +
>> +If the pipeline is not in a merge request and a branch with the same 
>> name exists in the local tree, commits from that branch will be merged 
>> in as well.
>> +
>> +
>> +How to deal with automated testing labs that may be down
>> +========================================================
>> +
>> +If a hardware farm is down and thus causing pipelines to fail that 
>> would otherwise pass, one can disable all jobs that would be submitted 
>> to that farm by editing the file at 
>> https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml. 
>>
> 
> 
> This part is the one that I really want to understand. How can we plug 
> our farm? Do you have a guideline? I want to know if we can run a 
> prototype.

We are reusing the device farms that were added to 
gitlab.freedesktop.org for testing Mesa.

You should be able to plug your own lab for use in your personal branch 
without requiring anything from anybody else. You just need an account 
in gitlab.freedesktop.org.

Currently, there are two modes through which DUTs are made available to 
Gitlab CI: baremetal and LAVA. The latter should better scale to several 
dozens of devices but has a higher initial installation effort. It would 
allow this farm to be used in other CIs such as kernelci.org, though.

There is some documentation about this in Mesa:

* https://docs.mesa3d.org/ci/bare-metal.html
* https://docs.mesa3d.org/ci/LAVA.html

Ask in #freedesktop in OFTC if you need any help.

>> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt 
>> b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
>> new file mode 100644
>> index 000000000000..e721568b6ec3
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
>> @@ -0,0 +1,13 @@
>> +kms_addfb_basic@bad-pitch-65536,Fail
>> +kms_addfb_basic@bo-too-small,Fail
>> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
>> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
>> +kms_hdr@bpc-switch,Fail
>> +kms_hdr@bpc-switch-dpms,Fail
>> +kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
>> +kms_rotation_crc@primary-rotation-180,Fail
>> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt 
>> b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
>> new file mode 100644
>> index 000000000000..a2cbc339838c
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
>> @@ -0,0 +1,20 @@
>> +kms_addfb_basic@too-high
>> +kms_atomic_transition@plane-all-modeset-transition-internal-panels
>> +kms_atomic_transition@plane-all-transition
>> +kms_atomic_transition@plane-use-after-nonblocking-unbind
>> +kms_bw@linear-tiling-1-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-3840x2160p
>> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
>> +kms_plane_multiple@atomic-pipe-B-tiling-none
>> +kms_plane@pixel-format
>> +kms_plane_scaling@downscale-with-rotation-factor-0-5
>> +kms_universal_plane@disable-primary-vs-flip-pipe-A
> 
> We have multiple test lists for different ASICs, and we are working to 
> have one unified list. I guess we just need to that list here?
> 
> Maybe Nicholas/Herrison can provide a complete list here.

Maybe add it to the IGT repo and from there it will be used in this CI 
once it can run reliably enough?

Cheers,

Tomeu

> Thanks
> Siqueira
> 
> 
>> diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt 
>> b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
>> new file mode 100644
>> index 000000000000..e2c538a0f954
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml 
>> b/drivers/gpu/drm/ci/gitlab-ci.yml
>> new file mode 100644
>> index 000000000000..8898bbe0f180
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/gitlab-ci.yml
>> @@ -0,0 +1,13 @@
>> +variables:
>> +  # Change this to use your fork of drm-ci
>> +  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
>> +  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 
>> bccff240ca042820484096f0782e8f060bd83bca
>> +
>> +  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
>> +  TARGET_BRANCH: drm-next
>> +
>> +include:
>> +  - project: *drm-ci-project-path
>> +    ref: *drm-ci-commit-sha
>> +    file:
>> +      - '.gitlab-ci.yml'
>> diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-amly-flakes.txt
>> new file mode 100644
>> index 000000000000..67d78f10b671
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
>> @@ -0,0 +1,32 @@
>> +kms_bw@linear-tiling-2-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-2560x1440p
>> +kms_bw@linear-tiling-2-displays-3840x2160p
>> +kms_bw@linear-tiling-3-displays-1920x1080p
>> +kms_bw@linear-tiling-3-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-3840x2160p
>> +kms_bw@linear-tiling-4-displays-1920x1080p
>> +kms_bw@linear-tiling-4-displays-2560x1440p
>> +kms_bw@linear-tiling-4-displays-3840x2160p
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
>> +kms_plane_alpha_blend@pipe-A-alpha-basic
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-B-alpha-basic
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-C-alpha-basic
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
>> +kms_sysfs_edid_timing
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt 
>> b/drivers/gpu/drm/ci/i915-amly-skips.txt
>> new file mode 100644
>> index 000000000000..e2c538a0f954
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt 
>> b/drivers/gpu/drm/ci/i915-apl-fails.txt
>> new file mode 100644
>> index 000000000000..a077f29d5cba
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
>> @@ -0,0 +1,29 @@
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail 
>>
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
>> +kms_sysfs_edid_timing,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-apl-flakes.txt
>> new file mode 100644
>> index 000000000000..618bbe6bb793
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
>> @@ -0,0 +1 @@
>> +kms_frontbuffer_tracking@fbc-tiling-linear
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt 
>> b/drivers/gpu/drm/ci/i915-apl-skips.txt
>> new file mode 100644
>> index 000000000000..e2c538a0f954
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-cml-flakes.txt
>> new file mode 100644
>> index 000000000000..a214419c6dc5
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
>> @@ -0,0 +1,36 @@
>> +kms_bw@linear-tiling-2-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-2560x1440p
>> +kms_bw@linear-tiling-2-displays-3840x2160p
>> +kms_bw@linear-tiling-3-displays-1920x1080p
>> +kms_bw@linear-tiling-3-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-3840x2160p
>> +kms_bw@linear-tiling-4-displays-1920x1080p
>> +kms_bw@linear-tiling-4-displays-2560x1440p
>> +kms_bw@linear-tiling-4-displays-3840x2160p
>> +kms_flip@flip-vs-suspend
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
>> +kms_hdr@bpc-switch-suspend
>> +kms_plane_alpha_blend@pipe-A-alpha-basic
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-B-alpha-basic
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-C-alpha-basic
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
>> +kms_psr2_su@page_flip-NV12
>> +kms_psr2_su@page_flip-P010
>> +kms_setmode@basic
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-glk-flakes.txt
>> new file mode 100644
>> index 000000000000..aedb3d2ef885
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
>> @@ -0,0 +1,40 @@
>> +kms_bw@linear-tiling-1-displays-3840x2160p
>> +kms_bw@linear-tiling-2-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-2560x1440p
>> +kms_bw@linear-tiling-2-displays-3840x2160p
>> +kms_bw@linear-tiling-3-displays-1920x1080p
>> +kms_bw@linear-tiling-3-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-3840x2160p
>> +kms_bw@linear-tiling-4-displays-1920x1080p
>> +kms_bw@linear-tiling-4-displays-2560x1440p
>> +kms_bw@linear-tiling-4-displays-3840x2160p
>> +kms_flip@blocking-wf_vblank
>> +kms_flip@wf_vblank-ts-check
>> +kms_flip@wf_vblank-ts-check-interruptible
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
>> +kms_frontbuffer_tracking@fbc-tiling-linear
>> +kms_plane_alpha_blend@pipe-A-alpha-basic
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-B-alpha-basic
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-C-alpha-basic
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max
>> +kms_rotation_crc@multiplane-rotation
>> +kms_rotation_crc@multiplane-rotation-cropping-bottom
>> +kms_rotation_crc@multiplane-rotation-cropping-top
>> +kms_setmode@basic
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt 
>> b/drivers/gpu/drm/ci/i915-glk-skips.txt
>> new file mode 100644
>> index 000000000000..e2c538a0f954
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt 
>> b/drivers/gpu/drm/ci/i915-kbl-fails.txt
>> new file mode 100644
>> index 000000000000..f81f065e27e8
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
>> @@ -0,0 +1,8 @@
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
>> new file mode 100644
>> index 000000000000..f82017654cf5
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
>> @@ -0,0 +1,24 @@
>> +kms_bw@linear-tiling-2-displays-1920x1080p
>> +kms_bw@linear-tiling-2-displays-3840x2160p
>> +kms_bw@linear-tiling-3-displays-1920x1080p
>> +kms_bw@linear-tiling-3-displays-2560x1440p
>> +kms_bw@linear-tiling-3-displays-3840x2160p
>> +kms_bw@linear-tiling-4-displays-1920x1080p
>> +kms_bw@linear-tiling-4-displays-3840x2160p
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
>> +kms_plane_alpha_blend@pipe-A-alpha-basic
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-B-alpha-basic
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max
>> +kms_plane_alpha_blend@pipe-C-alpha-basic
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
>> +kms_sysfs_edid_timing
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt 
>> b/drivers/gpu/drm/ci/i915-kbl-skips.txt
>> new file mode 100644
>> index 000000000000..e2c538a0f954
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt 
>> b/drivers/gpu/drm/ci/i915-tgl-fails.txt
>> new file mode 100644
>> index 000000000000..270644ce220c
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
>> @@ -0,0 +1,19 @@
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-5-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-5-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-5-displays-3840x2160p,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail 
>>
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
>> +kms_rotation_crc@bad-pixel-format,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
>> new file mode 100644
>> index 000000000000..15fba77ff19d
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
>> @@ -0,0 +1,6 @@
>> +kms_draw_crc@draw-method-rgb565-blt-untiled
>> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
>> +kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
>> +kms_flip@blocking-absolute-wf_vblank
>> +kms_flip@bo-too-big-interruptible
>> +kms_flip@flip-vs-rmfb-interruptible
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt 
>> b/drivers/gpu/drm/ci/i915-tgl-skips.txt
>> new file mode 100644
>> index 000000000000..14f33c7ba03b
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
>> @@ -0,0 +1,8 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> +
>> +# GPU hangs, then the whole machine
>> +gem_eio.*
>> +
>> +# Whole machine hangs
>> +kms_flip@absolute-wf_vblank@a-edp1
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt 
>> b/drivers/gpu/drm/ci/i915-whl-fails.txt
>> new file mode 100644
>> index 000000000000..dbfedfa5d63f
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
>> @@ -0,0 +1,30 @@
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-4-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-4-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-4-displays-3840x2160p,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail 
>>
>> +kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
>> +kms_frontbuffer_tracking@fbc-tiling-linear,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
>> +kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
>> +kms_sysfs_edid_timing,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt 
>> b/drivers/gpu/drm/ci/i915-whl-flakes.txt
>> new file mode 100644
>> index 000000000000..6bcac5b84fbb
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
>> @@ -0,0 +1 @@
>> +kms_flip@flip-vs-suspend
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt 
>> b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
>> new file mode 100644
>> index 000000000000..7f74fb9afc99
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
>> @@ -0,0 +1,29 @@
>> +kms_3d,Fail
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
>> +kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
>> +kms_force_connector_basic@force-connector-state,Fail
>> +kms_force_connector_basic@force-edid,Fail
>> +kms_force_connector_basic@force-load-detect,Fail
>> +kms_force_connector_basic@prune-stale-modes,Fail
>> +kms_invalid_mode@int-max-clock,Fail
>> +kms_plane_scaling@planes-upscale-20x20,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
>> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
>> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
>> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
>> +kms_properties@get_properties-sanity-atomic,Fail
>> +kms_properties@plane-properties-atomic,Fail
>> +kms_properties@plane-properties-legacy,Fail
>> +kms_rmfb@close-fd,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt 
>> b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
>> new file mode 100644
>> index 000000000000..6ff81d00e84e
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
>> @@ -0,0 +1,10 @@
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
>> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
>> +kms_rmfb@close-fd,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt 
>> b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
>> new file mode 100644
>> index 000000000000..208890b79eb0
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
>> @@ -0,0 +1,14 @@
>> +core_setmaster_vs_auth
>> +kms_bw@linear-tiling-1-displays-1920x1080p
>> +kms_bw@linear-tiling-1-displays-3840x2160p
>> +kms_bw@linear-tiling-3-displays-1920x1080p
>> +kms_cursor_legacy@cursor-vs-flip-atomic
>> +kms_plane_scaling@invalid-num-scalers
>> +kms_plane_scaling@planes-upscale-20x20
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
>> +kms_plane_scaling@upscale-with-modifier-20x20
>> +kms_plane_scaling@upscale-with-pixel-format-20x20
>> +kms_prop_blob@invalid-set-prop-any
>> +kms_properties@get_properties-sanity-atomic
>> +kms_properties@plane-properties-atomic
>> +kms_properties@plane-properties-legacy
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt 
>> b/drivers/gpu/drm/ci/meson-g12b-fails.txt
>> new file mode 100644
>> index 000000000000..345e1426b3a3
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
>> @@ -0,0 +1,5 @@
>> +kms_3d,Fail
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_properties@connector-properties-atomic,Fail
>> +kms_properties@get_properties-sanity-atomic,Fail
>> +kms_properties@get_properties-sanity-non-atomic,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt 
>> b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
>> new file mode 100644
>> index 000000000000..b63329d06767
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
>> @@ -0,0 +1,4 @@
>> +kms_force_connector_basic@force-connector-state
>> +kms_force_connector_basic@force-edid
>> +kms_force_connector_basic@force-load-detect
>> +kms_force_connector_basic@prune-stale-modes
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt 
>> b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
>> new file mode 100644
>> index 000000000000..16c2dfab3f16
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
>> @@ -0,0 +1,15 @@
>> +kms_3d,Fail
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_cursor_legacy@all-pipes-forked-move,Fail
>> +kms_cursor_legacy@all-pipes-torture-bo,Fail
>> +kms_cursor_legacy@all-pipes-forked-bo,Fail
>> +kms_cursor_legacy@all-pipes-single-bo,Fail
>> +kms_cursor_legacy@all-pipes-single-move,Fail
>> +kms_cursor_legacy@all-pipes-torture-move,Fail
>> +kms_cursor_legacy@pipe-A-forked-bo,Fail
>> +kms_cursor_legacy@pipe-A-forked-move,Fail
>> +kms_cursor_legacy@pipe-A-single-bo,Fail
>> +kms_cursor_legacy@pipe-A-single-move,Fail
>> +kms_cursor_legacy@pipe-A-torture-bo,Fail
>> +kms_cursor_legacy@pipe-A-torture-move,Fail
>> +kms_hdmi_inject@inject-4k,Fail
>> diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt 
>> b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
>> new file mode 100644
>> index 000000000000..b63329d06767
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
>> @@ -0,0 +1,4 @@
>> +kms_force_connector_basic@force-connector-state
>> +kms_force_connector_basic@force-edid
>> +kms_force_connector_basic@force-load-detect
>> +kms_force_connector_basic@prune-stale-modes
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt 
>> b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
>> new file mode 100644
>> index 000000000000..88a1fc0a3b0d
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
>> @@ -0,0 +1,2 @@
>> +kms_3d,Fail
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt 
>> b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
>> new file mode 100644
>> index 000000000000..b63329d06767
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
>> @@ -0,0 +1,4 @@
>> +kms_force_connector_basic@force-connector-state
>> +kms_force_connector_basic@force-edid
>> +kms_force_connector_basic@force-load-detect
>> +kms_force_connector_basic@prune-stale-modes
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt 
>> b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
>> new file mode 100644
>> index 000000000000..cd49c8ce2059
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Whole machine hangs
>> +kms_cursor_legacy@all-pipes-torture-move
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt 
>> b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
>> new file mode 100644
>> index 000000000000..0b159b6469c8
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
>> @@ -0,0 +1,22 @@
>> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
>> +kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
>> +kms_plane@pixel-format,Fail
>> +kms_plane@pixel-format-source-clamping,Fail
>> +kms_plane@plane-position-covered,Fail
>> +kms_plane@plane-position-hole,Fail
>> +kms_plane@plane-position-hole-dpms,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
>> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
>> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
>> +kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
>> +kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
>> +kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
>> +kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
>> +kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
>> +kms_rmfb@close-fd,Fail
>> +kms_rotation_crc@primary-rotation-180,Fail
>> +kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
>> +kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
>> diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt 
>> b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
>> new file mode 100644
>> index 000000000000..6022edace190
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
>> @@ -0,0 +1,14 @@
>> +# Test ends up reading CRC from frame before cursor update
>> +# sometimes.. tbd if this is a kernel CRC bug or a test
>> +# bug
>> +kms_cursor_crc@pipe-.*
>> +
>> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
>> +kms_plane_multiple@atomic-pipe-A-tiling-none
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt 
>> b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
>> new file mode 100644
>> index 000000000000..40916ddff81f
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
>> @@ -0,0 +1,18 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> +
>> +# Test incorrectly assumes that CTM support implies gamma/degamma
>> +# LUT support.  None of the subtests handle the case of only having
>> +# CTM support
>> +kms_color.*
>> +
>> +# 4k@60 is not supported on this hw, but driver doesn't handle it
>> +# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
>> +kms_bw@linear-tiling-.*-displays-3840x2160p
>> +
>> +# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
>> +kms_bw@linear-tiling-2.*
>> +kms_bw@linear-tiling-3.*
>> +kms_bw@linear-tiling-4.*
>> +kms_bw@linear-tiling-5.*
>> +kms_bw@linear-tiling-6.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt 
>> b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
>> new file mode 100644
>> index 000000000000..8133927a44df
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
>> @@ -0,0 +1,44 @@
>> +kms_color@pipe-A-ctm-0-25,Fail
>> +kms_color@pipe-A-ctm-0-5,Fail
>> +kms_color@pipe-A-ctm-0-75,Fail
>> +kms_color@pipe-A-ctm-blue-to-red,Fail
>> +kms_color@pipe-A-ctm-green-to-red,Fail
>> +kms_color@pipe-A-ctm-max,Fail
>> +kms_color@pipe-A-ctm-negative,Fail
>> +kms_color@pipe-A-ctm-red-to-blue,Fail
>> +kms_color@pipe-A-legacy-gamma,Fail
>> +kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
>> +kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
>> +kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
>> +kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
>> +kms_cursor_legacy@cursor-vs-flip-atomic,Fail
>> +kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
>> +kms_cursor_legacy@cursor-vs-flip-legacy,Fail
>> +kms_cursor_legacy@cursor-vs-flip-toggle,Fail
>> +kms_cursor_legacy@flip-vs-cursor-atomic,Fail
>> +kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
>> +kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
>> +kms_cursor_legacy@flip-vs-cursor-legacy,Fail
>> +kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
>> +kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
>> +kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
>> +kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
>> +kms_plane@pixel-format,Fail
>> +kms_plane@pixel-format-source-clamping,Fail
>> +kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
>> +kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
>> +kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
>> +kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
>> +kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
>> +kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
>> +kms_plane_scaling@invalid-num-scalers,Fail
>> +kms_plane_scaling@planes-downscale-factor-0-25,Fail
>> +kms_plane_scaling@scaler-with-clipping-clamping,Timeout
>> +kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
>> +kms_rmfb@close-fd,Fail
>> diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt 
>> b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
>> new file mode 100644
>> index 000000000000..da4406b74828
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
>> @@ -0,0 +1,33 @@
>> +# Test ends up reading CRC from frame before cursor update
>> +# sometimes.. tbd if this is a kernel CRC bug or a test
>> +# bug
>> +kms_cursor_crc@pipe-A-cursor-alpha-opaque
>> +kms_cursor_crc@pipe-A-cursor-dpms
>> +kms_cursor_crc@pipe-A-cursor-size-change
>> +kms_cursor_legacy@flip-vs-cursor-toggle
>> +kms_cursor_legacy@pipe-A-forked-bo
>> +kms_cursor_legacy@pipe-A-forked-move
>> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen
>> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen
>> +kms_cursor_crc@pipe-A-cursor-32x10-random
>> +kms_cursor_crc@pipe-A-cursor-32x10-sliding
>> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen
>> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen
>> +kms_cursor_crc@pipe-A-cursor-32x32-random
>> +kms_cursor_crc@pipe-A-cursor-32x32-sliding
>> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
>> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen
>> +kms_cursor_crc@pipe-A-cursor-64x21-random
>> +kms_cursor_crc@pipe-A-cursor-64x21-sliding
>> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen
>> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen
>> +kms_cursor_crc@pipe-A-cursor-64x64-random
>> +kms_cursor_crc@pipe-A-cursor-64x64-sliding
>> +kms_cursor_crc@pipe-A-cursor-dpms
>> +kms_cursor_crc@pipe-A-cursor-suspend
>> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-left-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
>> +
>> +kms_flip@dpms-vs-vblank-race-interruptible
>> diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt 
>> b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
>> new file mode 100644
>> index 000000000000..42675f1c6d76
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
>> @@ -0,0 +1,2 @@
>> +# Hangs machine
>> +kms_bw.*
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
>> new file mode 100644
>> index 000000000000..e2a264647a58
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
>> @@ -0,0 +1,75 @@
>> +kms_3d,Fail
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-3-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-3-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-3-displays-3840x2160p,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
>> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
>> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
>> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
>> +kms_force_connector_basic@force-connector-state,Fail
>> +kms_force_connector_basic@force-edid,Fail
>> +kms_force_connector_basic@force-load-detect,Fail
>> +kms_force_connector_basic@prune-stale-modes,Fail
>> +kms_invalid_mode@int-max-clock,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-128,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-256,Fail
>> +kms_plane_cursor@pipe-A-overlay-size-64,Fail
>> +kms_plane_cursor@pipe-A-primary-size-128,Fail
>> +kms_plane_cursor@pipe-A-primary-size-256,Fail
>> +kms_plane_cursor@pipe-A-primary-size-64,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-128,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-256,Fail
>> +kms_plane_cursor@pipe-A-viewport-size-64,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
>> +kms_plane_cursor@pipe-B-primary-size-128,Fail
>> +kms_plane_cursor@pipe-B-primary-size-256,Fail
>> +kms_plane_cursor@pipe-B-primary-size-64,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
>> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
>> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
>> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
>> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
>> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
>> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
>> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
>> +kms_prime@basic-crc,Fail
>> +kms_properties@connector-properties-atomic,Fail
>> +kms_properties@get_properties-sanity-atomic,Fail
>> +kms_properties@get_properties-sanity-non-atomic,Fail
>> +kms_rmfb@close-fd,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
>> new file mode 100644
>> index 000000000000..447fb26cc344
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
>> @@ -0,0 +1,5 @@
>> +kms_cursor_crc@pipe-B-cursor-dpms
>> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-A-64x64-top-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-top-edge
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
>> new file mode 100644
>> index 000000000000..26f9a03fc074
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
>> @@ -0,0 +1,46 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> +
>> +# Too unstable, machine ends up hanging after lots of Oopses
>> +kms_cursor_legacy.*
>> +
>> +# Started hanging the machine on Linux 5.19-rc2:
>> +#
>> +# [IGT] kms_plane_lowres: executing
>> +# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
>> +# [IGT] kms_plane_lowres: exiting, ret=77
>> +# Console: switching to colour frame buffer device 170x48
>> +# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
>> +# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] 
>> commit wait timed out
>> +# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
>> +# 8<--- cut here ---
>> +# Unable to handle kernel paging request at virtual address 7812078e
>> +# [7812078e] *pgd=00000000
>> +# Internal error: Oops: 5 [#1] SMP ARM
>> +# Modules linked in:
>> +# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         
>> 5.19.0-rc2-323596-g00535de92171 #1
>> +# Hardware name: Rockchip (Device Tree)
>> +# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
>> +#  spin_dump from do_raw_spin_lock+0xa4/0xe8
>> +#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
>> +#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
>> +#  drm_crtc_commit_wait from 
>> drm_atomic_helper_wait_for_dependencies+0x44/0x168
>> +#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
>> +#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
>> +#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
>> +#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
>> +#  drm_client_modeset_commit_atomic from 
>> drm_client_modeset_commit_locked+0x60/0x1c8
>> +#  drm_client_modeset_commit_locked from 
>> drm_client_modeset_commit+0x24/0x40
>> +#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
>> +#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
>> +#  drm_client_dev_restore from drm_release+0xf4/0x114
>> +#  drm_release from __fput+0x74/0x240
>> +#  __fput from task_work_run+0x84/0xb4
>> +#  task_work_run from do_exit+0x34c/0xa20
>> +#  do_exit from do_group_exit+0x34/0x98
>> +#  do_group_exit from __wake_up_parent+0x0/0x18
>> +# Code: e595c008 12843d19 03e00000 03093168 (15940508)
>> +# ---[ end trace 0000000000000000 ]---
>> +# note: kms_plane_lowre[482] exited with preempt_count 1
>> +# Fixing recursive fault but reboot is needed!
>> +kms_plane_lowres@pipe-F-tiling-y
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
>> new file mode 100644
>> index 000000000000..0f8afa7e2adc
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
>> @@ -0,0 +1,86 @@
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_color@pipe-A-legacy-gamma,Fail
>> +kms_color@pipe-B-legacy-gamma,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
>> +kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
>> +kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
>> +kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
>> +kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
>> +kms_cursor_crc@pipe-A-cursor-dpms,Fail
>> +kms_cursor_crc@pipe-A-cursor-size-change,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
>> +kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
>> +kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
>> +kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
>> +kms_cursor_crc@pipe-B-cursor-dpms,Fail
>> +kms_cursor_crc@pipe-B-cursor-size-change,Fail
>> +kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
>> +kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
>> +kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
>> +kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
>> +kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
>> +kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
>> +kms_invalid_mode@int-max-clock,Fail
>> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
>> +kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-A,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-B,Fail
>> +kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
>> +kms_plane@plane-panning-bottom-right,Fail
>> +kms_plane@plane-panning-top-left,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-128,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-256,Fail
>> +kms_plane_cursor@pipe-B-overlay-size-64,Fail
>> +kms_plane_cursor@pipe-B-primary-size-128,Fail
>> +kms_plane_cursor@pipe-B-primary-size-256,Fail
>> +kms_plane_cursor@pipe-B-primary-size-64,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-128,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-256,Fail
>> +kms_plane_cursor@pipe-B-viewport-size-64,Fail
>> +kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
>> +kms_prime@basic-crc,Fail
>> +kms_rmfb@close-fd,Fail
>> +kms_universal_plane@universal-plane-pipe-B-functional,Fail
>> +kms_vblank@pipe-A-query-forked,Fail
>> +kms_vblank@pipe-A-query-idle,Fail
>> +kms_vblank@pipe-A-wait-busy,Fail
>> +kms_vblank@pipe-A-wait-forked,Fail
>> +kms_vblank@pipe-A-wait-forked-busy,Fail
>> +kms_vblank@pipe-B-query-forked,Fail
>> +kms_vblank@pipe-B-query-idle,Fail
>> +kms_vblank@pipe-B-wait-busy,Fail
>> +kms_vblank@pipe-B-wait-forked,Fail
>> +kms_vblank@pipe-B-wait-forked-busy,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
>> new file mode 100644
>> index 000000000000..b4c37a4315e4
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
>> @@ -0,0 +1,25 @@
>> +kms_cursor_crc@pipe-A-cursor-64x21-offscreen
>> +kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
>> +kms_cursor_crc@pipe-B-cursor-32x10-offscreen
>> +kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
>> +kms_cursor_crc@pipe-B-cursor-32x32-offscreen
>> +kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
>> +kms_cursor_crc@pipe-B-cursor-64x64-offscreen
>> +kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
>> +kms_cursor_edge_walk@pipe-A-64x64-right-edge
>> +kms_cursor_edge_walk@pipe-B-64x64-left-edge
>> +kms_flip@flip-vs-expired-vblank
>> +kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
>> +kms_plane_multiple@atomic-pipe-A-tiling-none
>> +kms_plane@plane-position-hole
>> +kms_sequence@get-forked
>> +kms_sequence@get-forked-busy
>> +kms_setmode@basic
>> +kms_vblank@pipe-A-accuracy-idle
>> +kms_vblank@pipe-A-query-busy
>> +kms_vblank@pipe-A-query-forked-busy
>> +kms_vblank@pipe-A-wait-idle
>> +kms_vblank@pipe-B-accuracy-idle
>> +kms_vblank@pipe-B-query-busy
>> +kms_vblank@pipe-B-query-forked-busy
>> +kms_vblank@pipe-B-wait-idle
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt 
>> b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
>> new file mode 100644
>> index 000000000000..10c3d81a919a
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
>> @@ -0,0 +1,5 @@
>> +# Suspend to RAM seems to be broken on this machine
>> +.*suspend.*
>> +
>> +# Too unstable, machine ends up hanging after lots of Oopses
>> +kms_cursor_legacy.*
>> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt 
>> b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
>> new file mode 100644
>> index 000000000000..2178bdd064dd
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
>> @@ -0,0 +1,38 @@
>> +kms_addfb_basic@addfb25-bad-modifier,Fail
>> +kms_addfb_basic@bad-pitch-65536,Fail
>> +kms_addfb_basic@bo-too-small,Fail
>> +kms_addfb_basic@size-max,Fail
>> +kms_addfb_basic@too-high,Fail
>> +kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
>> +kms_bw@linear-tiling-1-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-1-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-1-displays-3840x2160p,Fail
>> +kms_bw@linear-tiling-2-displays-1920x1080p,Fail
>> +kms_bw@linear-tiling-2-displays-2560x1440p,Fail
>> +kms_bw@linear-tiling-2-displays-3840x2160p,Fail
>> +kms_invalid_mode@int-max-clock,Fail
>> +kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
>> +kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
>> +kms_plane_scaling@planes-upscale-20x20,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
>> +kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
>> +kms_plane_scaling@upscale-with-modifier-20x20,Fail
>> +kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
>> +kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
>> +kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
>> +kms_plane_scaling@upscale-with-rotation-20x20,Fail
>> +kms_vblank@crtc-id,Fail
>> +kms_vblank@invalid,Fail
>> +kms_vblank@pipe-A-accuracy-idle,Fail
>> +kms_vblank@pipe-A-query-busy,Fail
>> +kms_vblank@pipe-A-query-forked,Fail
>> +kms_vblank@pipe-A-query-forked-busy,Fail
>> +kms_vblank@pipe-A-query-idle,Fail
>> +kms_vblank@pipe-A-ts-continuation-idle,Fail
>> +kms_vblank@pipe-A-ts-continuation-modeset,Fail
>> +kms_vblank@pipe-A-ts-continuation-suspend,Fail
>> +kms_vblank@pipe-A-wait-busy,Fail
>> +kms_vblank@pipe-A-wait-forked,Fail
>> +kms_vblank@pipe-A-wait-forked-busy,Fail
>> +kms_vblank@pipe-A-wait-idle,Fail
>> \ No newline at end of file
>> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt 
>> b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
>> new file mode 100644
>> index 000000000000..e69de29bb2d1
>> diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt 
>> b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
>> new file mode 100644
>> index 000000000000..78be18174012
>> --- /dev/null
>> +++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
>> @@ -0,0 +1,6 @@
>> +# Hits a "refcount_t: underflow; use-after-free" in 
>> virtio_gpu_fence_event_process
>> +# When run in a particular order with other tests
>> +kms_cursor_legacy.*
>> +
>> +# Job just hangs without any output
>> +kms_flip@flip-vs-suspend.*
>> \ No newline at end of file
> 
> 

-- 
Tomeu Vizoso
Consultant Principal Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, United 
Kingdom
Registered in England & Wales, no. 5513718

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH v6] drm: Add initial ci/ subdirectory
  2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
  2022-07-29 15:43     ` Rob Clark
  2022-08-10 18:24     ` Rodrigo Siqueira Jordao
@ 2022-08-16 15:37     ` Tomeu Vizoso
  2022-08-29  9:31       ` [PATCH v7] " Tomeu Vizoso
  2 siblings, 1 reply; 13+ messages in thread
From: Tomeu Vizoso @ 2022-08-16 15:37 UTC (permalink / raw)
  To: David Airlie, Jonathan Corbet, Carlo Caione, Kevin Hilman,
	Heiko Stuebner, Matthias Brugger, Rob Clark
  Cc: dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Tomeu Vizoso, Neil Armstrong, Rob Clark

And use it to store expectations about what the DRM drivers are
supposed to pass in the IGT test suite.

Also include a configuration file that points to the out-of-tree CI
scripts.

By storing the test expectations along the code we can make sure both
stay in sync with each other, and so we can know when a code change
breaks those expectations.

This will allow all contributors to drm to reuse the infrastructure
already in gitlab.freedesktop.org to test the driver on several
generations of the hardware.

v2:
  - Fix names of result expectation files to match SoC
  - Don't execute tests that are going to skip on all boards

v3:
  - Remove tracking of dmesg output during test execution

v4:
  - Move up to drivers/gpu/drm
  - Add support for a bunch of other drivers
  - Explain how to incorporate fixes for CI from a
    ${TARGET_BRANCH}-external-fixes branch
  - Remove tests that pass from expected results file, to reduce the
    size of in-tree files
  - Add docs about how to deal with outages in automated testing labs
  - Specify the exact SHA of the CI scripts to be used

v5:
  - Remove unneeded skips from Meson expectations file
  - Use a more advanced runner that detects flakes automatically
  - Use a more succint format for the expectations
  - Run many more tests (and use sharding to finish in time)
  - Use skip lists to avoid hanging machines
  - Add some build testing
  - Build IGT in each pipeline for faster uprevs
  - List failures in the GitLab UI

v6:
  - Rebase on top of latest drm-next
  - Lower priority of LAVA jobs to not impact Mesa CI as much
  - Update docs

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>
---
 Documentation/gpu/automated_testing.rst       | 86 +++++++++++++++++++
 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 17 ++++
 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 20 +++++
 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
 drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
 drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
 drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
 drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
 drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
 drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
 drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
 drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
 drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
 drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
 drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
 drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
 drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
 drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
 drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
 drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
 drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
 drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
 drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 22 +++++
 drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
 drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 18 ++++
 drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
 drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 33 +++++++
 drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 75 ++++++++++++++++
 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  9 ++
 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 86 +++++++++++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 26 ++++++
 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
 46 files changed, 893 insertions(+)
 create mode 100644 Documentation/gpu/automated_testing.rst
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
 create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
 create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt

diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
new file mode 100644
index 000000000000..defee4f5a470
--- /dev/null
+++ b/Documentation/gpu/automated_testing.rst
@@ -0,0 +1,86 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+=========================================
+Automated testing of the DRM subsystem
+=========================================
+
+
+Introduction
+============
+
+Making sure that changes to the core or drivers don't introduce regressions can be very time-consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
+
+Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers to find them and then reporting back.
+
+There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
+
+
+Relevant files
+==============
+
+drivers/gpu/drm/ci/gitlab-ci.yml
+--------------------------------
+
+Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_fails.txt
+----------------------------------------------------------
+
+Lists the known failures for a given driver on a specific hardware revision.
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_flakes.txt
+-----------------------------------------------------------
+
+Lists the tests that for a given driver on a specific hardware revision are known to behave unreliably. These tests won't cause a job to fail regardless of the result. They will still be run.
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_skips.txt
+-----------------------------------------------------------
+
+Lists the tests that won't be run for a given driver on a specific hardware revision. These are usually tests that interfere with the running of the test list due to hanging the machine, causing OOM, taking too long, etc.
+
+
+How to enable automated testing on your tree
+============================================
+
+1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
+
+2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
+
+3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
+
+4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
+
+
+How to update test expectations
+===============================
+
+If your changes to the code fix any tests, you will have to remove one or more lines from one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_fails.txt, for each of the test platforms affected by the change.
+
+
+How to expand coverage
+======================
+
+If your code changes make it possible to run more tests (by solving reliability issues, for example), you can remove tests from the flakes and/or skips lists, and then the expected results if there are any known failures.
+
+If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
+
+
+How to test your changes to the scripts
+=======================================
+
+For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH and DRM_CI_COMMIT_SHA variables in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
+
+
+How to incorporate external fixes in your testing
+=================================================
+
+Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
+
+If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
+
+
+How to deal with automated testing labs that may be down
+========================================================
+
+If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
new file mode 100644
index 000000000000..39bbf5f22828
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
@@ -0,0 +1,17 @@
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_hdr@bpc-switch,Fail
+kms_hdr@bpc-switch-dpms,Fail
+kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
+kms_async_flips@alternate-sync-async-flip,Fail
+kms_async_flips@async-flip-with-page-flip-events,Fail
+kms_async_flips@crc,Fail
+kms_async_flips@invalid-async-flip,Fail
+kms_async_flips@test-cursor,Fail
+kms_async_flips@test-time-stamp,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
new file mode 100644
index 000000000000..a2cbc339838c
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
@@ -0,0 +1,20 @@
+kms_addfb_basic@too-high
+kms_atomic_transition@plane-all-modeset-transition-internal-panels
+kms_atomic_transition@plane-all-transition
+kms_atomic_transition@plane-use-after-nonblocking-unbind
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-B-tiling-none
+kms_plane@pixel-format
+kms_plane_scaling@downscale-with-rotation-factor-0-5
+kms_universal_plane@disable-primary-vs-flip-pipe-A
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
new file mode 100644
index 000000000000..e36e457d8017
--- /dev/null
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -0,0 +1,13 @@
+variables:
+  # Change this to use your fork of drm-ci
+  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
+  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 00aff4ab1923ac6fda7c984d74f9e67b2d627727
+
+  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
+  TARGET_BRANCH: drm-next
+
+include:
+  - project: *drm-ci-project-path
+    ref: *drm-ci-commit-sha
+    file:
+      - '.gitlab-ci.yml'
diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
new file mode 100644
index 000000000000..67d78f10b671
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
@@ -0,0 +1,32 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
new file mode 100644
index 000000000000..a077f29d5cba
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
@@ -0,0 +1,29 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
new file mode 100644
index 000000000000..618bbe6bb793
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
@@ -0,0 +1 @@
+kms_frontbuffer_tracking@fbc-tiling-linear
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
new file mode 100644
index 000000000000..a214419c6dc5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
@@ -0,0 +1,36 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@flip-vs-suspend
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_hdr@bpc-switch-suspend
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_psr2_su@page_flip-NV12
+kms_psr2_su@page_flip-P010
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
new file mode 100644
index 000000000000..aedb3d2ef885
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
@@ -0,0 +1,40 @@
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@blocking-wf_vblank
+kms_flip@wf_vblank-ts-check
+kms_flip@wf_vblank-ts-check-interruptible
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_frontbuffer_tracking@fbc-tiling-linear
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_rotation_crc@multiplane-rotation
+kms_rotation_crc@multiplane-rotation-cropping-bottom
+kms_rotation_crc@multiplane-rotation-cropping-top
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
new file mode 100644
index 000000000000..f81f065e27e8
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
@@ -0,0 +1,8 @@
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
new file mode 100644
index 000000000000..f82017654cf5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
@@ -0,0 +1,24 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
new file mode 100644
index 000000000000..270644ce220c
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
@@ -0,0 +1,19 @@
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_bw@linear-tiling-5-displays-1920x1080p,Fail
+kms_bw@linear-tiling-5-displays-2560x1440p,Fail
+kms_bw@linear-tiling-5-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_rotation_crc@bad-pixel-format,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
new file mode 100644
index 000000000000..15fba77ff19d
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
@@ -0,0 +1,6 @@
+kms_draw_crc@draw-method-rgb565-blt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
+kms_flip@blocking-absolute-wf_vblank
+kms_flip@bo-too-big-interruptible
+kms_flip@flip-vs-rmfb-interruptible
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
new file mode 100644
index 000000000000..14f33c7ba03b
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
@@ -0,0 +1,8 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# GPU hangs, then the whole machine
+gem_eio.*
+
+# Whole machine hangs
+kms_flip@absolute-wf_vblank@a-edp1
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
new file mode 100644
index 000000000000..dbfedfa5d63f
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
@@ -0,0 +1,30 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_frontbuffer_tracking@fbc-tiling-linear,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
new file mode 100644
index 000000000000..6bcac5b84fbb
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
@@ -0,0 +1 @@
+kms_flip@flip-vs-suspend
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
new file mode 100644
index 000000000000..7f74fb9afc99
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
@@ -0,0 +1,29 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@plane-properties-atomic,Fail
+kms_properties@plane-properties-legacy,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
new file mode 100644
index 000000000000..6ff81d00e84e
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
@@ -0,0 +1,10 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
new file mode 100644
index 000000000000..208890b79eb0
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
@@ -0,0 +1,14 @@
+core_setmaster_vs_auth
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_cursor_legacy@cursor-vs-flip-atomic
+kms_plane_scaling@invalid-num-scalers
+kms_plane_scaling@planes-upscale-20x20
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
+kms_plane_scaling@upscale-with-modifier-20x20
+kms_plane_scaling@upscale-with-pixel-format-20x20
+kms_prop_blob@invalid-set-prop-any
+kms_properties@get_properties-sanity-atomic
+kms_properties@plane-properties-atomic
+kms_properties@plane-properties-legacy
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
new file mode 100644
index 000000000000..345e1426b3a3
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
@@ -0,0 +1,5 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
new file mode 100644
index 000000000000..16c2dfab3f16
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
@@ -0,0 +1,15 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_cursor_legacy@all-pipes-forked-move,Fail
+kms_cursor_legacy@all-pipes-torture-bo,Fail
+kms_cursor_legacy@all-pipes-forked-bo,Fail
+kms_cursor_legacy@all-pipes-single-bo,Fail
+kms_cursor_legacy@all-pipes-single-move,Fail
+kms_cursor_legacy@all-pipes-torture-move,Fail
+kms_cursor_legacy@pipe-A-forked-bo,Fail
+kms_cursor_legacy@pipe-A-forked-move,Fail
+kms_cursor_legacy@pipe-A-single-bo,Fail
+kms_cursor_legacy@pipe-A-single-move,Fail
+kms_cursor_legacy@pipe-A-torture-bo,Fail
+kms_cursor_legacy@pipe-A-torture-move,Fail
+kms_hdmi_inject@inject-4k,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
new file mode 100644
index 000000000000..88a1fc0a3b0d
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
@@ -0,0 +1,2 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
new file mode 100644
index 000000000000..cd49c8ce2059
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
@@ -0,0 +1,2 @@
+# Whole machine hangs
+kms_cursor_legacy@all-pipes-torture-move
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
new file mode 100644
index 000000000000..0b159b6469c8
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
@@ -0,0 +1,22 @@
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane@plane-position-covered,Fail
+kms_plane@plane-position-hole,Fail
+kms_plane@plane-position-hole-dpms,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
+kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
+kms_plane_multiple@atomic-pipe-A-tiling-none,Crash
+kms_rmfb@close-fd,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
new file mode 100644
index 000000000000..6022edace190
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
@@ -0,0 +1,14 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@pipe-.*
+
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-A-tiling-none
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
new file mode 100644
index 000000000000..40916ddff81f
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
@@ -0,0 +1,18 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Test incorrectly assumes that CTM support implies gamma/degamma
+# LUT support.  None of the subtests handle the case of only having
+# CTM support
+kms_color.*
+
+# 4k@60 is not supported on this hw, but driver doesn't handle it
+# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
+kms_bw@linear-tiling-.*-displays-3840x2160p
+
+# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
+kms_bw@linear-tiling-2.*
+kms_bw@linear-tiling-3.*
+kms_bw@linear-tiling-4.*
+kms_bw@linear-tiling-5.*
+kms_bw@linear-tiling-6.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
new file mode 100644
index 000000000000..8133927a44df
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
@@ -0,0 +1,44 @@
+kms_color@pipe-A-ctm-0-25,Fail
+kms_color@pipe-A-ctm-0-5,Fail
+kms_color@pipe-A-ctm-0-75,Fail
+kms_color@pipe-A-ctm-blue-to-red,Fail
+kms_color@pipe-A-ctm-green-to-red,Fail
+kms_color@pipe-A-ctm-max,Fail
+kms_color@pipe-A-ctm-negative,Fail
+kms_color@pipe-A-ctm-red-to-blue,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
+kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
+kms_cursor_legacy@cursor-vs-flip-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@flip-vs-cursor-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
+kms_cursor_legacy@flip-vs-cursor-legacy,Fail
+kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
+kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
+kms_plane_scaling@invalid-num-scalers,Fail
+kms_plane_scaling@planes-downscale-factor-0-25,Fail
+kms_plane_scaling@scaler-with-clipping-clamping,Timeout
+kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
+kms_rmfb@close-fd,Fail
diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
new file mode 100644
index 000000000000..da4406b74828
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
@@ -0,0 +1,33 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@pipe-A-cursor-alpha-opaque
+kms_cursor_crc@pipe-A-cursor-dpms
+kms_cursor_crc@pipe-A-cursor-size-change
+kms_cursor_legacy@flip-vs-cursor-toggle
+kms_cursor_legacy@pipe-A-forked-bo
+kms_cursor_legacy@pipe-A-forked-move
+kms_cursor_crc@pipe-A-cursor-32x10-offscreen
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen
+kms_cursor_crc@pipe-A-cursor-32x10-random
+kms_cursor_crc@pipe-A-cursor-32x10-sliding
+kms_cursor_crc@pipe-A-cursor-32x32-offscreen
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen
+kms_cursor_crc@pipe-A-cursor-32x32-random
+kms_cursor_crc@pipe-A-cursor-32x32-sliding
+kms_cursor_crc@pipe-A-cursor-64x21-offscreen
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen
+kms_cursor_crc@pipe-A-cursor-64x21-random
+kms_cursor_crc@pipe-A-cursor-64x21-sliding
+kms_cursor_crc@pipe-A-cursor-64x64-offscreen
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen
+kms_cursor_crc@pipe-A-cursor-64x64-random
+kms_cursor_crc@pipe-A-cursor-64x64-sliding
+kms_cursor_crc@pipe-A-cursor-dpms
+kms_cursor_crc@pipe-A-cursor-suspend
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+
+kms_flip@dpms-vs-vblank-race-interruptible
diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
new file mode 100644
index 000000000000..42675f1c6d76
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
@@ -0,0 +1,2 @@
+# Hangs machine
+kms_bw.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
new file mode 100644
index 000000000000..e2a264647a58
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
@@ -0,0 +1,75 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-A-cursor-dpms,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-primary-size-128,Fail
+kms_plane_cursor@pipe-A-primary-size-256,Fail
+kms_plane_cursor@pipe-A-primary-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_prime@basic-crc,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
new file mode 100644
index 000000000000..ab4effd9dc43
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
@@ -0,0 +1,9 @@
+kms_cursor_crc@pipe-A-cursor-32x10-offscreen
+kms_cursor_crc@pipe-B-cursor-dpms
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_pipe_crc_basic@disable-crc-after-crtc-pipe-B
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
new file mode 100644
index 000000000000..26f9a03fc074
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
@@ -0,0 +1,46 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
+
+# Started hanging the machine on Linux 5.19-rc2:
+#
+# [IGT] kms_plane_lowres: executing
+# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
+# [IGT] kms_plane_lowres: exiting, ret=77
+# Console: switching to colour frame buffer device 170x48
+# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
+# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
+# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
+# 8<--- cut here ---
+# Unable to handle kernel paging request at virtual address 7812078e
+# [7812078e] *pgd=00000000
+# Internal error: Oops: 5 [#1] SMP ARM
+# Modules linked in:
+# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
+# Hardware name: Rockchip (Device Tree)
+# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
+#  spin_dump from do_raw_spin_lock+0xa4/0xe8
+#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
+#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
+#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
+#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
+#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
+#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
+#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
+#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
+#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
+#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
+#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
+#  drm_client_dev_restore from drm_release+0xf4/0x114
+#  drm_release from __fput+0x74/0x240
+#  __fput from task_work_run+0x84/0xb4
+#  task_work_run from do_exit+0x34c/0xa20
+#  do_exit from do_group_exit+0x34/0x98
+#  do_group_exit from __wake_up_parent+0x0/0x18
+# Code: e595c008 12843d19 03e00000 03093168 (15940508) 
+# ---[ end trace 0000000000000000 ]---
+# note: kms_plane_lowre[482] exited with preempt_count 1
+# Fixing recursive fault but reboot is needed!
+kms_plane_lowres@pipe-F-tiling-y
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
new file mode 100644
index 000000000000..0f8afa7e2adc
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
@@ -0,0 +1,86 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_color@pipe-B-legacy-gamma,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x21-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-offscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-rapid-movement,Fail
+kms_cursor_crc@pipe-A-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-A-cursor-alpha-transparent,Fail
+kms_cursor_crc@pipe-A-cursor-dpms,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-32x10-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-offscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-random,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-32x32-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x21-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-offscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-onscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-random,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-sliding,Fail
+kms_cursor_crc@pipe-B-cursor-alpha-opaque,Fail
+kms_cursor_crc@pipe-B-cursor-dpms,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane@plane-panning-bottom-right,Fail
+kms_plane@plane-panning-top-left,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
+kms_prime@basic-crc,Fail
+kms_rmfb@close-fd,Fail
+kms_universal_plane@universal-plane-pipe-B-functional,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-B-query-forked,Fail
+kms_vblank@pipe-B-query-idle,Fail
+kms_vblank@pipe-B-wait-busy,Fail
+kms_vblank@pipe-B-wait-forked,Fail
+kms_vblank@pipe-B-wait-forked-busy,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
new file mode 100644
index 000000000000..de1258fc030b
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
@@ -0,0 +1,26 @@
+kms_cursor_crc@pipe-A-cursor-64x21-offscreen
+kms_cursor_crc@pipe-A-cursor-64x21-rapid-movement
+kms_cursor_crc@pipe-B-cursor-32x10-offscreen
+kms_cursor_crc@pipe-B-cursor-32x10-rapid-movement
+kms_cursor_crc@pipe-B-cursor-32x32-offscreen
+kms_cursor_crc@pipe-B-cursor-32x32-rapid-movement
+kms_cursor_crc@pipe-B-cursor-64x64-offscreen
+kms_cursor_crc@pipe-B-cursor-64x21-offscreen,Fail
+kms_cursor_crc@pipe-B-cursor-64x64-rapid-movement
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_flip@flip-vs-expired-vblank
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
+kms_plane_multiple@atomic-pipe-A-tiling-none
+kms_plane@plane-position-hole
+kms_sequence@get-forked
+kms_sequence@get-forked-busy
+kms_setmode@basic
+kms_vblank@pipe-A-accuracy-idle
+kms_vblank@pipe-A-query-busy
+kms_vblank@pipe-A-query-forked-busy
+kms_vblank@pipe-A-wait-idle
+kms_vblank@pipe-B-accuracy-idle
+kms_vblank@pipe-B-query-busy
+kms_vblank@pipe-B-query-forked-busy
+kms_vblank@pipe-B-wait-idle
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
new file mode 100644
index 000000000000..10c3d81a919a
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
@@ -0,0 +1,5 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
new file mode 100644
index 000000000000..2178bdd064dd
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
@@ -0,0 +1,38 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_addfb_basic@bad-pitch-65536,Fail
+kms_addfb_basic@bo-too-small,Fail
+kms_addfb_basic@size-max,Fail
+kms_addfb_basic@too-high,Fail
+kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_vblank@crtc-id,Fail
+kms_vblank@invalid,Fail
+kms_vblank@pipe-A-accuracy-idle,Fail
+kms_vblank@pipe-A-query-busy,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-forked-busy,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-ts-continuation-idle,Fail
+kms_vblank@pipe-A-ts-continuation-modeset,Fail
+kms_vblank@pipe-A-ts-continuation-suspend,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-A-wait-idle,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
new file mode 100644
index 000000000000..78be18174012
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
@@ -0,0 +1,6 @@
+# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
+# When run in a particular order with other tests
+kms_cursor_legacy.*
+
+# Job just hangs without any output
+kms_flip@flip-vs-suspend.*
\ No newline at end of file
-- 
2.31.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH v7] drm: Add initial ci/ subdirectory
  2022-08-16 15:37     ` [PATCH v6] " Tomeu Vizoso
@ 2022-08-29  9:31       ` Tomeu Vizoso
       [not found]         ` <20220909141528.5090-1-tomeu.vizoso@collabora.com>
  0 siblings, 1 reply; 13+ messages in thread
From: Tomeu Vizoso @ 2022-08-29  9:31 UTC (permalink / raw)
  To: David Airlie, Jonathan Corbet, Carlo Caione, Kevin Hilman,
	Heiko Stuebner, Matthias Brugger, Rob Clark
  Cc: dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Tomeu Vizoso, Neil Armstrong, Rob Clark

And use it to store expectations about what the DRM drivers are
supposed to pass in the IGT test suite.

Also include a configuration file that points to the out-of-tree CI
scripts.

By storing the test expectations along the code we can make sure both
stay in sync with each other, and so we can know when a code change
breaks those expectations.

This will allow all contributors to drm to reuse the infrastructure
already in gitlab.freedesktop.org to test the driver on several
generations of the hardware.

v2:
  - Fix names of result expectation files to match SoC
  - Don't execute tests that are going to skip on all boards

v3:
  - Remove tracking of dmesg output during test execution

v4:
  - Move up to drivers/gpu/drm
  - Add support for a bunch of other drivers
  - Explain how to incorporate fixes for CI from a
    ${TARGET_BRANCH}-external-fixes branch
  - Remove tests that pass from expected results file, to reduce the
    size of in-tree files
  - Add docs about how to deal with outages in automated testing labs
  - Specify the exact SHA of the CI scripts to be used

v5:
  - Remove unneeded skips from Meson expectations file
  - Use a more advanced runner that detects flakes automatically
  - Use a more succint format for the expectations
  - Run many more tests (and use sharding to finish in time)
  - Use skip lists to avoid hanging machines
  - Add some build testing
  - Build IGT in each pipeline for faster uprevs
  - List failures in the GitLab UI

v6:
  - Rebase on top of latest drm-next
  - Lower priority of LAVA jobs to not impact Mesa CI as much
  - Update docs

v7:
  - Rebase on top of latest drm-next

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Neil Armstrong <narmstrong@baylibre.com>
Reviewed-by: Rob Clark <robdclark@gmail.com>
---
 Documentation/gpu/automated_testing.rst       | 86 +++++++++++++++++++
 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt    | 19 ++++
 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt   | 22 +++++
 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt    |  2 +
 drivers/gpu/drm/ci/gitlab-ci.yml              | 13 +++
 drivers/gpu/drm/ci/i915-amly-flakes.txt       | 32 +++++++
 drivers/gpu/drm/ci/i915-amly-skips.txt        |  2 +
 drivers/gpu/drm/ci/i915-apl-fails.txt         | 29 +++++++
 drivers/gpu/drm/ci/i915-apl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/i915-apl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-cml-flakes.txt        | 36 ++++++++
 drivers/gpu/drm/ci/i915-glk-flakes.txt        | 40 +++++++++
 drivers/gpu/drm/ci/i915-glk-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-kbl-fails.txt         |  8 ++
 drivers/gpu/drm/ci/i915-kbl-flakes.txt        | 24 ++++++
 drivers/gpu/drm/ci/i915-kbl-skips.txt         |  2 +
 drivers/gpu/drm/ci/i915-tgl-fails.txt         | 19 ++++
 drivers/gpu/drm/ci/i915-tgl-flakes.txt        |  6 ++
 drivers/gpu/drm/ci/i915-tgl-skips.txt         |  8 ++
 drivers/gpu/drm/ci/i915-whl-fails.txt         | 30 +++++++
 drivers/gpu/drm/ci/i915-whl-flakes.txt        |  1 +
 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt  | 29 +++++++
 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt  | 10 +++
 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt | 14 +++
 drivers/gpu/drm/ci/meson-g12b-fails.txt       |  5 ++
 drivers/gpu/drm/ci/meson-g12b-flakes.txt      |  4 +
 drivers/gpu/drm/ci/msm-apq8016-fails.txt      | 15 ++++
 drivers/gpu/drm/ci/msm-apq8016-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-fails.txt      |  2 +
 drivers/gpu/drm/ci/msm-apq8096-flakes.txt     |  4 +
 drivers/gpu/drm/ci/msm-apq8096-skips.txt      |  2 +
 drivers/gpu/drm/ci/msm-sc7180-fails.txt       | 21 +++++
 drivers/gpu/drm/ci/msm-sc7180-flakes.txt      | 14 +++
 drivers/gpu/drm/ci/msm-sc7180-skips.txt       | 23 +++++
 drivers/gpu/drm/ci/msm-sdm845-fails.txt       | 44 ++++++++++
 drivers/gpu/drm/ci/msm-sdm845-flakes.txt      | 14 +++
 drivers/gpu/drm/ci/msm-sdm845-skips.txt       |  2 +
 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt  | 47 ++++++++++
 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt |  9 ++
 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt  | 46 ++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt  | 44 ++++++++++
 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt | 19 ++++
 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt  |  5 ++
 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt  | 38 ++++++++
 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt |  0
 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt  |  6 ++
 46 files changed, 805 insertions(+)
 create mode 100644 Documentation/gpu/automated_testing.rst
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
 create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
 create mode 100644 drivers/gpu/drm/ci/i915-amly-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-amly-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-apl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-cml-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-glk-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-kbl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/i915-tgl-skips.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-fails.txt
 create mode 100644 drivers/gpu/drm/ci/i915-whl-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
 create mode 100644 drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-fails.txt
 create mode 100644 drivers/gpu/drm/ci/meson-g12b-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8016-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-apq8096-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sc7180-skips.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-fails.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/msm-sdm845-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
 create mode 100644 drivers/gpu/drm/ci/virtio_gpu-none-skips.txt

diff --git a/Documentation/gpu/automated_testing.rst b/Documentation/gpu/automated_testing.rst
new file mode 100644
index 000000000000..defee4f5a470
--- /dev/null
+++ b/Documentation/gpu/automated_testing.rst
@@ -0,0 +1,86 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+=========================================
+Automated testing of the DRM subsystem
+=========================================
+
+
+Introduction
+============
+
+Making sure that changes to the core or drivers don't introduce regressions can be very time-consuming when lots of different hardware configurations need to be tested. Moreover, it isn't practical for each person interested in this testing to have to acquire and maintain what can be a considerable amount of hardware.
+
+Also, it is desirable for developers to check for regressions in their code by themselves, instead of relying on the maintainers to find them and then reporting back.
+
+There are facilities in gitlab.freedesktop.org to automatically test Mesa that can be used as well for testing the DRM subsystem. This document explains how people interested in testing it can use this shared infrastructure to save quite some time and effort.
+
+
+Relevant files
+==============
+
+drivers/gpu/drm/ci/gitlab-ci.yml
+--------------------------------
+
+Specifies the specific version of the scripts to be used. GitLab CI will use the values defined in this file to fetch the right scripts.
+
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_fails.txt
+----------------------------------------------------------
+
+Lists the known failures for a given driver on a specific hardware revision.
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_flakes.txt
+-----------------------------------------------------------
+
+Lists the tests that for a given driver on a specific hardware revision are known to behave unreliably. These tests won't cause a job to fail regardless of the result. They will still be run.
+
+drivers/gpu/drm/ci/${DRIVER_NAME}_${HW_REVISION}_skips.txt
+-----------------------------------------------------------
+
+Lists the tests that won't be run for a given driver on a specific hardware revision. These are usually tests that interfere with the running of the test list due to hanging the machine, causing OOM, taking too long, etc.
+
+
+How to enable automated testing on your tree
+============================================
+
+1. Create a Linux tree in https://gitlab.freedesktop.org/ if you don't have one yet
+
+2. In your kernel repo's configuration (eg. https://gitlab.freedesktop.org/tomeu/linux/-/settings/ci_cd), change the CI/CD configuration file from .gitlab-ci.yml to drivers/gpu/drm/ci/gitlab-ci.yml.
+
+3. Next time you push to this repository, you will see a CI pipeline being created (eg. https://gitlab.freedesktop.org/tomeu/linux/-/pipelines)
+
+4. The various jobs will be run and when the pipeline is finished, all jobs should be green unless a regression has been found.
+
+
+How to update test expectations
+===============================
+
+If your changes to the code fix any tests, you will have to remove one or more lines from one or more of the files in drivers/gpu/drm/ci/${DRIVER_NAME}_*_fails.txt, for each of the test platforms affected by the change.
+
+
+How to expand coverage
+======================
+
+If your code changes make it possible to run more tests (by solving reliability issues, for example), you can remove tests from the flakes and/or skips lists, and then the expected results if there are any known failures.
+
+If there is a need for updating the version of IGT being used (maybe you have added more tests to it), follow the instructions at https://gitlab.freedesktop.org/gfx-ci/drm-ci/-/tree/main#upreving-igt.
+
+
+How to test your changes to the scripts
+=======================================
+
+For testing changes to the scripts in the drm-ci repo, change the DRM_CI_PROJECT_PATH and DRM_CI_COMMIT_SHA variables in drivers/gpu/drm/ci/gitlab-ci.yml to match your fork of the project (eg. tomeu/drm-ci). This fork needs to be in https://gitlab.freedesktop.org/.
+
+
+How to incorporate external fixes in your testing
+=================================================
+
+Often, regressions in other trees will prevent testing changes local to the tree under test. These fixes will be automatically merged in during the build jobs from a branch in the target tree that is named as ${TARGET_BRANCH}-external-fixes.
+
+If the pipeline is not in a merge request and a branch with the same name exists in the local tree, commits from that branch will be merged in as well.
+
+
+How to deal with automated testing labs that may be down
+========================================================
+
+If a hardware farm is down and thus causing pipelines to fail that would otherwise pass, one can disable all jobs that would be submitted to that farm by editing the file at https://gitlab.freedesktop.org/gfx-ci/lab-status/-/blob/main/lab-status.yml.
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
new file mode 100644
index 000000000000..b596e5bf5348
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-fails.txt
@@ -0,0 +1,19 @@
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_cursor_crc@pipe-A-cursor-size-change,Fail
+kms_cursor_crc@pipe-B-cursor-size-change,Fail
+kms_hdr@bpc-switch,Fail
+kms_hdr@bpc-switch-dpms,Fail
+kms_plane_multiple@atomic-pipe-A-tiling-none,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
+kms_async_flips@alternate-sync-async-flip,Fail
+kms_async_flips@async-flip-with-page-flip-events,Fail
+kms_async_flips@crc,Fail
+kms_async_flips@invalid-async-flip,Fail
+kms_async_flips@test-cursor,Fail
+kms_async_flips@test-time-stamp,Fail
+kms_addfb_basic@bad-pitch-65536,Fail
+kms_addfb_basic@bo-too-small,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
new file mode 100644
index 000000000000..f8efb3fd43e0
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-flakes.txt
@@ -0,0 +1,22 @@
+kms_addfb_basic@too-high
+kms_atomic_transition@plane-all-modeset-transition-internal-panels
+kms_atomic_transition@plane-all-transition
+kms_atomic_transition@plane-use-after-nonblocking-unbind
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_cursor_crc@pipe-A-cursor-alpha-opaque
+kms_cursor_crc@pipe-B-cursor-alpha-opaque
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-B-tiling-none
+kms_plane@pixel-format
+kms_plane_scaling@downscale-with-rotation-factor-0-5
+kms_universal_plane@disable-primary-vs-flip-pipe-A
diff --git a/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/amdgpu-stoney-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/gitlab-ci.yml b/drivers/gpu/drm/ci/gitlab-ci.yml
new file mode 100644
index 000000000000..c31e01d55e0f
--- /dev/null
+++ b/drivers/gpu/drm/ci/gitlab-ci.yml
@@ -0,0 +1,13 @@
+variables:
+  # Change this to use your fork of drm-ci
+  DRM_CI_PROJECT_PATH: &drm-ci-project-path gfx-ci/drm-ci
+  DRM_CI_COMMIT_SHA: &drm-ci-commit-sha 49a26ec13b6c
+
+  UPSTREAM_REPO: git://anongit.freedesktop.org/drm/drm
+  TARGET_BRANCH: drm-next
+
+include:
+  - project: *drm-ci-project-path
+    ref: *drm-ci-commit-sha
+    file:
+      - '.gitlab-ci.yml'
diff --git a/drivers/gpu/drm/ci/i915-amly-flakes.txt b/drivers/gpu/drm/ci/i915-amly-flakes.txt
new file mode 100644
index 000000000000..67d78f10b671
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-flakes.txt
@@ -0,0 +1,32 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-amly-skips.txt b/drivers/gpu/drm/ci/i915-amly-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-amly-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-fails.txt b/drivers/gpu/drm/ci/i915-apl-fails.txt
new file mode 100644
index 000000000000..a077f29d5cba
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-fails.txt
@@ -0,0 +1,29 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-flakes.txt b/drivers/gpu/drm/ci/i915-apl-flakes.txt
new file mode 100644
index 000000000000..618bbe6bb793
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-flakes.txt
@@ -0,0 +1 @@
+kms_frontbuffer_tracking@fbc-tiling-linear
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-apl-skips.txt b/drivers/gpu/drm/ci/i915-apl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-apl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-cml-flakes.txt b/drivers/gpu/drm/ci/i915-cml-flakes.txt
new file mode 100644
index 000000000000..a214419c6dc5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-cml-flakes.txt
@@ -0,0 +1,36 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@flip-vs-suspend
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_hdr@bpc-switch-suspend
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_psr2_su@page_flip-NV12
+kms_psr2_su@page_flip-P010
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-flakes.txt b/drivers/gpu/drm/ci/i915-glk-flakes.txt
new file mode 100644
index 000000000000..aedb3d2ef885
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-flakes.txt
@@ -0,0 +1,40 @@
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-2560x1440p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-2560x1440p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip@blocking-wf_vblank
+kms_flip@wf_vblank-ts-check
+kms_flip@wf_vblank-ts-check-interruptible
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling
+kms_frontbuffer_tracking@fbc-tiling-linear
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-A-constant-alpha-max
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-C-constant-alpha-max
+kms_rotation_crc@multiplane-rotation
+kms_rotation_crc@multiplane-rotation-cropping-bottom
+kms_rotation_crc@multiplane-rotation-cropping-top
+kms_setmode@basic
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-glk-skips.txt b/drivers/gpu/drm/ci/i915-glk-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-glk-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-fails.txt b/drivers/gpu/drm/ci/i915-kbl-fails.txt
new file mode 100644
index 000000000000..f81f065e27e8
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-fails.txt
@@ -0,0 +1,8 @@
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-flakes.txt b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
new file mode 100644
index 000000000000..f82017654cf5
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-flakes.txt
@@ -0,0 +1,24 @@
+kms_bw@linear-tiling-2-displays-1920x1080p
+kms_bw@linear-tiling-2-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_bw@linear-tiling-3-displays-2560x1440p
+kms_bw@linear-tiling-3-displays-3840x2160p
+kms_bw@linear-tiling-4-displays-1920x1080p
+kms_bw@linear-tiling-4-displays-3840x2160p
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling
+kms_plane_alpha_blend@pipe-A-alpha-basic
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-alpha-basic
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb
+kms_plane_alpha_blend@pipe-B-constant-alpha-max
+kms_plane_alpha_blend@pipe-C-alpha-basic
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb
+kms_sysfs_edid_timing
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-kbl-skips.txt b/drivers/gpu/drm/ci/i915-kbl-skips.txt
new file mode 100644
index 000000000000..e2c538a0f954
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-kbl-skips.txt
@@ -0,0 +1,2 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-fails.txt b/drivers/gpu/drm/ci/i915-tgl-fails.txt
new file mode 100644
index 000000000000..270644ce220c
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-fails.txt
@@ -0,0 +1,19 @@
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_bw@linear-tiling-5-displays-1920x1080p,Fail
+kms_bw@linear-tiling-5-displays-2560x1440p,Fail
+kms_bw@linear-tiling-5-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_rotation_crc@bad-pixel-format,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-flakes.txt b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
new file mode 100644
index 000000000000..15fba77ff19d
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-flakes.txt
@@ -0,0 +1,6 @@
+kms_draw_crc@draw-method-rgb565-blt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-untiled
+kms_draw_crc@draw-method-xrgb8888-mmap-gtt-xtiled
+kms_flip@blocking-absolute-wf_vblank
+kms_flip@bo-too-big-interruptible
+kms_flip@flip-vs-rmfb-interruptible
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-tgl-skips.txt b/drivers/gpu/drm/ci/i915-tgl-skips.txt
new file mode 100644
index 000000000000..14f33c7ba03b
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-tgl-skips.txt
@@ -0,0 +1,8 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# GPU hangs, then the whole machine
+gem_eio.*
+
+# Whole machine hangs
+kms_flip@absolute-wf_vblank@a-edp1
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-fails.txt b/drivers/gpu/drm/ci/i915-whl-fails.txt
new file mode 100644
index 000000000000..dbfedfa5d63f
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-fails.txt
@@ -0,0 +1,30 @@
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_bw@linear-tiling-4-displays-1920x1080p,Fail
+kms_bw@linear-tiling-4-displays-2560x1440p,Fail
+kms_bw@linear-tiling-4-displays-3840x2160p,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytile-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-32bpp-ytileccs-to-64bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytile-upscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling,Fail
+kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-upscaling,Fail
+kms_frontbuffer_tracking@fbc-tiling-linear,Fail
+kms_plane_alpha_blend@pipe-A-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-A-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-A-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-C-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-C-alpha-transparent-fb,Fail
+kms_plane_alpha_blend@pipe-C-constant-alpha-max,Fail
+kms_sysfs_edid_timing,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/i915-whl-flakes.txt b/drivers/gpu/drm/ci/i915-whl-flakes.txt
new file mode 100644
index 000000000000..6bcac5b84fbb
--- /dev/null
+++ b/drivers/gpu/drm/ci/i915-whl-flakes.txt
@@ -0,0 +1 @@
+kms_flip@flip-vs-suspend
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
new file mode 100644
index 000000000000..7f74fb9afc99
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8173-fails.txt
@@ -0,0 +1,29 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_color@pipe-B-invalid-gamma-lut-sizes,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@plane-properties-atomic,Fail
+kms_properties@plane-properties-legacy,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
new file mode 100644
index 000000000000..6ff81d00e84e
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-fails.txt
@@ -0,0 +1,10 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_color@pipe-A-invalid-gamma-lut-sizes,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
new file mode 100644
index 000000000000..208890b79eb0
--- /dev/null
+++ b/drivers/gpu/drm/ci/mediatek-mt8183-flakes.txt
@@ -0,0 +1,14 @@
+core_setmaster_vs_auth
+kms_bw@linear-tiling-1-displays-1920x1080p
+kms_bw@linear-tiling-1-displays-3840x2160p
+kms_bw@linear-tiling-3-displays-1920x1080p
+kms_cursor_legacy@cursor-vs-flip-atomic
+kms_plane_scaling@invalid-num-scalers
+kms_plane_scaling@planes-upscale-20x20
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5
+kms_plane_scaling@upscale-with-modifier-20x20
+kms_plane_scaling@upscale-with-pixel-format-20x20
+kms_prop_blob@invalid-set-prop-any
+kms_properties@get_properties-sanity-atomic
+kms_properties@plane-properties-atomic
+kms_properties@plane-properties-legacy
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-fails.txt b/drivers/gpu/drm/ci/meson-g12b-fails.txt
new file mode 100644
index 000000000000..345e1426b3a3
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-fails.txt
@@ -0,0 +1,5 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/meson-g12b-flakes.txt b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/meson-g12b-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8016-fails.txt b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
new file mode 100644
index 000000000000..16c2dfab3f16
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-fails.txt
@@ -0,0 +1,15 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_cursor_legacy@all-pipes-forked-move,Fail
+kms_cursor_legacy@all-pipes-torture-bo,Fail
+kms_cursor_legacy@all-pipes-forked-bo,Fail
+kms_cursor_legacy@all-pipes-single-bo,Fail
+kms_cursor_legacy@all-pipes-single-move,Fail
+kms_cursor_legacy@all-pipes-torture-move,Fail
+kms_cursor_legacy@pipe-A-forked-bo,Fail
+kms_cursor_legacy@pipe-A-forked-move,Fail
+kms_cursor_legacy@pipe-A-single-bo,Fail
+kms_cursor_legacy@pipe-A-single-move,Fail
+kms_cursor_legacy@pipe-A-torture-bo,Fail
+kms_cursor_legacy@pipe-A-torture-move,Fail
+kms_hdmi_inject@inject-4k,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8016-flakes.txt b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8016-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-fails.txt b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
new file mode 100644
index 000000000000..88a1fc0a3b0d
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-fails.txt
@@ -0,0 +1,2 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
diff --git a/drivers/gpu/drm/ci/msm-apq8096-flakes.txt b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
new file mode 100644
index 000000000000..b63329d06767
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-flakes.txt
@@ -0,0 +1,4 @@
+kms_force_connector_basic@force-connector-state
+kms_force_connector_basic@force-edid
+kms_force_connector_basic@force-load-detect
+kms_force_connector_basic@prune-stale-modes
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-apq8096-skips.txt b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
new file mode 100644
index 000000000000..cd49c8ce2059
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-apq8096-skips.txt
@@ -0,0 +1,2 @@
+# Whole machine hangs
+kms_cursor_legacy@all-pipes-torture-move
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/msm-sc7180-fails.txt b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
new file mode 100644
index 000000000000..cfa25fe97498
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-fails.txt
@@ -0,0 +1,21 @@
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@cursorA-vs-flipA-atomic-transitions,Crash
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane@plane-position-covered,Fail
+kms_plane@plane-position-hole,Fail
+kms_plane@plane-position-hole-dpms,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_alpha_blend@pipe-B-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-B-alpha-basic,Fail
+kms_plane_alpha_blend@pipe-B-alpha-opaque-fb,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-max,Fail
+kms_plane_alpha_blend@pipe-B-constant-alpha-mid,Fail
+kms_plane_alpha_blend@pipe-B-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-B-coverage-vs-premult-vs-constant,Fail
+kms_rmfb@close-fd,Fail
+kms_rotation_crc@primary-rotation-180,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-B,Fail
+kms_universal_plane@disable-primary-vs-flip-pipe-b,Fail
diff --git a/drivers/gpu/drm/ci/msm-sc7180-flakes.txt b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
new file mode 100644
index 000000000000..49790ece3c13
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-flakes.txt
@@ -0,0 +1,14 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@.*
+
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_plane_multiple@atomic-pipe-A-tiling-none
diff --git a/drivers/gpu/drm/ci/msm-sc7180-skips.txt b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
new file mode 100644
index 000000000000..410e0eeb3161
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sc7180-skips.txt
@@ -0,0 +1,23 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Test incorrectly assumes that CTM support implies gamma/degamma
+# LUT support.  None of the subtests handle the case of only having
+# CTM support
+kms_color.*
+
+# 4k@60 is not supported on this hw, but driver doesn't handle it
+# too gracefully.. https://gitlab.freedesktop.org/drm/msm/-/issues/15
+kms_bw@linear-tiling-.*-displays-3840x2160p
+
+# Until igt fix lands: https://patchwork.freedesktop.org/patch/493175/
+kms_bw@linear-tiling-2.*
+kms_bw@linear-tiling-3.*
+kms_bw@linear-tiling-4.*
+kms_bw@linear-tiling-5.*
+kms_bw@linear-tiling-6.*
+
+# igt fix posted: https://patchwork.freedesktop.org/patch/499926/
+# failure mode is flakey due to randomization but fails frequently
+# enough to be detected as a Crash or occasionally UnexpectedPass.
+kms_plane_multiple@atomic-pipe-A-tiling-none
diff --git a/drivers/gpu/drm/ci/msm-sdm845-fails.txt b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
new file mode 100644
index 000000000000..8133927a44df
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-fails.txt
@@ -0,0 +1,44 @@
+kms_color@pipe-A-ctm-0-25,Fail
+kms_color@pipe-A-ctm-0-5,Fail
+kms_color@pipe-A-ctm-0-75,Fail
+kms_color@pipe-A-ctm-blue-to-red,Fail
+kms_color@pipe-A-ctm-green-to-red,Fail
+kms_color@pipe-A-ctm-max,Fail
+kms_color@pipe-A-ctm-negative,Fail
+kms_color@pipe-A-ctm-red-to-blue,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_cursor_legacy@basic-flip-after-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-after-cursor-legacy,Fail
+kms_cursor_legacy@basic-flip-before-cursor-atomic,Fail
+kms_cursor_legacy@basic-flip-before-cursor-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic,Fail
+kms_cursor_legacy@cursor-vs-flip-atomic-transitions,Fail
+kms_cursor_legacy@cursor-vs-flip-legacy,Fail
+kms_cursor_legacy@cursor-vs-flip-toggle,Fail
+kms_cursor_legacy@flip-vs-cursor-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-atomic,Fail
+kms_cursor_legacy@flip-vs-cursor-crc-legacy,Fail
+kms_cursor_legacy@flip-vs-cursor-legacy,Fail
+kms_cursor_legacy@short-flip-after-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-after-cursor-toggle,Fail
+kms_cursor_legacy@short-flip-before-cursor-atomic-transitions,Fail
+kms_cursor_legacy@short-flip-before-cursor-toggle,Fail
+kms_plane@pixel-format,Fail
+kms_plane@pixel-format-source-clamping,Fail
+kms_plane_alpha_blend@pipe-A-alpha-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-7efc,Fail
+kms_plane_alpha_blend@pipe-A-coverage-vs-premult-vs-constant,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-pixel-format-factor-0-25,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-5,Timeout
+kms_plane_scaling@downscale-with-pixel-format-factor-0-75,Timeout
+kms_plane_scaling@invalid-num-scalers,Fail
+kms_plane_scaling@planes-downscale-factor-0-25,Fail
+kms_plane_scaling@scaler-with-clipping-clamping,Timeout
+kms_plane_scaling@scaler-with-pixel-format-unity-scaling,Timeout
+kms_rmfb@close-fd,Fail
diff --git a/drivers/gpu/drm/ci/msm-sdm845-flakes.txt b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
new file mode 100644
index 000000000000..def5865f92a5
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-flakes.txt
@@ -0,0 +1,14 @@
+# Test ends up reading CRC from frame before cursor update
+# sometimes.. tbd if this is a kernel CRC bug or a test
+# bug
+kms_cursor_crc@.*
+
+kms_cursor_legacy@flip-vs-cursor-toggle
+kms_cursor_legacy@pipe-A-forked-bo
+kms_cursor_legacy@pipe-A-forked-move
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+
+kms_flip@dpms-vs-vblank-race-interruptible
diff --git a/drivers/gpu/drm/ci/msm-sdm845-skips.txt b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
new file mode 100644
index 000000000000..42675f1c6d76
--- /dev/null
+++ b/drivers/gpu/drm/ci/msm-sdm845-skips.txt
@@ -0,0 +1,2 @@
+# Hangs machine
+kms_bw.*
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
new file mode 100644
index 000000000000..06d4a587ef24
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-fails.txt
@@ -0,0 +1,47 @@
+kms_3d,Fail
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_bw@linear-tiling-3-displays-1920x1080p,Fail
+kms_bw@linear-tiling-3-displays-2560x1440p,Fail
+kms_bw@linear-tiling-3-displays-3840x2160p,Fail
+kms_force_connector_basic@force-connector-state,Fail
+kms_force_connector_basic@force-edid,Fail
+kms_force_connector_basic@force-load-detect,Fail
+kms_force_connector_basic@prune-stale-modes,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane_cursor@pipe-A-overlay-size-128,Fail
+kms_plane_cursor@pipe-A-overlay-size-256,Fail
+kms_plane_cursor@pipe-A-overlay-size-64,Fail
+kms_plane_cursor@pipe-A-primary-size-128,Fail
+kms_plane_cursor@pipe-A-primary-size-256,Fail
+kms_plane_cursor@pipe-A-primary-size-64,Fail
+kms_plane_cursor@pipe-A-viewport-size-128,Fail
+kms_plane_cursor@pipe-A-viewport-size-256,Fail
+kms_plane_cursor@pipe-A-viewport-size-64,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_prime@basic-crc,Fail
+kms_properties@connector-properties-atomic,Fail
+kms_properties@get_properties-sanity-atomic,Fail
+kms_properties@get_properties-sanity-non-atomic,Fail
+kms_rmfb@close-fd,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
new file mode 100644
index 000000000000..d562589eaf16
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-flakes.txt
@@ -0,0 +1,9 @@
+kms_cursor_crc@.*
+
+kms_cursor_edge_walk@pipe-A-64x64-left-edge
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-A-64x64-top-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_cursor_edge_walk@pipe-B-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-top-edge
+kms_pipe_crc_basic@disable-crc-after-crtc-pipe-B
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
new file mode 100644
index 000000000000..26f9a03fc074
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3288-skips.txt
@@ -0,0 +1,46 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
+
+# Started hanging the machine on Linux 5.19-rc2:
+#
+# [IGT] kms_plane_lowres: executing
+# [IGT] kms_plane_lowres: starting subtest pipe-F-tiling-y
+# [IGT] kms_plane_lowres: exiting, ret=77
+# Console: switching to colour frame buffer device 170x48
+# rockchip-drm display-subsystem: [drm] *ERROR* flip_done timed out
+# rockchip-drm display-subsystem: [drm] *ERROR* [CRTC:35:crtc-0] commit wait timed out
+# BUG: spinlock bad magic on CPU#3, kms_plane_lowre/482
+# 8<--- cut here ---
+# Unable to handle kernel paging request at virtual address 7812078e
+# [7812078e] *pgd=00000000
+# Internal error: Oops: 5 [#1] SMP ARM
+# Modules linked in:
+# CPU: 3 PID: 482 Comm: kms_plane_lowre Tainted: G        W         5.19.0-rc2-323596-g00535de92171 #1
+# Hardware name: Rockchip (Device Tree)
+# Process kms_plane_lowre (pid: 482, stack limit = 0x1193ac2b)
+#  spin_dump from do_raw_spin_lock+0xa4/0xe8
+#  do_raw_spin_lock from wait_for_completion_timeout+0x2c/0x120
+#  wait_for_completion_timeout from drm_crtc_commit_wait+0x18/0x7c
+#  drm_crtc_commit_wait from drm_atomic_helper_wait_for_dependencies+0x44/0x168
+#  drm_atomic_helper_wait_for_dependencies from commit_tail+0x34/0x180
+#  commit_tail from drm_atomic_helper_commit+0x164/0x18c
+#  drm_atomic_helper_commit from drm_atomic_commit+0xac/0xe4
+#  drm_atomic_commit from drm_client_modeset_commit_atomic+0x23c/0x284
+#  drm_client_modeset_commit_atomic from drm_client_modeset_commit_locked+0x60/0x1c8
+#  drm_client_modeset_commit_locked from drm_client_modeset_commit+0x24/0x40
+#  drm_client_modeset_commit from drm_fbdev_client_restore+0x58/0x94
+#  drm_fbdev_client_restore from drm_client_dev_restore+0x70/0xbc
+#  drm_client_dev_restore from drm_release+0xf4/0x114
+#  drm_release from __fput+0x74/0x240
+#  __fput from task_work_run+0x84/0xb4
+#  task_work_run from do_exit+0x34c/0xa20
+#  do_exit from do_group_exit+0x34/0x98
+#  do_group_exit from __wake_up_parent+0x0/0x18
+# Code: e595c008 12843d19 03e00000 03093168 (15940508) 
+# ---[ end trace 0000000000000000 ]---
+# note: kms_plane_lowre[482] exited with preempt_count 1
+# Fixing recursive fault but reboot is needed!
+kms_plane_lowres@pipe-F-tiling-y
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
new file mode 100644
index 000000000000..7a99e41268a9
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-fails.txt
@@ -0,0 +1,44 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_color@pipe-A-legacy-gamma,Fail
+kms_color@pipe-B-legacy-gamma,Fail
+kms_cursor_edge_walk@pipe-A-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-left-edge,Fail
+kms_cursor_edge_walk@pipe-A-64x64-top-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-bottom-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-right-edge,Fail
+kms_cursor_edge_walk@pipe-B-64x64-top-edge,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B,Fail
+kms_pipe_crc_basic@nonblocking-crc-pipe-B-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-A,Fail
+kms_pipe_crc_basic@read-crc-pipe-A-frame-sequence,Fail
+kms_pipe_crc_basic@read-crc-pipe-B,Fail
+kms_pipe_crc_basic@read-crc-pipe-B-frame-sequence,Fail
+kms_plane@plane-panning-bottom-right,Fail
+kms_plane@plane-panning-top-left,Fail
+kms_plane_cursor@pipe-B-overlay-size-128,Fail
+kms_plane_cursor@pipe-B-overlay-size-256,Fail
+kms_plane_cursor@pipe-B-overlay-size-64,Fail
+kms_plane_cursor@pipe-B-primary-size-128,Fail
+kms_plane_cursor@pipe-B-primary-size-256,Fail
+kms_plane_cursor@pipe-B-primary-size-64,Fail
+kms_plane_cursor@pipe-B-viewport-size-128,Fail
+kms_plane_cursor@pipe-B-viewport-size-256,Fail
+kms_plane_cursor@pipe-B-viewport-size-64,Fail
+kms_plane_multiple@atomic-pipe-B-tiling-none,Fail
+kms_prime@basic-crc,Fail
+kms_rmfb@close-fd,Fail
+kms_universal_plane@universal-plane-pipe-B-functional,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-B-query-forked,Fail
+kms_vblank@pipe-B-query-idle,Fail
+kms_vblank@pipe-B-wait-busy,Fail
+kms_vblank@pipe-B-wait-forked,Fail
+kms_vblank@pipe-B-wait-forked-busy,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
new file mode 100644
index 000000000000..217139a869bc
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-flakes.txt
@@ -0,0 +1,19 @@
+kms_cursor_crc@.*
+
+kms_cursor_edge_walk@pipe-A-64x64-right-edge
+kms_cursor_edge_walk@pipe-B-64x64-left-edge
+kms_flip@flip-vs-expired-vblank
+kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-B
+kms_plane_multiple@atomic-pipe-A-tiling-none
+kms_plane@plane-position-hole
+kms_sequence@get-forked
+kms_sequence@get-forked-busy
+kms_setmode@basic
+kms_vblank@pipe-A-accuracy-idle
+kms_vblank@pipe-A-query-busy
+kms_vblank@pipe-A-query-forked-busy
+kms_vblank@pipe-A-wait-idle
+kms_vblank@pipe-B-accuracy-idle
+kms_vblank@pipe-B-query-busy
+kms_vblank@pipe-B-query-forked-busy
+kms_vblank@pipe-B-wait-idle
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
new file mode 100644
index 000000000000..10c3d81a919a
--- /dev/null
+++ b/drivers/gpu/drm/ci/rockchip-rk3399-skips.txt
@@ -0,0 +1,5 @@
+# Suspend to RAM seems to be broken on this machine
+.*suspend.*
+
+# Too unstable, machine ends up hanging after lots of Oopses
+kms_cursor_legacy.*
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
new file mode 100644
index 000000000000..2178bdd064dd
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-fails.txt
@@ -0,0 +1,38 @@
+kms_addfb_basic@addfb25-bad-modifier,Fail
+kms_addfb_basic@bad-pitch-65536,Fail
+kms_addfb_basic@bo-too-small,Fail
+kms_addfb_basic@size-max,Fail
+kms_addfb_basic@too-high,Fail
+kms_atomic_transition@plane-primary-toggle-with-vblank-wait,Fail
+kms_bw@linear-tiling-1-displays-1920x1080p,Fail
+kms_bw@linear-tiling-1-displays-2560x1440p,Fail
+kms_bw@linear-tiling-1-displays-3840x2160p,Fail
+kms_bw@linear-tiling-2-displays-1920x1080p,Fail
+kms_bw@linear-tiling-2-displays-2560x1440p,Fail
+kms_bw@linear-tiling-2-displays-3840x2160p,Fail
+kms_invalid_mode@int-max-clock,Fail
+kms_plane_scaling@downscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@downscale-with-rotation-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-25,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5,Fail
+kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-75,Fail
+kms_plane_scaling@upscale-with-modifier-20x20,Fail
+kms_plane_scaling@upscale-with-modifier-factor-0-25,Fail
+kms_plane_scaling@upscale-with-pixel-format-20x20,Fail
+kms_plane_scaling@upscale-with-pixel-format-factor-0-25,Fail
+kms_plane_scaling@upscale-with-rotation-20x20,Fail
+kms_vblank@crtc-id,Fail
+kms_vblank@invalid,Fail
+kms_vblank@pipe-A-accuracy-idle,Fail
+kms_vblank@pipe-A-query-busy,Fail
+kms_vblank@pipe-A-query-forked,Fail
+kms_vblank@pipe-A-query-forked-busy,Fail
+kms_vblank@pipe-A-query-idle,Fail
+kms_vblank@pipe-A-ts-continuation-idle,Fail
+kms_vblank@pipe-A-ts-continuation-modeset,Fail
+kms_vblank@pipe-A-ts-continuation-suspend,Fail
+kms_vblank@pipe-A-wait-busy,Fail
+kms_vblank@pipe-A-wait-forked,Fail
+kms_vblank@pipe-A-wait-forked-busy,Fail
+kms_vblank@pipe-A-wait-idle,Fail
\ No newline at end of file
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt b/drivers/gpu/drm/ci/virtio_gpu-none-flakes.txt
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
new file mode 100644
index 000000000000..78be18174012
--- /dev/null
+++ b/drivers/gpu/drm/ci/virtio_gpu-none-skips.txt
@@ -0,0 +1,6 @@
+# Hits a "refcount_t: underflow; use-after-free" in virtio_gpu_fence_event_process
+# When run in a particular order with other tests
+kms_cursor_legacy.*
+
+# Job just hangs without any output
+kms_flip@flip-vs-suspend.*
\ No newline at end of file
-- 
2.31.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH v9] drm: Add initial ci/ subdirectory
       [not found]           ` <20220912072903.14626-1-tomeu.vizoso@collabora.com>
@ 2022-10-14 17:10             ` Rob Clark
  0 siblings, 0 replies; 13+ messages in thread
From: Rob Clark @ 2022-10-14 17:10 UTC (permalink / raw)
  To: Tomeu Vizoso
  Cc: David Airlie, Daniel Vetter, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger, Rob Clark,
	dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Neil Armstrong

On Mon, Sep 12, 2022 at 12:29 AM Tomeu Vizoso
<tomeu.vizoso@collabora.com> wrote:
>
> And use it to store expectations about what the DRM drivers are
> supposed to pass in the IGT test suite.
>
> Also include a configuration file that points to the out-of-tree CI
> scripts.
>
> By storing the test expectations along the code we can make sure both
> stay in sync with each other, and so we can know when a code change
> breaks those expectations.
>
> This will allow all contributors to drm to reuse the infrastructure
> already in gitlab.freedesktop.org to test the driver on several
> generations of the hardware.
>
> v2:
>   - Fix names of result expectation files to match SoC
>   - Don't execute tests that are going to skip on all boards
>
> v3:
>   - Remove tracking of dmesg output during test execution
>
> v4:
>   - Move up to drivers/gpu/drm
>   - Add support for a bunch of other drivers
>   - Explain how to incorporate fixes for CI from a
>     ${TARGET_BRANCH}-external-fixes branch
>   - Remove tests that pass from expected results file, to reduce the
>     size of in-tree files
>   - Add docs about how to deal with outages in automated testing labs
>   - Specify the exact SHA of the CI scripts to be used
>
> v5:
>   - Remove unneeded skips from Meson expectations file
>   - Use a more advanced runner that detects flakes automatically
>   - Use a more succint format for the expectations
>   - Run many more tests (and use sharding to finish in time)
>   - Use skip lists to avoid hanging machines
>   - Add some build testing
>   - Build IGT in each pipeline for faster uprevs
>   - List failures in the GitLab UI
>
> v6:
>   - Rebase on top of latest drm-next
>   - Lower priority of LAVA jobs to not impact Mesa CI as much
>   - Update docs
>
> v7:
>   - Rebase on top of latest drm-next
>
> v8:
>   - Move all files specific to testing the kernel into the kernel tree
>     (thus I have dropped the r-bs I had collected so far)
>   - Uprev Gitlab CI infrastructure scripts to the latest from Mesa
>   - Add MAINTAINERS entry
>   - Fix boot on MT8173 by adding some Kconfigs that are now needed
>   - Link to the docs from index.rst and hard-wrap the file
>
> v9:
>   - Only automatically run the pipelines for merge requests
>   - Switch to zstd for the build artifacts to align with Mesa
>   - Add Qcom USB PHYs to config as they are now =m in the defconfig
>
> Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>

Reviewed-by: Rob Clark <robdclark@gmail.com>

> ---
>  Documentation/gpu/automated_testing.rst       |  144 +
>  Documentation/gpu/index.rst                   |    1 +
>  MAINTAINERS                                   |    8 +
>  drivers/gpu/drm/ci/arm.config                 |   57 +
>  drivers/gpu/drm/ci/arm64.config               |  179 ++
>  drivers/gpu/drm/ci/build-igt.sh               |   43 +
>  drivers/gpu/drm/ci/build.sh                   |  158 +
>  drivers/gpu/drm/ci/build.yml                  |  110 +
>  drivers/gpu/drm/ci/check-patch.py             |   57 +
>  drivers/gpu/drm/ci/container.yml              |   54 +
>  drivers/gpu/drm/ci/gitlab-ci.yml              |  225 ++
>  drivers/gpu/drm/ci/igt_runner.sh              |   77 +
>  drivers/gpu/drm/ci/image-tags.yml             |   13 +
>  drivers/gpu/drm/ci/lava-submit.sh             |   53 +
>  drivers/gpu/drm/ci/static-checks.yml          |   12 +
>  drivers/gpu/drm/ci/test.yml                   |  322 ++
>  drivers/gpu/drm/ci/testlist.txt               | 2763 +++++++++++++++++
>  drivers/gpu/drm/ci/x86_64.config              |  105 +
>  .../gpu/drm/ci/xfails/amdgpu-stoney-fails.txt |   19 +
>  .../drm/ci/xfails/amdgpu-stoney-flakes.txt    |   15 +
>  .../gpu/drm/ci/xfails/amdgpu-stoney-skips.txt |    2 +
>  .../gpu/drm/ci/xfails/i915-amly-flakes.txt    |   32 +
>  drivers/gpu/drm/ci/xfails/i915-amly-skips.txt |    2 +
>  drivers/gpu/drm/ci/xfails/i915-apl-fails.txt  |   29 +
>  drivers/gpu/drm/ci/xfails/i915-apl-flakes.txt |    1 +
>  drivers/gpu/drm/ci/xfails/i915-apl-skips.txt  |    2 +
>  drivers/gpu/drm/ci/xfails/i915-cml-flakes.txt |   37 +
>  drivers/gpu/drm/ci/xfails/i915-glk-flakes.txt |   40 +
>  drivers/gpu/drm/ci/xfails/i915-glk-skips.txt  |    2 +
>  drivers/gpu/drm/ci/xfails/i915-kbl-fails.txt  |    8 +
>  drivers/gpu/drm/ci/xfails/i915-kbl-flakes.txt |   25 +
>  drivers/gpu/drm/ci/xfails/i915-kbl-skips.txt  |    2 +
>  drivers/gpu/drm/ci/xfails/i915-tgl-fails.txt  |   19 +
>  drivers/gpu/drm/ci/xfails/i915-tgl-flakes.txt |    5 +
>  drivers/gpu/drm/ci/xfails/i915-tgl-skips.txt  |    8 +
>  drivers/gpu/drm/ci/xfails/i915-whl-fails.txt  |   30 +
>  drivers/gpu/drm/ci/xfails/i915-whl-flakes.txt |    1 +
>  .../drm/ci/xfails/mediatek-mt8173-fails.txt   |   29 +
>  .../drm/ci/xfails/mediatek-mt8183-fails.txt   |   10 +
>  .../drm/ci/xfails/mediatek-mt8183-flakes.txt  |   14 +
>  .../gpu/drm/ci/xfails/meson-g12b-fails.txt    |    5 +
>  .../gpu/drm/ci/xfails/meson-g12b-flakes.txt   |    4 +
>  .../gpu/drm/ci/xfails/msm-apq8016-fails.txt   |   15 +
>  .../gpu/drm/ci/xfails/msm-apq8016-flakes.txt  |    4 +
>  .../gpu/drm/ci/xfails/msm-apq8096-fails.txt   |    2 +
>  .../gpu/drm/ci/xfails/msm-apq8096-flakes.txt  |    4 +
>  .../gpu/drm/ci/xfails/msm-apq8096-skips.txt   |    2 +
>  .../gpu/drm/ci/xfails/msm-sc7180-fails.txt    |   21 +
>  .../gpu/drm/ci/xfails/msm-sc7180-flakes.txt   |    6 +
>  .../gpu/drm/ci/xfails/msm-sc7180-skips.txt    |   23 +
>  .../gpu/drm/ci/xfails/msm-sdm845-fails.txt    |   43 +
>  .../gpu/drm/ci/xfails/msm-sdm845-flakes.txt   |   11 +
>  .../gpu/drm/ci/xfails/msm-sdm845-skips.txt    |    2 +
>  .../drm/ci/xfails/rockchip-rk3288-fails.txt   |   43 +
>  .../drm/ci/xfails/rockchip-rk3288-flakes.txt  |    2 +
>  .../drm/ci/xfails/rockchip-rk3288-skips.txt   |   49 +
>  .../drm/ci/xfails/rockchip-rk3399-fails.txt   |   29 +
>  .../drm/ci/xfails/rockchip-rk3399-flakes.txt  |   20 +
>  .../drm/ci/xfails/rockchip-rk3399-skips.txt   |    5 +
>  .../drm/ci/xfails/virtio_gpu-none-fails.txt   |   38 +
>  .../drm/ci/xfails/virtio_gpu-none-flakes.txt  |    0
>  .../drm/ci/xfails/virtio_gpu-none-skips.txt   |    6 +
>  62 files changed, 5047 insertions(+)
>  create mode 100644 Documentation/gpu/automated_testing.rst
>  create mode 100644 drivers/gpu/drm/ci/arm.config
>  create mode 100644 drivers/gpu/drm/ci/arm64.config
>  create mode 100644 drivers/gpu/drm/ci/build-igt.sh
>  create mode 100644 drivers/gpu/drm/ci/build.sh
>  create mode 100644 drivers/gpu/drm/ci/build.yml
>  create mode 100755 drivers/gpu/drm/ci/check-patch.py
>  create mode 100644 drivers/gpu/drm/ci/container.yml
>  create mode 100644 drivers/gpu/drm/ci/gitlab-ci.yml
>  create mode 100755 drivers/gpu/drm/ci/igt_runner.sh
>  create mode 100644 drivers/gpu/drm/ci/image-tags.yml
>  create mode 100755 drivers/gpu/drm/ci/lava-submit.sh
>  create mode 100644 drivers/gpu/drm/ci/static-checks.yml
>  create mode 100644 drivers/gpu/drm/ci/test.yml
>  create mode 100644 drivers/gpu/drm/ci/testlist.txt
>  create mode 100644 drivers/gpu/drm/ci/x86_64.config
>  create mode 100644 drivers/gpu/drm/ci/xfails/amdgpu-stoney-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/amdgpu-stoney-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/amdgpu-stoney-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-amly-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-amly-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-apl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-apl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-apl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-cml-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-glk-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-glk-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-kbl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-kbl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-kbl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-tgl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-tgl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-tgl-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-whl-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/i915-whl-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/mediatek-mt8173-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/mediatek-mt8183-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/mediatek-mt8183-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/meson-g12b-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/meson-g12b-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-apq8016-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-apq8016-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-apq8096-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-apq8096-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-apq8096-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sc7180-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sc7180-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sc7180-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sdm845-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sdm845-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/msm-sdm845-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3288-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3288-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3288-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3399-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3399-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/rockchip-rk3399-skips.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/virtio_gpu-none-fails.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/virtio_gpu-none-flakes.txt
>  create mode 100644 drivers/gpu/drm/ci/xfails/virtio_gpu-none-skips.txt
>
[snip]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v8] drm: Add initial ci/ subdirectory
       [not found]           ` <CAPj87rMYW1xZdacZ1Y0qk+D9xpqsqE+cwbQ1j1nZ7S+RKpzOoQ@mail.gmail.com>
@ 2022-10-25  7:32             ` Daniel Vetter
  2022-10-25 15:06               ` Daniel Stone
  0 siblings, 1 reply; 13+ messages in thread
From: Daniel Vetter @ 2022-10-25  7:32 UTC (permalink / raw)
  To: Daniel Stone
  Cc: Tomeu Vizoso, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger, Rob Clark,
	dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Neil Armstrong, Rob Clark

On Fri, 9 Sept 2022 at 19:18, Daniel Stone <daniel@fooishbar.org> wrote:
>
> Hi,
>
> On Fri, 9 Sept 2022 at 15:15, Tomeu Vizoso <tomeu.vizoso@collabora.com> wrote:
>>
>> Also include a configuration file that points to the out-of-tree CI
>> scripts.
>
>
>  I think this para is outdated given ...
>
>> v8:
>>   - Move all files specific to testing the kernel into the kernel tree
>>     (thus I have dropped the r-bs I had collected so far)
>>   - Uprev Gitlab CI infrastructure scripts to the latest from Mesa
>
>
> But equally - and sorry for not jumping on the IRC (?) discussion as I was in the middle of other stuff when it came up - I'm don't think this is the right plan.
>
> Mesa having all its CI in-tree makes sense, because merges happen rapidly to a single canonical tree. If the scripts need to be changed for whatever reason, we can merge something in under an hour and everyone immediately gets it. DRM is quite different though, given the forest of trees we have and the long merge paths between them. I worry that merging the CI scripts in-tree - especially for our initial attempt at it, when we're likely to need to make quite a lot of changes before it settles down to become a stable system that works for everyone - is shooting ourselves in the foot by limiting our flexibility.

So the entire "we have multiple trees" is why I want at least the
gitlab-ci.yaml in tree, to force people to collaborate more on one
thing instead of everyone rolling their own fun and hacking stuff up.
And there's still tons of stuff outside in the separate repo, like the
lab status so Linus doesn't get a silly history of lab flapping.

Also wrt "developers don't get the update right away due to
backmerge/pull delays", that's why integration trees like drm-tip or
linux-next exist. So maybe initially we should make sure the ci
patches go in through drm-misc, to maximize how many people see it.
And even for mesa it's not fully automatic, you still have the rebase
your branch if you picked a bad one for development (but yeah marge
does that if the MR is ready). If you're doing kernel development on a
linear tree instead of an integration tree, you're doing it very
wrong.

What I think everyone agrees on is that we probably get the split
wrong and need to shuffle some files back&forth, and that's something
we need to warn Linus about I guess. But somewhere we do need a split
between internal and external stuff, and personally I'd like if at
least the pure sw testing (build and virtual stuff) could be all in
upstream.

> Given that it's currently very dependent on fd.o infrastructure (published ci-templates, the registry, the specific-tag runners for Collabora/CrOS DUTs, etc), there isn't much of a portability gain in bringing the scripts into the tree either. It's a good goal, but not where we are today.

I don't think there's huge chances for any non-fdo gitlab anytime
soon. Once we get there we might need to figure out how to standardize
the hw lab interfacing, and if we have all the sw targets in upstream
that should help with shuffling stuff around for a hypothetical LF
gitlab CI (or whatever it is).
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v8] drm: Add initial ci/ subdirectory
  2022-10-25  7:32             ` [PATCH v8] " Daniel Vetter
@ 2022-10-25 15:06               ` Daniel Stone
  0 siblings, 0 replies; 13+ messages in thread
From: Daniel Stone @ 2022-10-25 15:06 UTC (permalink / raw)
  To: Daniel Vetter
  Cc: Tomeu Vizoso, David Airlie, Jonathan Corbet, Carlo Caione,
	Kevin Hilman, Heiko Stuebner, Matthias Brugger, Rob Clark,
	dri-devel, linux-doc, linux-kernel, linux-arm-kernel,
	linux-amlogic, linux-rockchip, linux-mediatek, kernel,
	Neil Armstrong, Rob Clark

Hi all,

On Tue, 25 Oct 2022 at 08:32, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Fri, 9 Sept 2022 at 19:18, Daniel Stone <daniel@fooishbar.org> wrote:
> > But equally - and sorry for not jumping on the IRC (?) discussion as I was in the middle of other stuff when it came up - I'm don't think this is the right plan.
> >
> > Mesa having all its CI in-tree makes sense, because merges happen rapidly to a single canonical tree. If the scripts need to be changed for whatever reason, we can merge something in under an hour and everyone immediately gets it. DRM is quite different though, given the forest of trees we have and the long merge paths between them. I worry that merging the CI scripts in-tree - especially for our initial attempt at it, when we're likely to need to make quite a lot of changes before it settles down to become a stable system that works for everyone - is shooting ourselves in the foot by limiting our flexibility.
>
> So the entire "we have multiple trees" is why I want at least the
> gitlab-ci.yaml in tree, to force people to collaborate more on one
> thing instead of everyone rolling their own fun and hacking stuff up.
> And there's still tons of stuff outside in the separate repo, like the
> lab status so Linus doesn't get a silly history of lab flapping.
>
> Also wrt "developers don't get the update right away due to
> backmerge/pull delays", that's why integration trees like drm-tip or
> linux-next exist. So maybe initially we should make sure the ci
> patches go in through drm-misc, to maximize how many people see it.
> And even for mesa it's not fully automatic, you still have the rebase
> your branch if you picked a bad one for development (but yeah marge
> does that if the MR is ready). If you're doing kernel development on a
> linear tree instead of an integration tree, you're doing it very
> wrong.
>
> What I think everyone agrees on is that we probably get the split
> wrong and need to shuffle some files back&forth, and that's something
> we need to warn Linus about I guess. But somewhere we do need a split
> between internal and external stuff, and personally I'd like if at
> least the pure sw testing (build and virtual stuff) could be all in
> upstream.
>
> > Given that it's currently very dependent on fd.o infrastructure (published ci-templates, the registry, the specific-tag runners for Collabora/CrOS DUTs, etc), there isn't much of a portability gain in bringing the scripts into the tree either. It's a good goal, but not where we are today.
>
> I don't think there's huge chances for any non-fdo gitlab anytime
> soon. Once we get there we might need to figure out how to standardize
> the hw lab interfacing, and if we have all the sw targets in upstream
> that should help with shuffling stuff around for a hypothetical LF
> gitlab CI (or whatever it is).

Yep, having talked through things on IRC, I'm happy with where we are;
let's give it a go and find out.

Acked-by: Daniel Stone <daniels@collabora.com>

Cheers,
Daniel

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-10-25 15:08 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <20220511061533.56881-1-tomeu.vizoso@collabora.com>
2022-05-17  8:16 ` [RFC v4] drm: Add initial ci/ subdirectory Tomeu Vizoso
2022-05-17  9:18   ` Neil Armstrong
2022-05-17  9:24     ` Tomeu Vizoso
2022-07-26 18:16   ` [PATCH v5] " Tomeu Vizoso
2022-07-29 15:43     ` Rob Clark
2022-08-10 18:24     ` Rodrigo Siqueira Jordao
2022-08-10 22:08       ` Rob Clark
2022-08-11 10:10       ` Tomeu Vizoso
2022-08-16 15:37     ` [PATCH v6] " Tomeu Vizoso
2022-08-29  9:31       ` [PATCH v7] " Tomeu Vizoso
     [not found]         ` <20220909141528.5090-1-tomeu.vizoso@collabora.com>
     [not found]           ` <20220912072903.14626-1-tomeu.vizoso@collabora.com>
2022-10-14 17:10             ` [PATCH v9] " Rob Clark
     [not found]           ` <CAPj87rMYW1xZdacZ1Y0qk+D9xpqsqE+cwbQ1j1nZ7S+RKpzOoQ@mail.gmail.com>
2022-10-25  7:32             ` [PATCH v8] " Daniel Vetter
2022-10-25 15:06               ` Daniel Stone

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).