[igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Rodrigo Siqueira]

Add various test suites relevant for the amdgpu driver.

Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Nicholas Choi <Nicholas.Choi@amd.com>
Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
Cc: Alexander Deucher <alexander.deucher@amd.com>
Cc: Martin Roukala <martin.roukala@mupuf.org>
Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Mark Yacoub <markyacoub@google.com>

Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
---
 tests/amdgpu-ci/README                 |  31 +++++
 tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
 tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
 3 files changed, 339 insertions(+)
 create mode 100644 tests/amdgpu-ci/README
 create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
 create mode 100644 tests/amdgpu-ci/full-feedback.testlist

diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
new file mode 100644
index 00000000..bd34245c
--- /dev/null
+++ b/tests/amdgpu-ci/README
@@ -0,0 +1,31 @@
+This directory contains test lists that are used by AMD's CI. The
+files are passed to piglit with the --test-list parameter directly.
+
+The test lists are contained in the IGT repository for several
+reasons:
+
+- The lists stay synchronized with the IGT codebase.
+- Public availability. Kernel developers can see what tests are run,
+  and can see what changes are done to the set, when, and why.
+
+Changing the test lists should only happen with approval from:
+- Harry Wentland <harry.wentland@amd.com>
+- Nicholas Choi <Nicholas.Choi@amd.com>
+- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
+- Sun peng (Leo) Li <sunpeng.li@amd.com>
+
+======================
+fast-feedback.testlist
+======================
+
+Fast-feedback contains tests that roughly tests a wide coverage of features in
+a short time. We use this list for presubmission validation. Results of a
+fast-feedback test round should only be considered to mean that the kernel is
+not obviously broken.
+
+======================
+full-feedback.testlist
+======================
+
+This is an extensive set of tests that takes a long time to complete in which
+we usually use as post-submission.
diff --git a/tests/amdgpu-ci/fast-feedback.testlist b/tests/amdgpu-ci/fast-feedback.testlist
new file mode 100644
index 00000000..025bea16
--- /dev/null
+++ b/tests/amdgpu-ci/fast-feedback.testlist
@@ -0,0 +1,135 @@
+# tests to be run at the beginning
+igt@kms_setmode@basic
+
+
+# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
+igt@amdgpu/amd_hotplug@basic
+igt@amdgpu/amd_hotplug@basic-suspend
+igt@kms_bw@linear-tiling-1-displays-1920x1080p
+
+
+# ATOMIC-MODESETTING - Atomic modesetting tests
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@plane-invalid-params-fence
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@crtc-invalid-params-fence
+igt@kms_atomic@atomic-invalid-params
+igt@kms_content_protection@atomic
+igt@kms_flip@basic-flip-vs-dpms@a-dp1
+igt@kms_flip@basic-flip-vs-dpms@c-dp1
+igt@kms_flip@basic-plain-flip@a-dp1
+igt@kms_flip@basic-plain-flip@c-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
+
+
+# COLOR - Color Adjustments
+igt@kms_color@pipe-a-gamma
+igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
+igt@kms_color@pipe-a-degamma
+igt@amdgpu/amd_color@crtc-linear-degamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
+igt@kms_color@pipe-a-ctm-0-25
+igt@amdgpu/amd_color@crtc-linear-regamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
+igt@kms_color@pipe-a-ctm-negative
+igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
+igt@kms_color@pipe-a-ctm-blue-to-red
+igt@kms_color@pipe-a-ctm-0-75
+igt@kms_color@pipe-a-legacy-gamma
+igt@kms_color@pipe-a-ctm-0-5
+igt@kms_color@pipe-a-ctm-max
+igt@kms_color@pipe-a-ctm-red-to-blue
+igt@amdgpu/amd_color@crtc-lut-accuracy
+igt@kms_color@pipe-a-ctm-green-to-red
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
+igt@kms_plane_alpha_blend@pipe-a-alpha-basic
+igt@kms_color@pipe-a-legacy-gamma-reset
+
+
+# VRR
+igt@kms_vrr@flip-basic
+igt@kms_vrr@flip-dpms
+
+
+# MODES - Tests mode settings
+igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
+igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
+igt@kms_setmode@basic-clone-single-crtc
+igt@kms_setmode@invalid-clone-single-crtc
+igt@kms_setmode@invalid-clone-exclusive-crtc
+igt@kms_setmode@clone-exclusive-crtc
+igt@kms_setmode@invalid-clone-single-crtc-stealing
+
+
+# PLANE - DRM plane-level tests
+igt@kms_plane@plane-panning-top-left@pipe-a-planes
+igt@kms_cursor_legacy@flip-vs-cursor-atomic
+igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
+igt@kms_properties@plane-properties-legacy
+igt@kms_plane_cursor@pipe-a-primary-size-256
+igt@kms_plane@plane-position-covered@pipe-a-planes
+igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
+igt@kms_plane@plane-position-hole@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-a-tiling-none
+igt@kms_plane@plane-panning-top-left@pipe-b-planes
+igt@kms_plane@plane-position-covered@pipe-b-planes
+igt@kms_plane_cursor@pipe-a-primary-size-128
+igt@kms_cursor_legacy@cursor-vs-flip-legacy
+igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-c-tiling-none
+igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
+igt@kms_plane_cursor@pipe-a-primary-size-64
+igt@kms_plane@plane-position-hole@pipe-a-planes
+
+
+# DPMS - DPMS
+igt@kms_content_protection@atomic-dpms
+
+
+# HDCP - HDCP functionality
+igt@kms_content_protection@legacy
+
+
+# DISPLAY-LAYER - General DRM API tests and display driver tests
+igt@core_auth@basic-auth
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+
+
+# S3 - S3
+igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
+
+
+# ASSR - ASSR
+igt@amdgpu/amd_assr@assr-links
+igt@amdgpu/amd_assr@assr-links-dpms
diff --git a/tests/amdgpu-ci/full-feedback.testlist b/tests/amdgpu-ci/full-feedback.testlist
new file mode 100644
index 00000000..cc837d47
--- /dev/null
+++ b/tests/amdgpu-ci/full-feedback.testlist
@@ -0,0 +1,173 @@
+# tests to be run at the beginning
+igt@kms_setmode@basic
+
+
+# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
+igt@amdgpu/amd_hotplug@basic
+igt@amdgpu/amd_hotplug@basic-suspend
+igt@kms_bw@linear-tiling-1-displays-1920x1080p
+
+
+# ATOMIC-MODESETTING - Atomic modesetting tests
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@plane-invalid-params-fence
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@crtc-invalid-params-fence
+igt@kms_atomic@atomic-invalid-params
+igt@kms_content_protection@atomic
+igt@kms_flip@basic-flip-vs-dpms@a-dp1
+igt@kms_flip@basic-flip-vs-dpms@c-dp1
+igt@kms_flip@basic-plain-flip@a-dp1
+igt@kms_flip@basic-plain-flip@c-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
+igt@kms_flip_event_leak
+
+
+# COLOR - Color Adjustments
+igt@kms_color@pipe-a-gamma
+igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
+igt@kms_color@pipe-a-degamma
+igt@amdgpu/amd_color@crtc-linear-degamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
+igt@kms_color@pipe-a-ctm-0-25
+igt@amdgpu/amd_color@crtc-linear-regamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
+igt@kms_color@pipe-a-ctm-negative
+igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
+igt@kms_color@pipe-a-ctm-blue-to-red
+igt@kms_color@pipe-a-ctm-0-75
+igt@kms_color@pipe-a-legacy-gamma
+igt@kms_color@pipe-a-ctm-0-5
+igt@kms_color@pipe-a-ctm-max
+igt@kms_color@pipe-a-ctm-red-to-blue
+igt@amdgpu/amd_color@crtc-lut-accuracy
+igt@kms_color@pipe-a-ctm-green-to-red
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
+igt@kms_plane_alpha_blend@pipe-a-alpha-basic
+igt@kms_color@pipe-a-legacy-gamma-reset
+
+
+# VRR
+igt@kms_vrr@flip-basic
+igt@kms_vrr@flip-dpms
+igt@amdgpu/amd_vrr_range@freesync-parsing-dp
+igt@amdgpu/amd_vrr_range@freesync-parsing-dp-suspend
+igt@amdgpu/amd_vrr_range@freesync-range-dp
+igt@amdgpu/amd_vrr_range@freesync-range-dp-suspend
+
+
+# MODES - Tests mode settings
+igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
+igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
+igt@kms_setmode@basic-clone-single-crtc
+igt@kms_setmode@invalid-clone-single-crtc
+igt@kms_setmode@invalid-clone-exclusive-crtc
+igt@kms_setmode@clone-exclusive-crtc
+igt@kms_setmode@invalid-clone-single-crtc-stealing
+
+
+# PLANE - DRM plane-level tests
+igt@kms_plane@plane-panning-top-left@pipe-a-planes
+igt@kms_cursor_legacy@flip-vs-cursor-atomic
+igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
+igt@kms_properties@plane-properties-legacy
+igt@kms_plane_cursor@pipe-a-primary-size-256
+igt@kms_plane@plane-position-covered@pipe-a-planes
+igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
+igt@kms_plane@plane-position-hole@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-a-tiling-none
+igt@kms_plane@plane-panning-top-left@pipe-b-planes
+igt@kms_plane@plane-position-covered@pipe-b-planes
+igt@kms_plane_cursor@pipe-a-primary-size-128
+igt@kms_cursor_legacy@cursor-vs-flip-legacy
+igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-c-tiling-none
+igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
+igt@kms_plane_cursor@pipe-a-primary-size-64
+igt@kms_plane@plane-position-hole@pipe-a-planes
+igt@kms_cursor_crc@pipe-a-cursor-size-change
+igt@kms_cursor_crc@pipe-b-cursor-size-change
+igt@kms_cursor_crc@pipe-a-cursor-alpha-opaque
+igt@kms_cursor_crc@pipe-b-cursor-alpha-opaque
+igt@kms_cursor_crc@pipe-a-cursor-alpha-transparent
+igt@kms_cursor_crc@pipe-b-cursor-alpha-transparent
+igt@kms_cursor_crc@pipe-a-cursor-dpms
+igt@kms_cursor_crc@pipe-b-cursor-dpms
+igt@kms_cursor_crc@pipe-a-cursor-suspend
+igt@kms_cursor_crc@pipe-b-cursor-suspend
+igt@kms_cursor_crc@pipe-a-cursor-max-size-onscreen
+igt@kms_cursor_crc@pipe-b-cursor-max-size-onscreen
+igt@kms_cursor_crc@pipe-a-cursor-max-size-offscreen
+igt@kms_cursor_crc@pipe-b-cursor-max-size-offscreen
+igt@kms_cursor_crc@pipe-a-cursor-max-size-sliding
+igt@kms_cursor_crc@pipe-b-cursor-max-size-sliding
+igt@kms_cursor_crc@pipe-a-cursor-max-size-random
+igt@kms_cursor_crc@pipe-b-cursor-max-size-random
+igt@kms_cursor_crc@pipe-a-cursor-max-size-rapid-movement
+igt@kms_cursor_crc@pipe-b-cursor-max-size-rapid-movement
+igt@kms_concurrent@pipe-a
+igt@kms_concurrent@pipe-b
+
+
+# DPMS - DPMS
+igt@kms_content_protection@atomic-dpms
+
+
+# HDCP - HDCP functionality
+igt@kms_content_protection@legacy
+
+
+# DISPLAY-LAYER - General DRM API tests and display driver tests
+igt@core_auth@basic-auth
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_vblank@pipe-a-ts-continuation-suspend
+igt@amdgpu/amd_mem_leak@connector-suspend-resume
+igt@amdgpu/amd_mem_leak@connector-hotplug
+igt@kms_atomic_interruptible@legacy-setmode@dp-1-pipe-a
+igt@kms_atomic_interruptible@atomic-setmode@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_sysfs_edid_timing
+
+
+# S3 - S3
+igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
+
+
+# ASSR - ASSR
+igt@amdgpu/amd_assr@assr-links
+igt@amdgpu/amd_assr@assr-links-dpms
+igt@amdgpu/amd_assr@assr-links-suspend
+
+
+# tests with higher probability of hanging the system
+igt@amdgpu/amd_link_settings@link-training-configs
-- 
2.25.1

Re: [igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Deucher, Alexander]

[-- Attachment #1: Type: text/plain, Size: 14783 bytes --]

[Public]

Acked-by: Alex Deucher <alexander.deucher@amd.com>
________________________________
From: Siqueira, Rodrigo <Rodrigo.Siqueira@amd.com>
Sent: Friday, October 8, 2021 3:02 PM
To: igt-dev@lists.freedesktop.org <igt-dev@lists.freedesktop.org>
Cc: Wentland, Harry <Harry.Wentland@amd.com>; Choi, Nicholas <Nicholas.Choi@amd.com>; Li, Sun peng (Leo) <Sunpeng.Li@amd.com>; Deucher, Alexander <Alexander.Deucher@amd.com>; Martin Roukala <martin.roukala@mupuf.org>; Goodfellow, Hayden <Hayden.Goodfellow@amd.com>; Simon Ser <contact@emersion.fr>; Mark Yacoub <markyacoub@google.com>
Subject: [PATCH i-g-t] tests: Add amdgpu test suite

Add various test suites relevant for the amdgpu driver.

Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Nicholas Choi <Nicholas.Choi@amd.com>
Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
Cc: Alexander Deucher <alexander.deucher@amd.com>
Cc: Martin Roukala <martin.roukala@mupuf.org>
Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
Cc: Simon Ser <contact@emersion.fr>
Cc: Mark Yacoub <markyacoub@google.com>

Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
---
 tests/amdgpu-ci/README                 |  31 +++++
 tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
 tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
 3 files changed, 339 insertions(+)
 create mode 100644 tests/amdgpu-ci/README
 create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
 create mode 100644 tests/amdgpu-ci/full-feedback.testlist

diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
new file mode 100644
index 00000000..bd34245c
--- /dev/null
+++ b/tests/amdgpu-ci/README
@@ -0,0 +1,31 @@
+This directory contains test lists that are used by AMD's CI. The
+files are passed to piglit with the --test-list parameter directly.
+
+The test lists are contained in the IGT repository for several
+reasons:
+
+- The lists stay synchronized with the IGT codebase.
+- Public availability. Kernel developers can see what tests are run,
+  and can see what changes are done to the set, when, and why.
+
+Changing the test lists should only happen with approval from:
+- Harry Wentland <harry.wentland@amd.com>
+- Nicholas Choi <Nicholas.Choi@amd.com>
+- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
+- Sun peng (Leo) Li <sunpeng.li@amd.com>
+
+======================
+fast-feedback.testlist
+======================
+
+Fast-feedback contains tests that roughly tests a wide coverage of features in
+a short time. We use this list for presubmission validation. Results of a
+fast-feedback test round should only be considered to mean that the kernel is
+not obviously broken.
+
+======================
+full-feedback.testlist
+======================
+
+This is an extensive set of tests that takes a long time to complete in which
+we usually use as post-submission.
diff --git a/tests/amdgpu-ci/fast-feedback.testlist b/tests/amdgpu-ci/fast-feedback.testlist
new file mode 100644
index 00000000..025bea16
--- /dev/null
+++ b/tests/amdgpu-ci/fast-feedback.testlist
@@ -0,0 +1,135 @@
+# tests to be run at the beginning
+igt@kms_setmode@basic
+
+
+# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
+igt@amdgpu/amd_hotplug@basic
+igt@amdgpu/amd_hotplug@basic-suspend
+igt@kms_bw@linear-tiling-1-displays-1920x1080p
+
+
+# ATOMIC-MODESETTING - Atomic modesetting tests
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@plane-invalid-params-fence
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@crtc-invalid-params-fence
+igt@kms_atomic@atomic-invalid-params
+igt@kms_content_protection@atomic
+igt@kms_flip@basic-flip-vs-dpms@a-dp1
+igt@kms_flip@basic-flip-vs-dpms@c-dp1
+igt@kms_flip@basic-plain-flip@a-dp1
+igt@kms_flip@basic-plain-flip@c-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
+
+
+# COLOR - Color Adjustments
+igt@kms_color@pipe-a-gamma
+igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
+igt@kms_color@pipe-a-degamma
+igt@amdgpu/amd_color@crtc-linear-degamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
+igt@kms_color@pipe-a-ctm-0-25
+igt@amdgpu/amd_color@crtc-linear-regamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
+igt@kms_color@pipe-a-ctm-negative
+igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
+igt@kms_color@pipe-a-ctm-blue-to-red
+igt@kms_color@pipe-a-ctm-0-75
+igt@kms_color@pipe-a-legacy-gamma
+igt@kms_color@pipe-a-ctm-0-5
+igt@kms_color@pipe-a-ctm-max
+igt@kms_color@pipe-a-ctm-red-to-blue
+igt@amdgpu/amd_color@crtc-lut-accuracy
+igt@kms_color@pipe-a-ctm-green-to-red
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
+igt@kms_plane_alpha_blend@pipe-a-alpha-basic
+igt@kms_color@pipe-a-legacy-gamma-reset
+
+
+# VRR
+igt@kms_vrr@flip-basic
+igt@kms_vrr@flip-dpms
+
+
+# MODES - Tests mode settings
+igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
+igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
+igt@kms_setmode@basic-clone-single-crtc
+igt@kms_setmode@invalid-clone-single-crtc
+igt@kms_setmode@invalid-clone-exclusive-crtc
+igt@kms_setmode@clone-exclusive-crtc
+igt@kms_setmode@invalid-clone-single-crtc-stealing
+
+
+# PLANE - DRM plane-level tests
+igt@kms_plane@plane-panning-top-left@pipe-a-planes
+igt@kms_cursor_legacy@flip-vs-cursor-atomic
+igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
+igt@kms_properties@plane-properties-legacy
+igt@kms_plane_cursor@pipe-a-primary-size-256
+igt@kms_plane@plane-position-covered@pipe-a-planes
+igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
+igt@kms_plane@plane-position-hole@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-a-tiling-none
+igt@kms_plane@plane-panning-top-left@pipe-b-planes
+igt@kms_plane@plane-position-covered@pipe-b-planes
+igt@kms_plane_cursor@pipe-a-primary-size-128
+igt@kms_cursor_legacy@cursor-vs-flip-legacy
+igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-c-tiling-none
+igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
+igt@kms_plane_cursor@pipe-a-primary-size-64
+igt@kms_plane@plane-position-hole@pipe-a-planes
+
+
+# DPMS - DPMS
+igt@kms_content_protection@atomic-dpms
+
+
+# HDCP - HDCP functionality
+igt@kms_content_protection@legacy
+
+
+# DISPLAY-LAYER - General DRM API tests and display driver tests
+igt@core_auth@basic-auth
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+
+
+# S3 - S3
+igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
+
+
+# ASSR - ASSR
+igt@amdgpu/amd_assr@assr-links
+igt@amdgpu/amd_assr@assr-links-dpms
diff --git a/tests/amdgpu-ci/full-feedback.testlist b/tests/amdgpu-ci/full-feedback.testlist
new file mode 100644
index 00000000..cc837d47
--- /dev/null
+++ b/tests/amdgpu-ci/full-feedback.testlist
@@ -0,0 +1,173 @@
+# tests to be run at the beginning
+igt@kms_setmode@basic
+
+
+# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
+igt@amdgpu/amd_hotplug@basic
+igt@amdgpu/amd_hotplug@basic-suspend
+igt@kms_bw@linear-tiling-1-displays-1920x1080p
+
+
+# ATOMIC-MODESETTING - Atomic modesetting tests
+igt@kms_atomic@plane-overlay-legacy
+igt@kms_atomic@plane-primary-legacy
+igt@kms_atomic@plane-immutable-zpos
+igt@kms_atomic@test-only
+igt@kms_atomic@plane-cursor-legacy
+igt@kms_atomic@plane-invalid-params
+igt@kms_atomic@plane-invalid-params-fence
+igt@kms_atomic@crtc-invalid-params
+igt@kms_atomic@crtc-invalid-params-fence
+igt@kms_atomic@atomic-invalid-params
+igt@kms_content_protection@atomic
+igt@kms_flip@basic-flip-vs-dpms@a-dp1
+igt@kms_flip@basic-flip-vs-dpms@c-dp1
+igt@kms_flip@basic-plain-flip@a-dp1
+igt@kms_flip@basic-plain-flip@c-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
+igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
+igt@kms_flip_event_leak
+
+
+# COLOR - Color Adjustments
+igt@kms_color@pipe-a-gamma
+igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
+igt@kms_color@pipe-a-degamma
+igt@amdgpu/amd_color@crtc-linear-degamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
+igt@kms_color@pipe-a-ctm-0-25
+igt@amdgpu/amd_color@crtc-linear-regamma
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
+igt@kms_color@pipe-a-ctm-negative
+igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
+igt@kms_color@pipe-a-ctm-blue-to-red
+igt@kms_color@pipe-a-ctm-0-75
+igt@kms_color@pipe-a-legacy-gamma
+igt@kms_color@pipe-a-ctm-0-5
+igt@kms_color@pipe-a-ctm-max
+igt@kms_color@pipe-a-ctm-red-to-blue
+igt@amdgpu/amd_color@crtc-lut-accuracy
+igt@kms_color@pipe-a-ctm-green-to-red
+igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
+igt@kms_plane_alpha_blend@pipe-a-alpha-basic
+igt@kms_color@pipe-a-legacy-gamma-reset
+
+
+# VRR
+igt@kms_vrr@flip-basic
+igt@kms_vrr@flip-dpms
+igt@amdgpu/amd_vrr_range@freesync-parsing-dp
+igt@amdgpu/amd_vrr_range@freesync-parsing-dp-suspend
+igt@amdgpu/amd_vrr_range@freesync-range-dp
+igt@amdgpu/amd_vrr_range@freesync-range-dp-suspend
+
+
+# MODES - Tests mode settings
+igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
+igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
+igt@kms_setmode@basic-clone-single-crtc
+igt@kms_setmode@invalid-clone-single-crtc
+igt@kms_setmode@invalid-clone-exclusive-crtc
+igt@kms_setmode@clone-exclusive-crtc
+igt@kms_setmode@invalid-clone-single-crtc-stealing
+
+
+# PLANE - DRM plane-level tests
+igt@kms_plane@plane-panning-top-left@pipe-a-planes
+igt@kms_cursor_legacy@flip-vs-cursor-atomic
+igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
+igt@kms_properties@plane-properties-legacy
+igt@kms_plane_cursor@pipe-a-primary-size-256
+igt@kms_plane@plane-position-covered@pipe-a-planes
+igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
+igt@kms_plane@plane-position-hole@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-a-tiling-none
+igt@kms_plane@plane-panning-top-left@pipe-b-planes
+igt@kms_plane@plane-position-covered@pipe-b-planes
+igt@kms_plane_cursor@pipe-a-primary-size-128
+igt@kms_cursor_legacy@cursor-vs-flip-legacy
+igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
+igt@kms_plane_multiple@atomic-pipe-c-tiling-none
+igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
+igt@kms_plane_cursor@pipe-a-primary-size-64
+igt@kms_plane@plane-position-hole@pipe-a-planes
+igt@kms_cursor_crc@pipe-a-cursor-size-change
+igt@kms_cursor_crc@pipe-b-cursor-size-change
+igt@kms_cursor_crc@pipe-a-cursor-alpha-opaque
+igt@kms_cursor_crc@pipe-b-cursor-alpha-opaque
+igt@kms_cursor_crc@pipe-a-cursor-alpha-transparent
+igt@kms_cursor_crc@pipe-b-cursor-alpha-transparent
+igt@kms_cursor_crc@pipe-a-cursor-dpms
+igt@kms_cursor_crc@pipe-b-cursor-dpms
+igt@kms_cursor_crc@pipe-a-cursor-suspend
+igt@kms_cursor_crc@pipe-b-cursor-suspend
+igt@kms_cursor_crc@pipe-a-cursor-max-size-onscreen
+igt@kms_cursor_crc@pipe-b-cursor-max-size-onscreen
+igt@kms_cursor_crc@pipe-a-cursor-max-size-offscreen
+igt@kms_cursor_crc@pipe-b-cursor-max-size-offscreen
+igt@kms_cursor_crc@pipe-a-cursor-max-size-sliding
+igt@kms_cursor_crc@pipe-b-cursor-max-size-sliding
+igt@kms_cursor_crc@pipe-a-cursor-max-size-random
+igt@kms_cursor_crc@pipe-b-cursor-max-size-random
+igt@kms_cursor_crc@pipe-a-cursor-max-size-rapid-movement
+igt@kms_cursor_crc@pipe-b-cursor-max-size-rapid-movement
+igt@kms_concurrent@pipe-a
+igt@kms_concurrent@pipe-b
+
+
+# DPMS - DPMS
+igt@kms_content_protection@atomic-dpms
+
+
+# HDCP - HDCP functionality
+igt@kms_content_protection@legacy
+
+
+# DISPLAY-LAYER - General DRM API tests and display driver tests
+igt@core_auth@basic-auth
+igt@kms_prop_blob@basic
+igt@kms_prop_blob@blob-prop-core
+igt@kms_prop_blob@blob-prop-validate
+igt@kms_prop_blob@blob-prop-lifetime
+igt@kms_prop_blob@blob-multiple
+igt@kms_prop_blob@invalid-get-prop-any
+igt@kms_prop_blob@invalid-get-prop
+igt@kms_prop_blob@invalid-set-prop-any
+igt@kms_prop_blob@invalid-set-prop
+igt@kms_vblank@pipe-a-ts-continuation-suspend
+igt@amdgpu/amd_mem_leak@connector-suspend-resume
+igt@amdgpu/amd_mem_leak@connector-hotplug
+igt@kms_atomic_interruptible@legacy-setmode@dp-1-pipe-a
+igt@kms_atomic_interruptible@atomic-setmode@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
+igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
+igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
+igt@kms_getfb@getfb-handle-zero
+igt@kms_getfb@getfb-handle-valid
+igt@kms_getfb@getfb-handle-closed
+igt@kms_getfb@getfb-handle-not-fb
+igt@kms_getfb@getfb-addfb-different-handles
+igt@kms_getfb@getfb-repeated-different-handles
+igt@kms_getfb@getfb2-handle-zero
+igt@kms_getfb@getfb2-handle-closed
+igt@kms_getfb@getfb2-handle-not-fb
+igt@kms_getfb@getfb2-into-addfb2
+igt@kms_getfb@getfb-handle-protection
+igt@kms_getfb@getfb2-handle-protection
+igt@kms_sysfs_edid_timing
+
+
+# S3 - S3
+igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
+
+
+# ASSR - ASSR
+igt@amdgpu/amd_assr@assr-links
+igt@amdgpu/amd_assr@assr-links-dpms
+igt@amdgpu/amd_assr@assr-links-suspend
+
+
+# tests with higher probability of hanging the system
+igt@amdgpu/amd_link_settings@link-training-configs
--
2.25.1


[-- Attachment #2: Type: text/html, Size: 17415 bytes --]

Re: [igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Harry Wentland]

On 2021-10-08 15:02, Rodrigo Siqueira wrote:
> Add various test suites relevant for the amdgpu driver.
> 
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Nicholas Choi <Nicholas.Choi@amd.com>
> Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
> Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
> Cc: Alexander Deucher <alexander.deucher@amd.com>
> Cc: Martin Roukala <martin.roukala@mupuf.org>
> Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Mark Yacoub <markyacoub@google.com>
> 
> Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>

Reviewed-by: Harry Wentland <harry.wentland@amd.com>

Harry

> ---
>  tests/amdgpu-ci/README                 |  31 +++++
>  tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
>  tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
>  3 files changed, 339 insertions(+)
>  create mode 100644 tests/amdgpu-ci/README
>  create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
>  create mode 100644 tests/amdgpu-ci/full-feedback.testlist
> 
> diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
> new file mode 100644
> index 00000000..bd34245c
> --- /dev/null
> +++ b/tests/amdgpu-ci/README
> @@ -0,0 +1,31 @@
> +This directory contains test lists that are used by AMD's CI. The
> +files are passed to piglit with the --test-list parameter directly.
> +
> +The test lists are contained in the IGT repository for several
> +reasons:
> +
> +- The lists stay synchronized with the IGT codebase.
> +- Public availability. Kernel developers can see what tests are run,
> +  and can see what changes are done to the set, when, and why.
> +
> +Changing the test lists should only happen with approval from:
> +- Harry Wentland <harry.wentland@amd.com>
> +- Nicholas Choi <Nicholas.Choi@amd.com>
> +- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
> +- Sun peng (Leo) Li <sunpeng.li@amd.com>
> +
> +======================
> +fast-feedback.testlist
> +======================
> +
> +Fast-feedback contains tests that roughly tests a wide coverage of features in
> +a short time. We use this list for presubmission validation. Results of a
> +fast-feedback test round should only be considered to mean that the kernel is
> +not obviously broken.
> +
> +======================
> +full-feedback.testlist
> +======================
> +
> +This is an extensive set of tests that takes a long time to complete in which
> +we usually use as post-submission.
> diff --git a/tests/amdgpu-ci/fast-feedback.testlist b/tests/amdgpu-ci/fast-feedback.testlist
> new file mode 100644
> index 00000000..025bea16
> --- /dev/null
> +++ b/tests/amdgpu-ci/fast-feedback.testlist
> @@ -0,0 +1,135 @@
> +# tests to be run at the beginning
> +igt@kms_setmode@basic
> +
> +
> +# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
> +igt@amdgpu/amd_hotplug@basic
> +igt@amdgpu/amd_hotplug@basic-suspend
> +igt@kms_bw@linear-tiling-1-displays-1920x1080p
> +
> +
> +# ATOMIC-MODESETTING - Atomic modesetting tests
> +igt@kms_atomic@plane-overlay-legacy
> +igt@kms_atomic@plane-primary-legacy
> +igt@kms_atomic@plane-immutable-zpos
> +igt@kms_atomic@test-only
> +igt@kms_atomic@plane-cursor-legacy
> +igt@kms_atomic@plane-invalid-params
> +igt@kms_atomic@plane-invalid-params-fence
> +igt@kms_atomic@crtc-invalid-params
> +igt@kms_atomic@crtc-invalid-params-fence
> +igt@kms_atomic@atomic-invalid-params
> +igt@kms_content_protection@atomic
> +igt@kms_flip@basic-flip-vs-dpms@a-dp1
> +igt@kms_flip@basic-flip-vs-dpms@c-dp1
> +igt@kms_flip@basic-plain-flip@a-dp1
> +igt@kms_flip@basic-plain-flip@c-dp1
> +igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
> +igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
> +
> +
> +# COLOR - Color Adjustments
> +igt@kms_color@pipe-a-gamma
> +igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
> +igt@kms_color@pipe-a-degamma
> +igt@amdgpu/amd_color@crtc-linear-degamma
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
> +igt@kms_color@pipe-a-ctm-0-25
> +igt@amdgpu/amd_color@crtc-linear-regamma
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
> +igt@kms_color@pipe-a-ctm-negative
> +igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
> +igt@kms_color@pipe-a-ctm-blue-to-red
> +igt@kms_color@pipe-a-ctm-0-75
> +igt@kms_color@pipe-a-legacy-gamma
> +igt@kms_color@pipe-a-ctm-0-5
> +igt@kms_color@pipe-a-ctm-max
> +igt@kms_color@pipe-a-ctm-red-to-blue
> +igt@amdgpu/amd_color@crtc-lut-accuracy
> +igt@kms_color@pipe-a-ctm-green-to-red
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
> +igt@kms_plane_alpha_blend@pipe-a-alpha-basic
> +igt@kms_color@pipe-a-legacy-gamma-reset
> +
> +
> +# VRR
> +igt@kms_vrr@flip-basic
> +igt@kms_vrr@flip-dpms
> +
> +
> +# MODES - Tests mode settings
> +igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
> +igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
> +igt@kms_setmode@basic-clone-single-crtc
> +igt@kms_setmode@invalid-clone-single-crtc
> +igt@kms_setmode@invalid-clone-exclusive-crtc
> +igt@kms_setmode@clone-exclusive-crtc
> +igt@kms_setmode@invalid-clone-single-crtc-stealing
> +
> +
> +# PLANE - DRM plane-level tests
> +igt@kms_plane@plane-panning-top-left@pipe-a-planes
> +igt@kms_cursor_legacy@flip-vs-cursor-atomic
> +igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
> +igt@kms_properties@plane-properties-legacy
> +igt@kms_plane_cursor@pipe-a-primary-size-256
> +igt@kms_plane@plane-position-covered@pipe-a-planes
> +igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
> +igt@kms_plane@plane-position-hole@pipe-b-planes
> +igt@kms_plane_multiple@atomic-pipe-a-tiling-none
> +igt@kms_plane@plane-panning-top-left@pipe-b-planes
> +igt@kms_plane@plane-position-covered@pipe-b-planes
> +igt@kms_plane_cursor@pipe-a-primary-size-128
> +igt@kms_cursor_legacy@cursor-vs-flip-legacy
> +igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
> +igt@kms_plane_multiple@atomic-pipe-c-tiling-none
> +igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
> +igt@kms_plane_cursor@pipe-a-primary-size-64
> +igt@kms_plane@plane-position-hole@pipe-a-planes
> +
> +
> +# DPMS - DPMS
> +igt@kms_content_protection@atomic-dpms
> +
> +
> +# HDCP - HDCP functionality
> +igt@kms_content_protection@legacy
> +
> +
> +# DISPLAY-LAYER - General DRM API tests and display driver tests
> +igt@core_auth@basic-auth
> +igt@kms_prop_blob@basic
> +igt@kms_prop_blob@blob-prop-core
> +igt@kms_prop_blob@blob-prop-validate
> +igt@kms_prop_blob@blob-prop-lifetime
> +igt@kms_prop_blob@blob-multiple
> +igt@kms_prop_blob@invalid-get-prop-any
> +igt@kms_prop_blob@invalid-get-prop
> +igt@kms_prop_blob@invalid-set-prop-any
> +igt@kms_prop_blob@invalid-set-prop
> +igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
> +igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
> +igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
> +igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
> +igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
> +igt@kms_getfb@getfb-handle-zero
> +igt@kms_getfb@getfb-handle-valid
> +igt@kms_getfb@getfb-handle-closed
> +igt@kms_getfb@getfb-handle-not-fb
> +igt@kms_getfb@getfb-addfb-different-handles
> +igt@kms_getfb@getfb-repeated-different-handles
> +igt@kms_getfb@getfb2-handle-zero
> +igt@kms_getfb@getfb2-handle-closed
> +igt@kms_getfb@getfb2-handle-not-fb
> +igt@kms_getfb@getfb2-into-addfb2
> +igt@kms_getfb@getfb-handle-protection
> +igt@kms_getfb@getfb2-handle-protection
> +
> +
> +# S3 - S3
> +igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
> +
> +
> +# ASSR - ASSR
> +igt@amdgpu/amd_assr@assr-links
> +igt@amdgpu/amd_assr@assr-links-dpms
> diff --git a/tests/amdgpu-ci/full-feedback.testlist b/tests/amdgpu-ci/full-feedback.testlist
> new file mode 100644
> index 00000000..cc837d47
> --- /dev/null
> +++ b/tests/amdgpu-ci/full-feedback.testlist
> @@ -0,0 +1,173 @@
> +# tests to be run at the beginning
> +igt@kms_setmode@basic
> +
> +
> +# SINGLE-DISPLAY - Light-up, hotplug, bootup, and general functionality tests for single non tiled display
> +igt@amdgpu/amd_hotplug@basic
> +igt@amdgpu/amd_hotplug@basic-suspend
> +igt@kms_bw@linear-tiling-1-displays-1920x1080p
> +
> +
> +# ATOMIC-MODESETTING - Atomic modesetting tests
> +igt@kms_atomic@plane-overlay-legacy
> +igt@kms_atomic@plane-primary-legacy
> +igt@kms_atomic@plane-immutable-zpos
> +igt@kms_atomic@test-only
> +igt@kms_atomic@plane-cursor-legacy
> +igt@kms_atomic@plane-invalid-params
> +igt@kms_atomic@plane-invalid-params-fence
> +igt@kms_atomic@crtc-invalid-params
> +igt@kms_atomic@crtc-invalid-params-fence
> +igt@kms_atomic@atomic-invalid-params
> +igt@kms_content_protection@atomic
> +igt@kms_flip@basic-flip-vs-dpms@a-dp1
> +igt@kms_flip@basic-flip-vs-dpms@c-dp1
> +igt@kms_flip@basic-plain-flip@a-dp1
> +igt@kms_flip@basic-plain-flip@c-dp1
> +igt@kms_flip@flip-vs-dpms-off-vs-modeset@a-dp1
> +igt@kms_flip@flip-vs-dpms-off-vs-modeset@c-dp1
> +igt@kms_flip_event_leak
> +
> +
> +# COLOR - Color Adjustments
> +igt@kms_color@pipe-a-gamma
> +igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb
> +igt@kms_color@pipe-a-degamma
> +igt@amdgpu/amd_color@crtc-linear-degamma
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-mid
> +igt@kms_color@pipe-a-ctm-0-25
> +igt@amdgpu/amd_color@crtc-linear-regamma
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-max
> +igt@kms_color@pipe-a-ctm-negative
> +igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb
> +igt@kms_color@pipe-a-ctm-blue-to-red
> +igt@kms_color@pipe-a-ctm-0-75
> +igt@kms_color@pipe-a-legacy-gamma
> +igt@kms_color@pipe-a-ctm-0-5
> +igt@kms_color@pipe-a-ctm-max
> +igt@kms_color@pipe-a-ctm-red-to-blue
> +igt@amdgpu/amd_color@crtc-lut-accuracy
> +igt@kms_color@pipe-a-ctm-green-to-red
> +igt@kms_plane_alpha_blend@pipe-a-constant-alpha-min
> +igt@kms_plane_alpha_blend@pipe-a-alpha-basic
> +igt@kms_color@pipe-a-legacy-gamma-reset
> +
> +
> +# VRR
> +igt@kms_vrr@flip-basic
> +igt@kms_vrr@flip-dpms
> +igt@amdgpu/amd_vrr_range@freesync-parsing-dp
> +igt@amdgpu/amd_vrr_range@freesync-parsing-dp-suspend
> +igt@amdgpu/amd_vrr_range@freesync-range-dp
> +igt@amdgpu/amd_vrr_range@freesync-range-dp-suspend
> +
> +
> +# MODES - Tests mode settings
> +igt@kms_plane_scaling@plane-scaling@pipe-a-plane-scaling
> +igt@kms_plane_scaling@plane-scaling@pipe-c-plane-scaling
> +igt@kms_setmode@basic-clone-single-crtc
> +igt@kms_setmode@invalid-clone-single-crtc
> +igt@kms_setmode@invalid-clone-exclusive-crtc
> +igt@kms_setmode@clone-exclusive-crtc
> +igt@kms_setmode@invalid-clone-single-crtc-stealing
> +
> +
> +# PLANE - DRM plane-level tests
> +igt@kms_plane@plane-panning-top-left@pipe-a-planes
> +igt@kms_cursor_legacy@flip-vs-cursor-atomic
> +igt@kms_plane@plane-position-hole-dpms@pipe-a-planes
> +igt@kms_properties@plane-properties-legacy
> +igt@kms_plane_cursor@pipe-a-primary-size-256
> +igt@kms_plane@plane-position-covered@pipe-a-planes
> +igt@kms_plane@plane-position-hole-dpms@pipe-b-planes
> +igt@kms_plane@plane-position-hole@pipe-b-planes
> +igt@kms_plane_multiple@atomic-pipe-a-tiling-none
> +igt@kms_plane@plane-panning-top-left@pipe-b-planes
> +igt@kms_plane@plane-position-covered@pipe-b-planes
> +igt@kms_plane_cursor@pipe-a-primary-size-128
> +igt@kms_cursor_legacy@cursor-vs-flip-legacy
> +igt@kms_plane@plane-panning-bottom-right@pipe-b-planes
> +igt@kms_plane_multiple@atomic-pipe-c-tiling-none
> +igt@kms_plane@plane-panning-bottom-right@pipe-a-planes
> +igt@kms_plane_cursor@pipe-a-primary-size-64
> +igt@kms_plane@plane-position-hole@pipe-a-planes
> +igt@kms_cursor_crc@pipe-a-cursor-size-change
> +igt@kms_cursor_crc@pipe-b-cursor-size-change
> +igt@kms_cursor_crc@pipe-a-cursor-alpha-opaque
> +igt@kms_cursor_crc@pipe-b-cursor-alpha-opaque
> +igt@kms_cursor_crc@pipe-a-cursor-alpha-transparent
> +igt@kms_cursor_crc@pipe-b-cursor-alpha-transparent
> +igt@kms_cursor_crc@pipe-a-cursor-dpms
> +igt@kms_cursor_crc@pipe-b-cursor-dpms
> +igt@kms_cursor_crc@pipe-a-cursor-suspend
> +igt@kms_cursor_crc@pipe-b-cursor-suspend
> +igt@kms_cursor_crc@pipe-a-cursor-max-size-onscreen
> +igt@kms_cursor_crc@pipe-b-cursor-max-size-onscreen
> +igt@kms_cursor_crc@pipe-a-cursor-max-size-offscreen
> +igt@kms_cursor_crc@pipe-b-cursor-max-size-offscreen
> +igt@kms_cursor_crc@pipe-a-cursor-max-size-sliding
> +igt@kms_cursor_crc@pipe-b-cursor-max-size-sliding
> +igt@kms_cursor_crc@pipe-a-cursor-max-size-random
> +igt@kms_cursor_crc@pipe-b-cursor-max-size-random
> +igt@kms_cursor_crc@pipe-a-cursor-max-size-rapid-movement
> +igt@kms_cursor_crc@pipe-b-cursor-max-size-rapid-movement
> +igt@kms_concurrent@pipe-a
> +igt@kms_concurrent@pipe-b
> +
> +
> +# DPMS - DPMS
> +igt@kms_content_protection@atomic-dpms
> +
> +
> +# HDCP - HDCP functionality
> +igt@kms_content_protection@legacy
> +
> +
> +# DISPLAY-LAYER - General DRM API tests and display driver tests
> +igt@core_auth@basic-auth
> +igt@kms_prop_blob@basic
> +igt@kms_prop_blob@blob-prop-core
> +igt@kms_prop_blob@blob-prop-validate
> +igt@kms_prop_blob@blob-prop-lifetime
> +igt@kms_prop_blob@blob-multiple
> +igt@kms_prop_blob@invalid-get-prop-any
> +igt@kms_prop_blob@invalid-get-prop
> +igt@kms_prop_blob@invalid-set-prop-any
> +igt@kms_prop_blob@invalid-set-prop
> +igt@kms_vblank@pipe-a-ts-continuation-suspend
> +igt@amdgpu/amd_mem_leak@connector-suspend-resume
> +igt@amdgpu/amd_mem_leak@connector-hotplug
> +igt@kms_atomic_interruptible@legacy-setmode@dp-1-pipe-a
> +igt@kms_atomic_interruptible@atomic-setmode@dp-1-pipe-a
> +igt@kms_atomic_interruptible@legacy-dpms@dp-1-pipe-a
> +igt@kms_atomic_interruptible@legacy-pageflip@dp-1-pipe-a
> +igt@kms_atomic_interruptible@legacy-cursor@dp-1-pipe-a
> +igt@kms_atomic_interruptible@universal-setplane-primary@dp-1-pipe-a
> +igt@kms_atomic_interruptible@universal-setplane-cursor@dp-1-pipe-a
> +igt@kms_getfb@getfb-handle-zero
> +igt@kms_getfb@getfb-handle-valid
> +igt@kms_getfb@getfb-handle-closed
> +igt@kms_getfb@getfb-handle-not-fb
> +igt@kms_getfb@getfb-addfb-different-handles
> +igt@kms_getfb@getfb-repeated-different-handles
> +igt@kms_getfb@getfb2-handle-zero
> +igt@kms_getfb@getfb2-handle-closed
> +igt@kms_getfb@getfb2-handle-not-fb
> +igt@kms_getfb@getfb2-into-addfb2
> +igt@kms_getfb@getfb-handle-protection
> +igt@kms_getfb@getfb2-handle-protection
> +igt@kms_sysfs_edid_timing
> +
> +
> +# S3 - S3
> +igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes
> +
> +
> +# ASSR - ASSR
> +igt@amdgpu/amd_assr@assr-links
> +igt@amdgpu/amd_assr@assr-links-dpms
> +igt@amdgpu/amd_assr@assr-links-suspend
> +
> +
> +# tests with higher probability of hanging the system
> +igt@amdgpu/amd_link_settings@link-training-configs
> 

[igt-dev] ✗ Fi.CI.BAT: failure for tests: Add amdgpu test suite (rev2)

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 6014 bytes --]

== Series Details ==

Series: tests: Add amdgpu test suite (rev2)
URL   : https://patchwork.freedesktop.org/series/95626/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_10700 -> IGTPW_6304
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with IGTPW_6304 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_6304, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/index.html

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_6304:

### IGT changes ###

#### Possible regressions ####

  * igt@core_hotunplug@unbind-rebind:
    - fi-tgl-u2:          NOTRUN -> [INCOMPLETE][1]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@core_hotunplug@unbind-rebind.html

  
#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@i915_selftest@live@hangcheck:
    - {fi-hsw-gt1}:       [PASS][2] -> [DMESG-FAIL][3]
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10700/fi-hsw-gt1/igt@i915_selftest@live@hangcheck.html
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-hsw-gt1/igt@i915_selftest@live@hangcheck.html

  
Known issues
------------

  Here are the changes found in IGTPW_6304 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@amdgpu/amd_basic@cs-gfx:
    - fi-kbl-soraka:      NOTRUN -> [SKIP][4] ([fdo#109271]) +4 similar issues
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-kbl-soraka/igt@amdgpu/amd_basic@cs-gfx.html

  * igt@amdgpu/amd_cs_nop@sync-fork-compute0:
    - fi-snb-2600:        NOTRUN -> [SKIP][5] ([fdo#109271]) +17 similar issues
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-snb-2600/igt@amdgpu/amd_cs_nop@sync-fork-compute0.html

  * igt@gem_huc_copy@huc-copy:
    - fi-tgl-u2:          NOTRUN -> [SKIP][6] ([i915#2190])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@gem_huc_copy@huc-copy.html

  * igt@kms_chamelium@dp-hpd-fast:
    - fi-tgl-u2:          NOTRUN -> [SKIP][7] ([fdo#109284] / [fdo#111827]) +8 similar issues
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@kms_chamelium@dp-hpd-fast.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic:
    - fi-tgl-u2:          NOTRUN -> [SKIP][8] ([i915#4103]) +1 similar issue
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic.html

  * igt@kms_force_connector_basic@force-load-detect:
    - fi-tgl-u2:          NOTRUN -> [SKIP][9] ([fdo#109285])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@kms_force_connector_basic@force-load-detect.html

  * igt@prime_vgem@basic-userptr:
    - fi-tgl-u2:          NOTRUN -> [SKIP][10] ([i915#3301])
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@prime_vgem@basic-userptr.html

  * igt@runner@aborted:
    - fi-bdw-5557u:       NOTRUN -> [FAIL][11] ([i915#1602] / [i915#2029])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-bdw-5557u/igt@runner@aborted.html
    - fi-tgl-u2:          NOTRUN -> [FAIL][12] ([i915#1602] / [i915#2722])
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-u2/igt@runner@aborted.html

  
#### Possible fixes ####

  * igt@i915_selftest@live@hangcheck:
    - fi-snb-2600:        [INCOMPLETE][13] ([i915#3921]) -> [PASS][14]
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10700/fi-snb-2600/igt@i915_selftest@live@hangcheck.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-snb-2600/igt@i915_selftest@live@hangcheck.html

  * igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a:
    - {fi-tgl-dsi}:       [DMESG-WARN][15] ([i915#1982]) -> [PASS][16]
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10700/fi-tgl-dsi/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/fi-tgl-dsi/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-a.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109284]: https://bugs.freedesktop.org/show_bug.cgi?id=109284
  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1602]: https://gitlab.freedesktop.org/drm/intel/issues/1602
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#2029]: https://gitlab.freedesktop.org/drm/intel/issues/2029
  [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
  [i915#2722]: https://gitlab.freedesktop.org/drm/intel/issues/2722
  [i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
  [i915#3921]: https://gitlab.freedesktop.org/drm/intel/issues/3921
  [i915#4103]: https://gitlab.freedesktop.org/drm/intel/issues/4103


Participating hosts (40 -> 38)
------------------------------

  Additional (1): fi-tgl-u2 
  Missing    (3): fi-ilk-m540 fi-bsw-cyan fi-hsw-4200u 


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_6240 -> IGTPW_6304

  CI-20190529: 20190529
  CI_DRM_10700: 6ecdd5e29c83cd8fc191f8cce5c283eefb53c97e @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_6304: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/index.html
  IGT_6240: b232a092b9e1b10a8be13601acaa440903b226bc @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_6304/index.html

[-- Attachment #2: Type: text/html, Size: 7091 bytes --]

Re: [igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Martin Roukala (néé Peres)]

On 08/10/2021 22:02, Rodrigo Siqueira wrote:
> Add various test suites relevant for the amdgpu driver.
> 
> Cc: Harry Wentland <harry.wentland@amd.com>
> Cc: Nicholas Choi <Nicholas.Choi@amd.com>
> Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
> Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
> Cc: Alexander Deucher <alexander.deucher@amd.com>
> Cc: Martin Roukala <martin.roukala@mupuf.org>
> Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
> Cc: Simon Ser <contact@emersion.fr>
> Cc: Mark Yacoub <markyacoub@google.com>
> 
> Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
> ---
>   tests/amdgpu-ci/README                 |  31 +++++
>   tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
>   tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
>   3 files changed, 339 insertions(+)
>   create mode 100644 tests/amdgpu-ci/README
>   create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
>   create mode 100644 tests/amdgpu-ci/full-feedback.testlist
> 
> diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
> new file mode 100644
> index 00000000..bd34245c
> --- /dev/null
> +++ b/tests/amdgpu-ci/README
> @@ -0,0 +1,31 @@
> +This directory contains test lists that are used by AMD's CI. The
> +files are passed to piglit with the --test-list parameter directly.
> +
> +The test lists are contained in the IGT repository for several
> +reasons:
> +
> +- The lists stay synchronized with the IGT codebase.
> +- Public availability. Kernel developers can see what tests are run,
> +  and can see what changes are done to the set, when, and why.

Thanks for doing this!

> +
> +Changing the test lists should only happen with approval from:
> +- Harry Wentland <harry.wentland@amd.com>
> +- Nicholas Choi <Nicholas.Choi@amd.com>
> +- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
> +- Sun peng (Leo) Li <sunpeng.li@amd.com>
> +
> +======================
> +fast-feedback.testlist
> +======================
> +
> +Fast-feedback contains tests that roughly tests a wide coverage of features in
> +a short time. We use this list for presubmission validation. Results of a
> +fast-feedback test round should only be considered to mean that the kernel is
> +not obviously broken.

It would be good to have a target execution time for this list. ~10 
minutes and 12 minutes in the worst case has been working for Intel, but 
what would you like to use?

> +
> +======================
> +full-feedback.testlist
> +======================
> +
> +This is an extensive set of tests that takes a long time to complete in which
> +we usually use as post-submission.

 From experience, I would specify the full list as a opt-out, rather 
than opt-in. Otherwise, you will soon realize that it takes a lot of 
time and energy to maintain the list to always add new tests there.

By opting out of i915/gem/nouveau/udl/... tests, you will at least get 
every new KMS test by default.

Is that making sense?

Martin

Re: [igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Rodrigo Siqueira Jordao]

On 2021-10-11 4:44 a.m., Martin Roukala (néé Peres) wrote:
> 
> 
> On 08/10/2021 22:02, Rodrigo Siqueira wrote:
>> Add various test suites relevant for the amdgpu driver.
>>
>> Cc: Harry Wentland <harry.wentland@amd.com>
>> Cc: Nicholas Choi <Nicholas.Choi@amd.com>
>> Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>> Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
>> Cc: Alexander Deucher <alexander.deucher@amd.com>
>> Cc: Martin Roukala <martin.roukala@mupuf.org>
>> Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
>> Cc: Simon Ser <contact@emersion.fr>
>> Cc: Mark Yacoub <markyacoub@google.com>
>>
>> Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>> ---
>>   tests/amdgpu-ci/README                 |  31 +++++
>>   tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
>>   tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
>>   3 files changed, 339 insertions(+)
>>   create mode 100644 tests/amdgpu-ci/README
>>   create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
>>   create mode 100644 tests/amdgpu-ci/full-feedback.testlist
>>
>> diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
>> new file mode 100644
>> index 00000000..bd34245c
>> --- /dev/null
>> +++ b/tests/amdgpu-ci/README
>> @@ -0,0 +1,31 @@
>> +This directory contains test lists that are used by AMD's CI. The
>> +files are passed to piglit with the --test-list parameter directly.
>> +
>> +The test lists are contained in the IGT repository for several
>> +reasons:
>> +
>> +- The lists stay synchronized with the IGT codebase.
>> +- Public availability. Kernel developers can see what tests are run,
>> +  and can see what changes are done to the set, when, and why.
> 
> Thanks for doing this!
> 
>> +
>> +Changing the test lists should only happen with approval from:
>> +- Harry Wentland <harry.wentland@amd.com>
>> +- Nicholas Choi <Nicholas.Choi@amd.com>
>> +- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>> +- Sun peng (Leo) Li <sunpeng.li@amd.com>
>> +
>> +======================
>> +fast-feedback.testlist
>> +======================
>> +
>> +Fast-feedback contains tests that roughly tests a wide coverage of 
>> features in
>> +a short time. We use this list for presubmission validation. Results 
>> of a
>> +fast-feedback test round should only be considered to mean that the 
>> kernel is
>> +not obviously broken.
> 
> It would be good to have a target execution time for this list. ~10 
> minutes and 12 minutes in the worst case has been working for Intel, but 
> what would you like to use?

Good point,

We already use this testlist in our pre-submission patches.
Nicholas Choi and Hayden, from your experience, what is a good execution 
time for this list?

>> +
>> +======================
>> +full-feedback.testlist
>> +======================
>> +
>> +This is an extensive set of tests that takes a long time to complete 
>> in which
>> +we usually use as post-submission.
> 
>  From experience, I would specify the full list as a opt-out, rather 
> than opt-in. Otherwise, you will soon realize that it takes a lot of 
> time and energy to maintain the list to always add new tests there.
> 
> By opting out of i915/gem/nouveau/udl/... tests, you will at least get 
> every new KMS test by default.
> 
> Is that making sense?

Hmmm... Could you elaborate a little bit more on how I can use the 
opt-out behavior?

Also, I suppose that opting out will enable all tests, which means that 
we will have a lot of failures in this report for AMD, right? We still 
have a lot of IGT tests that only pass on Intel hardware (we are trying 
to fix that), and I'm wondering if it really makes sense to enable 
everything for AMD right now due to this limitation. Additionally, we 
use this full-feeback list in our nightly build to identify regressions 
in our amd-staging-drm-next; that's why we usually don't want to run 
tests that fail in that.

Thanks
Siqueira

> Martin

Re: [igt-dev] [PATCH i-g-t] tests: Add amdgpu test suite

[Martin Roukala (néé Peres)]

On 14/10/2021 17:47, Rodrigo Siqueira Jordao wrote:
> 
> 
> On 2021-10-11 4:44 a.m., Martin Roukala (néé Peres) wrote:
>>
>>
>> On 08/10/2021 22:02, Rodrigo Siqueira wrote:
>>> Add various test suites relevant for the amdgpu driver.
>>>
>>> Cc: Harry Wentland <harry.wentland@amd.com>
>>> Cc: Nicholas Choi <Nicholas.Choi@amd.com>
>>> Cc: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>>> Cc: Sun peng (Leo) Li <sunpeng.li@amd.com>
>>> Cc: Alexander Deucher <alexander.deucher@amd.com>
>>> Cc: Martin Roukala <martin.roukala@mupuf.org>
>>> Cc: Hayden Goodfellow <hayden.goodfellow@amd.com>
>>> Cc: Simon Ser <contact@emersion.fr>
>>> Cc: Mark Yacoub <markyacoub@google.com>
>>>
>>> Signed-off-by: Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>>> ---
>>>   tests/amdgpu-ci/README                 |  31 +++++
>>>   tests/amdgpu-ci/fast-feedback.testlist | 135 +++++++++++++++++++
>>>   tests/amdgpu-ci/full-feedback.testlist | 173 +++++++++++++++++++++++++
>>>   3 files changed, 339 insertions(+)
>>>   create mode 100644 tests/amdgpu-ci/README
>>>   create mode 100644 tests/amdgpu-ci/fast-feedback.testlist
>>>   create mode 100644 tests/amdgpu-ci/full-feedback.testlist
>>>
>>> diff --git a/tests/amdgpu-ci/README b/tests/amdgpu-ci/README
>>> new file mode 100644
>>> index 00000000..bd34245c
>>> --- /dev/null
>>> +++ b/tests/amdgpu-ci/README
>>> @@ -0,0 +1,31 @@
>>> +This directory contains test lists that are used by AMD's CI. The
>>> +files are passed to piglit with the --test-list parameter directly.
>>> +
>>> +The test lists are contained in the IGT repository for several
>>> +reasons:
>>> +
>>> +- The lists stay synchronized with the IGT codebase.
>>> +- Public availability. Kernel developers can see what tests are run,
>>> +  and can see what changes are done to the set, when, and why.
>>
>> Thanks for doing this!
>>
>>> +
>>> +Changing the test lists should only happen with approval from:
>>> +- Harry Wentland <harry.wentland@amd.com>
>>> +- Nicholas Choi <Nicholas.Choi@amd.com>
>>> +- Rodrigo Siqueira <Rodrigo.Siqueira@amd.com>
>>> +- Sun peng (Leo) Li <sunpeng.li@amd.com>
>>> +
>>> +======================
>>> +fast-feedback.testlist
>>> +======================
>>> +
>>> +Fast-feedback contains tests that roughly tests a wide coverage of 
>>> features in
>>> +a short time. We use this list for presubmission validation. Results 
>>> of a
>>> +fast-feedback test round should only be considered to mean that the 
>>> kernel is
>>> +not obviously broken.
>>
>> It would be good to have a target execution time for this list. ~10 
>> minutes and 12 minutes in the worst case has been working for Intel, 
>> but what would you like to use?
> 
> Good point,
> 
> We already use this testlist in our pre-submission patches.
> Nicholas Choi and Hayden, from your experience, what is a good execution 
> time for this list?
> 
>>> +
>>> +======================
>>> +full-feedback.testlist
>>> +======================
>>> +
>>> +This is an extensive set of tests that takes a long time to complete 
>>> in which
>>> +we usually use as post-submission.
>>
>>  From experience, I would specify the full list as a opt-out, rather 
>> than opt-in. Otherwise, you will soon realize that it takes a lot of 
>> time and energy to maintain the list to always add new tests there.
>>
>> By opting out of i915/gem/nouveau/udl/... tests, you will at least get 
>> every new KMS test by default.
>>
>> Is that making sense?
> 
> Hmmm... Could you elaborate a little bit more on how I can use the 
> opt-out behavior?

Sure, `igt_runner -b blacklist.txt` will prevent running any test that 
matches the regex...

Here is an example: 
https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/blob/master/tests/intel-ci/blacklist.txt



> 
> Also, I suppose that opting out will enable all tests, which means that 
> we will have a lot of failures in this report for AMD, right? We still 
> have a lot of IGT tests that only pass on Intel hardware (we are trying 
> to fix that), and I'm wondering if it really makes sense to enable 
> everything for AMD right now due to this limitation. Additionally, we 
> use this full-feeback list in our nightly build to identify regressions 
> in our amd-staging-drm-next; that's why we usually don't want to run 
> tests that fail in that.

Running only the "known to work" tests is a pitfall of testing. Run 
everything (unless it breaks your CI due to execution time), document 
failures in bugs, then address bugs. You'll increase your testing 
massively, and catch a lot more issues than your curated list.

Cheers,
Martin

> 
> Thanks
> Siqueira
> 
>> Martin
>