All of lore.kernel.org
 help / color / mirror / Atom feed
* [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
@ 2022-09-13 15:08 Hamza Mahfooz
  2022-09-13 16:46 ` [igt-dev] ✓ Fi.CI.BAT: success for tests/kms_setmode: degrade assert to debug (rev2) Patchwork
                   ` (3 more replies)
  0 siblings, 4 replies; 9+ messages in thread
From: Hamza Mahfooz @ 2022-09-13 15:08 UTC (permalink / raw)
  To: Development mailing list for IGT GPU Tools; +Cc: Hamza Mahfooz

Unfortunately, there are too many sources of jitter that can cause the
vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
off by more than a scanline for this assert to be useful. So, to allow
this test to pass consistently degrade the assert to a debug.

Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
---
v2: fix logic
---
 tests/kms_setmode.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
index bfa10891..58abfe14 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
 	 * See:
 	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
 	 */
-	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
-		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
-		     expected, mean, stddev,
-		     fabs(mean - expected),
-		     fabs(mean - expected) / stddev,
-		     fabs(mean - expected) / line_time(kmode));
+	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
+		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
+			 expected, mean, stddev,
+			 fabs(mean - expected),
+			 fabs(mean - expected) / stddev,
+			 fabs(mean - expected) / line_time(kmode));
 }
 
 static void test_crtc_config(const struct test_config *tconf,
-- 
2.37.2

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [igt-dev] ✓ Fi.CI.BAT: success for tests/kms_setmode: degrade assert to debug (rev2)
  2022-09-13 15:08 [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
@ 2022-09-13 16:46 ` Patchwork
  2022-09-14  4:00 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2022-09-13 16:46 UTC (permalink / raw)
  To: Hamza Mahfooz; +Cc: igt-dev

[-- Attachment #1: Type: text/plain, Size: 4788 bytes --]

== Series Details ==

Series: tests/kms_setmode: degrade assert to debug (rev2)
URL   : https://patchwork.freedesktop.org/series/108459/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12129 -> IGTPW_7779
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html

Participating hosts (44 -> 42)
------------------------------

  Missing    (2): fi-ctg-p8600 fi-bdw-samus 

Known issues
------------

  Here are the changes found in IGTPW_7779 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@i915_selftest@live@execlists:
    - fi-bdw-gvtdvm:      [PASS][1] -> [INCOMPLETE][2] ([i915#2940])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-bdw-gvtdvm/igt@i915_selftest@live@execlists.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-bdw-gvtdvm/igt@i915_selftest@live@execlists.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions-varying-size:
    - fi-bsw-kefka:       [PASS][3] -> [FAIL][4] ([i915#6298])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-bsw-kefka/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions-varying-size.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-bsw-kefka/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions-varying-size.html

  * igt@runner@aborted:
    - fi-bdw-gvtdvm:      NOTRUN -> [FAIL][5] ([i915#4312])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-bdw-gvtdvm/igt@runner@aborted.html

  
#### Possible fixes ####

  * igt@i915_selftest@live@late_gt_pm:
    - fi-cfl-8109u:       [DMESG-WARN][6] ([i915#5904]) -> [PASS][7] +35 similar issues
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-cfl-8109u/igt@i915_selftest@live@late_gt_pm.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-cfl-8109u/igt@i915_selftest@live@late_gt_pm.html

  * igt@i915_suspend@basic-s2idle-without-i915:
    - fi-cfl-8109u:       [DMESG-WARN][8] ([i915#5904] / [i915#62]) -> [PASS][9] +3 similar issues
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-cfl-8109u/igt@i915_suspend@basic-s2idle-without-i915.html
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-cfl-8109u/igt@i915_suspend@basic-s2idle-without-i915.html

  * igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions:
    - fi-bsw-kefka:       [FAIL][10] ([i915#6298]) -> [PASS][11]
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-bsw-kefka/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions.html
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-bsw-kefka/igt@kms_cursor_legacy@basic-busy-flip-before-cursor@atomic-transitions.html

  * igt@kms_flip@basic-plain-flip@c-dp2:
    - fi-cfl-8109u:       [DMESG-WARN][12] ([i915#165]) -> [PASS][13]
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-cfl-8109u/igt@kms_flip@basic-plain-flip@c-dp2.html
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-cfl-8109u/igt@kms_flip@basic-plain-flip@c-dp2.html

  * igt@kms_frontbuffer_tracking@basic:
    - fi-cfl-8109u:       [DMESG-WARN][14] ([i915#165] / [i915#62]) -> [PASS][15] +13 similar issues
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/fi-cfl-8109u/igt@kms_frontbuffer_tracking@basic.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/fi-cfl-8109u/igt@kms_frontbuffer_tracking@basic.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [i915#165]: https://gitlab.freedesktop.org/drm/intel/issues/165
  [i915#2867]: https://gitlab.freedesktop.org/drm/intel/issues/2867
  [i915#2940]: https://gitlab.freedesktop.org/drm/intel/issues/2940
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#5904]: https://gitlab.freedesktop.org/drm/intel/issues/5904
  [i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
  [i915#6298]: https://gitlab.freedesktop.org/drm/intel/issues/6298
  [i915#6670]: https://gitlab.freedesktop.org/drm/intel/issues/6670


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_6653 -> IGTPW_7779

  CI-20190529: 20190529
  CI_DRM_12129: 173a9b6d1e3242215f323c9e57f656af50b7bd73 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_7779: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html
  IGT_6653: 4f927248ebbf11f03f4c1ea2254f011e7575322f @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html

[-- Attachment #2: Type: text/html, Size: 5719 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [igt-dev] ✓ Fi.CI.IGT: success for tests/kms_setmode: degrade assert to debug (rev2)
  2022-09-13 15:08 [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
  2022-09-13 16:46 ` [igt-dev] ✓ Fi.CI.BAT: success for tests/kms_setmode: degrade assert to debug (rev2) Patchwork
@ 2022-09-14  4:00 ` Patchwork
  2022-09-15 16:06 ` [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
  2022-09-16 10:59 ` Petri Latvala
  3 siblings, 0 replies; 9+ messages in thread
From: Patchwork @ 2022-09-14  4:00 UTC (permalink / raw)
  To: Hamza Mahfooz; +Cc: igt-dev

[-- Attachment #1: Type: text/plain, Size: 37327 bytes --]

== Series Details ==

Series: tests/kms_setmode: degrade assert to debug (rev2)
URL   : https://patchwork.freedesktop.org/series/108459/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12129_full -> IGTPW_7779_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html

Participating hosts (10 -> 8)
------------------------------

  Additional (1): shard-rkl 
  Missing    (3): pig-skl-6260u pig-kbl-iris pig-glk-j5005 

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_7779_full:

### IGT changes ###

#### Suppressed ####

  The following results come from untrusted machines, tests, or statuses.
  They do not affect the overall result.

  * igt@kms_cursor_legacy@torture-bo@all-pipes:
    - {shard-rkl}:        NOTRUN -> [INCOMPLETE][1]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-rkl-5/igt@kms_cursor_legacy@torture-bo@all-pipes.html

  
Known issues
------------

  Here are the changes found in IGTPW_7779_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@feature_discovery@psr2:
    - shard-iclb:         [PASS][2] -> [SKIP][3] ([i915#658])
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb2/igt@feature_discovery@psr2.html
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb1/igt@feature_discovery@psr2.html

  * igt@gem_ccs@ctrl-surf-copy:
    - shard-tglb:         NOTRUN -> [SKIP][4] ([i915#3555] / [i915#5325])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@gem_ccs@ctrl-surf-copy.html
    - shard-iclb:         NOTRUN -> [SKIP][5] ([i915#5327])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb6/igt@gem_ccs@ctrl-surf-copy.html

  * igt@gem_ctx_persistence@engines-hostile-preempt:
    - shard-snb:          NOTRUN -> [SKIP][6] ([fdo#109271] / [i915#1099])
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-snb6/igt@gem_ctx_persistence@engines-hostile-preempt.html

  * igt@gem_eio@unwedge-stress:
    - shard-tglb:         [PASS][7] -> [TIMEOUT][8] ([i915#3063])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb5/igt@gem_eio@unwedge-stress.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_balancer@parallel-keep-in-fence:
    - shard-iclb:         [PASS][9] -> [SKIP][10] ([i915#4525]) +1 similar issue
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb4/igt@gem_exec_balancer@parallel-keep-in-fence.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb7/igt@gem_exec_balancer@parallel-keep-in-fence.html

  * igt@gem_exec_fair@basic-none-share@rcs0:
    - shard-iclb:         [PASS][11] -> [FAIL][12] ([i915#2842])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb8/igt@gem_exec_fair@basic-none-share@rcs0.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@gem_exec_fair@basic-none-share@rcs0.html

  * igt@gem_exec_fair@basic-none@vcs1:
    - shard-iclb:         NOTRUN -> [FAIL][13] ([i915#2842])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@gem_exec_fair@basic-none@vcs1.html

  * igt@gem_exec_fair@basic-pace-share@rcs0:
    - shard-glk:          [PASS][14] -> [FAIL][15] ([i915#2842]) +2 similar issues
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk8/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk9/igt@gem_exec_fair@basic-pace-share@rcs0.html
    - shard-tglb:         [PASS][16] -> [FAIL][17] ([i915#2842])
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb5/igt@gem_exec_fair@basic-pace-share@rcs0.html
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@gem_exec_fair@basic-pace-share@rcs0.html

  * igt@gem_lmem_swapping@heavy-verify-random:
    - shard-apl:          NOTRUN -> [SKIP][18] ([fdo#109271] / [i915#4613]) +1 similar issue
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl2/igt@gem_lmem_swapping@heavy-verify-random.html
    - shard-tglb:         NOTRUN -> [SKIP][19] ([i915#4613]) +1 similar issue
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb1/igt@gem_lmem_swapping@heavy-verify-random.html
    - shard-glk:          NOTRUN -> [SKIP][20] ([fdo#109271] / [i915#4613]) +1 similar issue
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk5/igt@gem_lmem_swapping@heavy-verify-random.html
    - shard-iclb:         NOTRUN -> [SKIP][21] ([i915#4613]) +1 similar issue
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@gem_lmem_swapping@heavy-verify-random.html

  * igt@gen9_exec_parse@cmd-crossing-page:
    - shard-tglb:         NOTRUN -> [SKIP][22] ([i915#2527] / [i915#2856])
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb3/igt@gen9_exec_parse@cmd-crossing-page.html
    - shard-iclb:         NOTRUN -> [SKIP][23] ([i915#2856])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb4/igt@gen9_exec_parse@cmd-crossing-page.html

  * igt@i915_pm_dc@dc6-dpms:
    - shard-iclb:         [PASS][24] -> [FAIL][25] ([i915#454])
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb7/igt@i915_pm_dc@dc6-dpms.html
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@i915_pm_dc@dc6-dpms.html

  * igt@i915_pm_dc@dc9-dpms:
    - shard-apl:          [PASS][26] -> [SKIP][27] ([fdo#109271])
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl8/igt@i915_pm_dc@dc9-dpms.html
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl8/igt@i915_pm_dc@dc9-dpms.html

  * igt@i915_pm_rps@engine-order:
    - shard-apl:          [PASS][28] -> [FAIL][29] ([i915#6537])
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl7/igt@i915_pm_rps@engine-order.html
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl7/igt@i915_pm_rps@engine-order.html

  * igt@kms_big_fb@y-tiled-8bpp-rotate-90:
    - shard-iclb:         NOTRUN -> [SKIP][30] ([fdo#110725] / [fdo#111614])
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@kms_big_fb@y-tiled-8bpp-rotate-90.html
    - shard-tglb:         NOTRUN -> [SKIP][31] ([fdo#111614])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb1/igt@kms_big_fb@y-tiled-8bpp-rotate-90.html

  * igt@kms_big_fb@yf-tiled-8bpp-rotate-0:
    - shard-iclb:         NOTRUN -> [SKIP][32] ([fdo#110723])
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb4/igt@kms_big_fb@yf-tiled-8bpp-rotate-0.html
    - shard-tglb:         NOTRUN -> [SKIP][33] ([fdo#111615])
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb3/igt@kms_big_fb@yf-tiled-8bpp-rotate-0.html

  * igt@kms_ccs@pipe-a-crc-primary-basic-4_tiled_dg2_mc_ccs:
    - shard-iclb:         NOTRUN -> [SKIP][34] ([fdo#109278]) +2 similar issues
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@kms_ccs@pipe-a-crc-primary-basic-4_tiled_dg2_mc_ccs.html
    - shard-tglb:         NOTRUN -> [SKIP][35] ([i915#3689] / [i915#6095])
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb3/igt@kms_ccs@pipe-a-crc-primary-basic-4_tiled_dg2_mc_ccs.html

  * igt@kms_ccs@pipe-b-crc-primary-rotation-180-y_tiled_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][36] ([i915#3689])
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb2/igt@kms_ccs@pipe-b-crc-primary-rotation-180-y_tiled_ccs.html

  * igt@kms_ccs@pipe-d-crc-primary-rotation-180-4_tiled_dg2_rc_ccs_cc:
    - shard-tglb:         NOTRUN -> [SKIP][37] ([i915#6095])
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb5/igt@kms_ccs@pipe-d-crc-primary-rotation-180-4_tiled_dg2_rc_ccs_cc.html

  * igt@kms_chamelium@dp-hpd-for-each-pipe:
    - shard-glk:          NOTRUN -> [SKIP][38] ([fdo#109271] / [fdo#111827]) +1 similar issue
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk1/igt@kms_chamelium@dp-hpd-for-each-pipe.html
    - shard-iclb:         NOTRUN -> [SKIP][39] ([fdo#109284] / [fdo#111827]) +1 similar issue
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb7/igt@kms_chamelium@dp-hpd-for-each-pipe.html

  * igt@kms_chamelium@vga-hpd-after-suspend:
    - shard-apl:          NOTRUN -> [SKIP][40] ([fdo#109271] / [fdo#111827]) +2 similar issues
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl1/igt@kms_chamelium@vga-hpd-after-suspend.html
    - shard-snb:          NOTRUN -> [SKIP][41] ([fdo#109271] / [fdo#111827]) +1 similar issue
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-snb6/igt@kms_chamelium@vga-hpd-after-suspend.html
    - shard-tglb:         NOTRUN -> [SKIP][42] ([fdo#109284] / [fdo#111827]) +1 similar issue
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@kms_chamelium@vga-hpd-after-suspend.html

  * igt@kms_cursor_legacy@cursor-vs-flip@atomic-transitions:
    - shard-iclb:         [PASS][43] -> [FAIL][44] ([i915#5072])
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb4/igt@kms_cursor_legacy@cursor-vs-flip@atomic-transitions.html
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb7/igt@kms_cursor_legacy@cursor-vs-flip@atomic-transitions.html

  * igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size:
    - shard-glk:          [PASS][45] -> [FAIL][46] ([i915#2346])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk6/igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size.html
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk2/igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size.html

  * igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2:
    - shard-glk:          [PASS][47] -> [FAIL][48] ([i915#2122])
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk5/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk8/igt@kms_flip@2x-flip-vs-expired-vblank-interruptible@ab-hdmi-a1-hdmi-a2.html

  * igt@kms_flip@2x-flip-vs-modeset:
    - shard-tglb:         NOTRUN -> [SKIP][49] ([fdo#109274] / [fdo#111825] / [i915#3637])
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@kms_flip@2x-flip-vs-modeset.html
    - shard-iclb:         NOTRUN -> [SKIP][50] ([fdo#109274])
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb5/igt@kms_flip@2x-flip-vs-modeset.html

  * igt@kms_flip_scaled_crc@flip-32bpp-4tile-to-32bpp-4tiledg2rcccs-upscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][51] ([i915#2672])
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@kms_flip_scaled_crc@flip-32bpp-4tile-to-32bpp-4tiledg2rcccs-upscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-4tile-to-32bpp-4tile-downscaling@pipe-a-valid-mode:
    - shard-iclb:         NOTRUN -> [SKIP][52] ([i915#2587] / [i915#2672]) +3 similar issues
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb6/igt@kms_flip_scaled_crc@flip-64bpp-4tile-to-32bpp-4tile-downscaling@pipe-a-valid-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-linear-to-16bpp-linear-downscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][53] ([i915#3555]) +1 similar issue
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@kms_flip_scaled_crc@flip-64bpp-linear-to-16bpp-linear-downscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilegen12rcccs-upscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][54] ([i915#2672] / [i915#3555]) +1 similar issue
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilegen12rcccs-upscaling@pipe-a-default-mode.html

  * igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-gtt:
    - shard-iclb:         NOTRUN -> [SKIP][55] ([fdo#109280]) +4 similar issues
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb3/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-gtt.html

  * igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-spr-indfb-onoff:
    - shard-apl:          NOTRUN -> [SKIP][56] ([fdo#109271]) +46 similar issues
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl7/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-spr-indfb-onoff.html

  * igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-spr-indfb-move:
    - shard-tglb:         NOTRUN -> [SKIP][57] ([i915#6497]) +2 similar issues
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-spr-indfb-move.html

  * igt@kms_frontbuffer_tracking@psr-2p-primscrn-pri-indfb-draw-blt:
    - shard-tglb:         NOTRUN -> [SKIP][58] ([fdo#109280] / [fdo#111825]) +4 similar issues
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@kms_frontbuffer_tracking@psr-2p-primscrn-pri-indfb-draw-blt.html

  * igt@kms_pipe_b_c_ivb@pipe-b-double-modeset-then-modeset-pipe-c:
    - shard-iclb:         NOTRUN -> [SKIP][59] ([fdo#109289])
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb1/igt@kms_pipe_b_c_ivb@pipe-b-double-modeset-then-modeset-pipe-c.html
    - shard-tglb:         NOTRUN -> [SKIP][60] ([fdo#109289])
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb2/igt@kms_pipe_b_c_ivb@pipe-b-double-modeset-then-modeset-pipe-c.html

  * igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb:
    - shard-apl:          NOTRUN -> [FAIL][61] ([i915#265])
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl1/igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb.html
    - shard-glk:          NOTRUN -> [FAIL][62] ([i915#265])
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk9/igt@kms_plane_alpha_blend@pipe-a-alpha-transparent-fb.html

  * igt@kms_psr@psr2_cursor_mmap_gtt:
    - shard-iclb:         [PASS][63] -> [SKIP][64] ([fdo#109441]) +1 similar issue
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb2/igt@kms_psr@psr2_cursor_mmap_gtt.html
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb6/igt@kms_psr@psr2_cursor_mmap_gtt.html

  * igt@kms_psr_stress_test@flip-primary-invalidate-overlay:
    - shard-tglb:         [PASS][65] -> [SKIP][66] ([i915#5519])
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb1/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html
    - shard-iclb:         [PASS][67] -> [SKIP][68] ([i915#5519])
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb5/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb6/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html

  * igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90:
    - shard-tglb:         NOTRUN -> [SKIP][69] ([fdo#111615] / [i915#5289])
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90.html

  * igt@kms_vblank@pipe-c-query-busy-hang:
    - shard-snb:          NOTRUN -> [SKIP][70] ([fdo#109271]) +33 similar issues
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-snb6/igt@kms_vblank@pipe-c-query-busy-hang.html

  * igt@perf_pmu@event-wait@rcs0:
    - shard-iclb:         NOTRUN -> [SKIP][71] ([fdo#112283])
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb8/igt@perf_pmu@event-wait@rcs0.html
    - shard-tglb:         NOTRUN -> [SKIP][72] ([fdo#112283])
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@perf_pmu@event-wait@rcs0.html

  * igt@perf_pmu@module-unload:
    - shard-snb:          [PASS][73] -> [DMESG-WARN][74] ([i915#4528])
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-snb6/igt@perf_pmu@module-unload.html
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-snb4/igt@perf_pmu@module-unload.html

  * igt@prime_nv_pcopy@test3_3:
    - shard-glk:          NOTRUN -> [SKIP][75] ([fdo#109271]) +16 similar issues
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk3/igt@prime_nv_pcopy@test3_3.html
    - shard-iclb:         NOTRUN -> [SKIP][76] ([fdo#109291])
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@prime_nv_pcopy@test3_3.html
    - shard-tglb:         NOTRUN -> [SKIP][77] ([fdo#109291])
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb5/igt@prime_nv_pcopy@test3_3.html

  * igt@sysfs_clients@busy:
    - shard-tglb:         NOTRUN -> [SKIP][78] ([i915#2994])
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@sysfs_clients@busy.html
    - shard-glk:          NOTRUN -> [SKIP][79] ([fdo#109271] / [i915#2994])
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk7/igt@sysfs_clients@busy.html
    - shard-iclb:         NOTRUN -> [SKIP][80] ([i915#2994])
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb6/igt@sysfs_clients@busy.html
    - shard-apl:          NOTRUN -> [SKIP][81] ([fdo#109271] / [i915#2994])
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl4/igt@sysfs_clients@busy.html

  * igt@sysfs_timeslice_duration@timeout@vecs0:
    - shard-apl:          [PASS][82] -> [FAIL][83] ([i915#1755])
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl2/igt@sysfs_timeslice_duration@timeout@vecs0.html
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl2/igt@sysfs_timeslice_duration@timeout@vecs0.html

  
#### Possible fixes ####

  * igt@gem_ctx_exec@basic-nohangcheck:
    - shard-tglb:         [FAIL][84] ([i915#6268]) -> [PASS][85]
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb5/igt@gem_ctx_exec@basic-nohangcheck.html
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb6/igt@gem_ctx_exec@basic-nohangcheck.html
    - {shard-tglu}:       [FAIL][86] ([i915#6268]) -> [PASS][87]
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglu-4/igt@gem_ctx_exec@basic-nohangcheck.html
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglu-3/igt@gem_ctx_exec@basic-nohangcheck.html

  * igt@gem_eio@kms:
    - shard-tglb:         [FAIL][88] ([i915#5784]) -> [PASS][89]
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb3/igt@gem_eio@kms.html
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb7/igt@gem_eio@kms.html

  * igt@gem_eio@unwedge-stress:
    - shard-iclb:         [TIMEOUT][90] ([i915#3070]) -> [PASS][91]
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb6/igt@gem_eio@unwedge-stress.html
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb7/igt@gem_eio@unwedge-stress.html
    - {shard-tglu}:       [TIMEOUT][92] ([i915#3063]) -> [PASS][93]
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglu-4/igt@gem_eio@unwedge-stress.html
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglu-3/igt@gem_eio@unwedge-stress.html

  * igt@gem_exec_balancer@parallel-out-fence:
    - shard-iclb:         [SKIP][94] ([i915#4525]) -> [PASS][95]
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb3/igt@gem_exec_balancer@parallel-out-fence.html
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb1/igt@gem_exec_balancer@parallel-out-fence.html

  * igt@gem_exec_fair@basic-none-share@rcs0:
    - shard-glk:          [FAIL][96] ([i915#2842]) -> [PASS][97]
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk1/igt@gem_exec_fair@basic-none-share@rcs0.html
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk5/igt@gem_exec_fair@basic-none-share@rcs0.html

  * igt@gem_exec_fair@basic-pace-solo@rcs0:
    - shard-apl:          [FAIL][98] ([i915#2842]) -> [PASS][99]
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl1/igt@gem_exec_fair@basic-pace-solo@rcs0.html
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl8/igt@gem_exec_fair@basic-pace-solo@rcs0.html

  * igt@gem_workarounds@suspend-resume-fd:
    - shard-apl:          [DMESG-WARN][100] ([i915#180]) -> [PASS][101]
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl6/igt@gem_workarounds@suspend-resume-fd.html
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl6/igt@gem_workarounds@suspend-resume-fd.html

  * igt@kms_big_fb@y-tiled-64bpp-rotate-180:
    - shard-glk:          [FAIL][102] ([i915#1888] / [i915#5138]) -> [PASS][103]
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk1/igt@kms_big_fb@y-tiled-64bpp-rotate-180.html
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk7/igt@kms_big_fb@y-tiled-64bpp-rotate-180.html

  * igt@kms_flip@flip-vs-blocking-wf-vblank@a-hdmi-a1:
    - shard-glk:          [FAIL][104] ([i915#2122]) -> [PASS][105]
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-glk7/igt@kms_flip@flip-vs-blocking-wf-vblank@a-hdmi-a1.html
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-glk3/igt@kms_flip@flip-vs-blocking-wf-vblank@a-hdmi-a1.html

  * igt@kms_hdmi_inject@inject-audio:
    - shard-tglb:         [SKIP][106] ([i915#433]) -> [PASS][107]
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-tglb6/igt@kms_hdmi_inject@inject-audio.html
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-tglb2/igt@kms_hdmi_inject@inject-audio.html

  * igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1:
    - shard-iclb:         [SKIP][108] ([i915#5176]) -> [PASS][109] +1 similar issue
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb3/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1.html
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb5/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1.html

  * igt@kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5@pipe-b-edp-1:
    - shard-iclb:         [SKIP][110] ([i915#5235]) -> [PASS][111] +2 similar issues
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb2/igt@kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5@pipe-b-edp-1.html
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb1/igt@kms_plane_scaling@planes-upscale-20x20-downscale-factor-0-5@pipe-b-edp-1.html

  * igt@kms_psr@psr2_cursor_blt:
    - shard-iclb:         [SKIP][112] ([fdo#109441]) -> [PASS][113] +1 similar issue
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb7/igt@kms_psr@psr2_cursor_blt.html
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@kms_psr@psr2_cursor_blt.html

  * igt@kms_setmode@basic@pipe-a-vga-1:
    - shard-snb:          [FAIL][114] ([i915#5465]) -> [PASS][115] +1 similar issue
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-snb2/igt@kms_setmode@basic@pipe-a-vga-1.html
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-snb2/igt@kms_setmode@basic@pipe-a-vga-1.html

  
#### Warnings ####

  * igt@gem_exec_balancer@parallel-ordering:
    - shard-iclb:         [FAIL][116] ([i915#6117]) -> [SKIP][117] ([i915#4525])
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb2/igt@gem_exec_balancer@parallel-ordering.html
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb5/igt@gem_exec_balancer@parallel-ordering.html

  * igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-fully-sf:
    - shard-iclb:         [SKIP][118] ([i915#658]) -> [SKIP][119] ([i915#2920])
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb5/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-fully-sf.html
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-fully-sf.html

  * igt@kms_psr2_sf@overlay-plane-update-continuous-sf:
    - shard-iclb:         [SKIP][120] ([fdo#111068] / [i915#658]) -> [SKIP][121] ([i915#2920])
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb6/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html

  * igt@kms_psr2_su@page_flip-p010:
    - shard-iclb:         [SKIP][122] ([fdo#109642] / [fdo#111068] / [i915#658]) -> [FAIL][123] ([i915#5939])
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-iclb6/igt@kms_psr2_su@page_flip-p010.html
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-iclb2/igt@kms_psr2_su@page_flip-p010.html

  * igt@runner@aborted:
    - shard-apl:          ([FAIL][124], [FAIL][125], [FAIL][126]) ([fdo#109271] / [i915#180] / [i915#3002] / [i915#4312] / [i915#5257] / [i915#6599]) -> [FAIL][127] ([i915#3002] / [i915#4312] / [i915#5257] / [i915#6599])
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl2/igt@runner@aborted.html
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl8/igt@runner@aborted.html
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12129/shard-apl6/igt@runner@aborted.html
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/shard-apl1/igt@runner@aborted.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109283]: https://bugs.freedesktop.org/show_bug.cgi?id=109283
  [fdo#109284]: https://bugs.freedesktop.org/show_bug.cgi?id=109284
  [fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109291]: https://bugs.freedesktop.org/show_bug.cgi?id=109291
  [fdo#109295]: https://bugs.freedesktop.org/show_bug.cgi?id=109295
  [fdo#109300]: https://bugs.freedesktop.org/show_bug.cgi?id=109300
  [fdo#109302]: https://bugs.freedesktop.org/show_bug.cgi?id=109302
  [fdo#109303]: https://bugs.freedesktop.org/show_bug.cgi?id=109303
  [fdo#109308]: https://bugs.freedesktop.org/show_bug.cgi?id=109308
  [fdo#109309]: https://bugs.freedesktop.org/show_bug.cgi?id=109309
  [fdo#109312]: https://bugs.freedesktop.org/show_bug.cgi?id=109312
  [fdo#109313]: https://bugs.freedesktop.org/show_bug.cgi?id=109313
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#109506]: https://bugs.freedesktop.org/show_bug.cgi?id=109506
  [fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
  [fdo#110189]: https://bugs.freedesktop.org/show_bug.cgi?id=110189
  [fdo#110254]: https://bugs.freedesktop.org/show_bug.cgi?id=110254
  [fdo#110542]: https://bugs.freedesktop.org/show_bug.cgi?id=110542
  [fdo#110723]: https://bugs.freedesktop.org/show_bug.cgi?id=110723
  [fdo#110725]: https://bugs.freedesktop.org/show_bug.cgi?id=110725
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111614]: https://bugs.freedesktop.org/show_bug.cgi?id=111614
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111644]: https://bugs.freedesktop.org/show_bug.cgi?id=111644
  [fdo#111656]: https://bugs.freedesktop.org/show_bug.cgi?id=111656
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [fdo#112283]: https://bugs.freedesktop.org/show_bug.cgi?id=112283
  [i915#1063]: https://gitlab.freedesktop.org/drm/intel/issues/1063
  [i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
  [i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
  [i915#1155]: https://gitlab.freedesktop.org/drm/intel/issues/1155
  [i915#1257]: https://gitlab.freedesktop.org/drm/intel/issues/1257
  [i915#132]: https://gitlab.freedesktop.org/drm/intel/issues/132
  [i915#1397]: https://gitlab.freedesktop.org/drm/intel/issues/1397
  [i915#1722]: https://gitlab.freedesktop.org/drm/intel/issues/1722
  [i915#1755]: https://gitlab.freedesktop.org/drm/intel/issues/1755
  [i915#1769]: https://gitlab.freedesktop.org/drm/intel/issues/1769
  [i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
  [i915#1825]: https://gitlab.freedesktop.org/drm/intel/issues/1825
  [i915#1839]: https://gitlab.freedesktop.org/drm/intel/issues/1839
  [i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
  [i915#1849]: https://gitlab.freedesktop.org/drm/intel/issues/1849
  [i915#1850]: https://gitlab.freedesktop.org/drm/intel/issues/1850
  [i915#1888]: https://gitlab.freedesktop.org/drm/intel/issues/1888
  [i915#1902]: https://gitlab.freedesktop.org/drm/intel/issues/1902
  [i915#2122]: https://gitlab.freedesktop.org/drm/intel/issues/2122
  [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2410]: https://gitlab.freedesktop.org/drm/intel/issues/2410
  [i915#2437]: https://gitlab.freedesktop.org/drm/intel/issues/2437
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2530]: https://gitlab.freedesktop.org/drm/intel/issues/2530
  [i915#2582]: https://gitlab.freedesktop.org/drm/intel/issues/2582
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
  [i915#2658]: https://gitlab.freedesktop.org/drm/intel/issues/2658
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#280]: https://gitlab.freedesktop.org/drm/intel/issues/280
  [i915#284]: https://gitlab.freedesktop.org/drm/intel/issues/284
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
  [i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
  [i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
  [i915#3012]: https://gitlab.freedesktop.org/drm/intel/issues/3012
  [i915#3063]: https://gitlab.freedesktop.org/drm/intel/issues/3063
  [i915#3070]: https://gitlab.freedesktop.org/drm/intel/issues/3070
  [i915#3116]: https://gitlab.freedesktop.org/drm/intel/issues/3116
  [i915#3281]: https://gitlab.freedesktop.org/drm/intel/issues/3281
  [i915#3282]: https://gitlab.freedesktop.org/drm/intel/issues/3282
  [i915#3291]: https://gitlab.freedesktop.org/drm/intel/issues/3291
  [i915#3297]: https://gitlab.freedesktop.org/drm/intel/issues/3297
  [i915#3299]: https://gitlab.freedesktop.org/drm/intel/issues/3299
  [i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
  [i915#3318]: https://gitlab.freedesktop.org/drm/intel/issues/3318
  [i915#3323]: https://gitlab.freedesktop.org/drm/intel/issues/3323
  [i915#3359]: https://gitlab.freedesktop.org/drm/intel/issues/3359
  [i915#3376]: https://gitlab.freedesktop.org/drm/intel/issues/3376
  [i915#3536]: https://gitlab.freedesktop.org/drm/intel/issues/3536
  [i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3558]: https://gitlab.freedesktop.org/drm/intel/issues/3558
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3638]: https://gitlab.freedesktop.org/drm/intel/issues/3638
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3692]: https://gitlab.freedesktop.org/drm/intel/issues/3692
  [i915#3708]: https://gitlab.freedesktop.org/drm/intel/issues/3708
  [i915#3734]: https://gitlab.freedesktop.org/drm/intel/issues/3734
  [i915#3742]: https://gitlab.freedesktop.org/drm/intel/issues/3742
  [i915#3804]: https://gitlab.freedesktop.org/drm/intel/issues/3804
  [i915#3810]: https://gitlab.freedesktop.org/drm/intel/issues/3810
  [i915#3826]: https://gitlab.freedesktop.org/drm/intel/issues/3826
  [i915#3828]: https://gitlab.freedesktop.org/drm/intel/issues/3828
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3955]: https://gitlab.freedesktop.org/drm/intel/issues/3955
  [i915#3966]: https://gitlab.freedesktop.org/drm/intel/issues/3966
  [i915#3989]: https://gitlab.freedesktop.org/drm/intel/issues/3989
  [i915#4016]: https://gitlab.freedesktop.org/drm/intel/issues/4016
  [i915#4070]: https://gitlab.freedesktop.org/drm/intel/issues/4070
  [i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
  [i915#426]: https://gitlab.freedesktop.org/drm/intel/issues/426
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#433]: https://gitlab.freedesktop.org/drm/intel/issues/433
  [i915#4387]: https://gitlab.freedesktop.org/drm/intel/issues/4387
  [i915#4525]: https://gitlab.freedesktop.org/drm/intel/issues/4525
  [i915#4528]: https://gitlab.freedesktop.org/drm/intel/issues/4528
  [i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4877]: https://gitlab.freedesktop.org/drm/intel/issues/4877
  [i915#4991]: https://gitlab.freedesktop.org/drm/intel/issues/4991
  [i915#5072]: https://gitlab.freedesktop.org/drm/intel/issues/5072
  [i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5235]: https://gitlab.freedesktop.org/drm/intel/issues/5235
  [i915#5257]: https://gitlab.freedesktop.org/drm/intel/issues/5257
  [i915#5286]: https://gitlab.freedesktop.org/drm/intel/issues/5286
  [i915#5289]: https://gitlab.freedesktop.org/drm/intel/issues/5289
  [i915#5325]: https://gitlab.freedesktop.org/drm/intel/issues/5325
  [i915#5327]: https://gitlab.freedesktop.org/drm/intel/issues/5327
  [i915#533]: https://gitlab.freedesktop.org/drm/intel/issues/533
  [i915#5439]: https://gitlab.freedesktop.org/drm/intel/issues/5439
  [i915#5461]: https://gitlab.freedesktop.org/drm/intel/issues/5461
  [i915#5465]: https://gitlab.freedesktop.org/drm/intel/issues/5465
  [i915#5519]: https://gitlab.freedesktop.org/drm/intel/issues/5519
  [i915#5723]: https://gitlab.freedesktop.org/drm/intel/issues/5723
  [i915#5784]: https://gitlab.freedesktop.org/drm/intel/issues/5784
  [i915#5939]: https://gitlab.freedesktop.org/drm/intel/issues/5939
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6117]: https://gitlab.freedesktop.org/drm/intel/issues/6117
  [i915#6227]: https://gitlab.freedesktop.org/drm/intel/issues/6227
  [i915#6230]: https://gitlab.freedesktop.org/drm/intel/issues/6230
  [i915#6245]: https://gitlab.freedesktop.org/drm/intel/issues/6245
  [i915#6268]: https://gitlab.freedesktop.org/drm/intel/issues/6268
  [i915#6301]: https://gitlab.freedesktop.org/drm/intel/issues/6301
  [i915#6334]: https://gitlab.freedesktop.org/drm/intel/issues/6334
  [i915#6335]: https://gitlab.freedesktop.org/drm/intel/issues/6335
  [i915#6344]: https://gitlab.freedesktop.org/drm/intel/issues/6344
  [i915#6412]: https://gitlab.freedesktop.org/drm/intel/issues/6412
  [i915#6433]: https://gitlab.freedesktop.org/drm/intel/issues/6433
  [i915#6497]: https://gitlab.freedesktop.org/drm/intel/issues/6497
  [i915#6524]: https://gitlab.freedesktop.org/drm/intel/issues/6524
  [i915#6537]: https://gitlab.freedesktop.org/drm/intel/issues/6537
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#6599]: https://gitlab.freedesktop.org/drm/intel/issues/6599


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_6653 -> IGTPW_7779
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_12129: 173a9b6d1e3242215f323c9e57f656af50b7bd73 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_7779: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html
  IGT_6653: 4f927248ebbf11f03f4c1ea2254f011e7575322f @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7779/index.html

[-- Attachment #2: Type: text/html, Size: 38358 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-13 15:08 [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
  2022-09-13 16:46 ` [igt-dev] ✓ Fi.CI.BAT: success for tests/kms_setmode: degrade assert to debug (rev2) Patchwork
  2022-09-14  4:00 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
@ 2022-09-15 16:06 ` Hamza Mahfooz
  2022-09-15 18:13   ` Rodrigo Siqueira Jordao
  2022-09-16 10:59 ` Petri Latvala
  3 siblings, 1 reply; 9+ messages in thread
From: Hamza Mahfooz @ 2022-09-15 16:06 UTC (permalink / raw)
  To: Development mailing list for IGT GPU Tools; +Cc: Petri Latvala, hamza.mahfooz

[Adding Petri and Siqueira]

On 2022-09-13 11:08, Hamza Mahfooz wrote:
> Unfortunately, there are too many sources of jitter that can cause the
> vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
> off by more than a scanline for this assert to be useful. So, to allow
> this test to pass consistently degrade the assert to a debug.
> 
> Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
> ---
> v2: fix logic
> ---
>   tests/kms_setmode.c | 12 ++++++------
>   1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> index bfa10891..58abfe14 100644
> --- a/tests/kms_setmode.c
> +++ b/tests/kms_setmode.c
> @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
>   	 * See:
>   	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
>   	 */
> -	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
> -		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> -		     expected, mean, stddev,
> -		     fabs(mean - expected),
> -		     fabs(mean - expected) / stddev,
> -		     fabs(mean - expected) / line_time(kmode));
> +	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
> +		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> +			 expected, mean, stddev,
> +			 fabs(mean - expected),
> +			 fabs(mean - expected) / stddev,
> +			 fabs(mean - expected) / line_time(kmode));
>   }
>   
>   static void test_crtc_config(const struct test_config *tconf,

-- 
Hamza

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-15 16:06 ` [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
@ 2022-09-15 18:13   ` Rodrigo Siqueira Jordao
  0 siblings, 0 replies; 9+ messages in thread
From: Rodrigo Siqueira Jordao @ 2022-09-15 18:13 UTC (permalink / raw)
  To: Hamza Mahfooz, Mark Yacoub
  Cc: Development mailing list for IGT GPU Tools, Petri Latvala

+Mark

On 2022-09-15 12:06, Hamza Mahfooz wrote:
> [Adding Petri and Siqueira]
> 
> On 2022-09-13 11:08, Hamza Mahfooz wrote:
>> Unfortunately, there are too many sources of jitter that can cause the
>> vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
>> off by more than a scanline for this assert to be useful. So, to allow
>> this test to pass consistently degrade the assert to a debug.
>>
>> Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
>> ---
>> v2: fix logic
>> ---
>>   tests/kms_setmode.c | 12 ++++++------
>>   1 file changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
>> index bfa10891..58abfe14 100644
>> --- a/tests/kms_setmode.c
>> +++ b/tests/kms_setmode.c
>> @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const 
>> drmModeModeInfo *kmode)
>>        * See:
>>        * 
>> https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
>>        */
>> -    igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 
>> * stddev),
>> -             "vblank interval differs from modeline! expected %.1fus, 
>> measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f 
>> scanlines)\n",
>> -             expected, mean, stddev,
>> -             fabs(mean - expected),
>> -             fabs(mean - expected) / stddev,
>> -             fabs(mean - expected) / line_time(kmode));
>> +    if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
>> +        igt_info("vblank interval differs from modeline! expected 
>> %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f 
>> scanlines)\n",
>> +             expected, mean, stddev,
>> +             fabs(mean - expected),
>> +             fabs(mean - expected) / stddev,
>> +             fabs(mean - expected) / line_time(kmode));
>>   }
>>   static void test_crtc_config(const struct test_config *tconf,
> 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-13 15:08 [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
                   ` (2 preceding siblings ...)
  2022-09-15 16:06 ` [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
@ 2022-09-16 10:59 ` Petri Latvala
  2022-09-16 11:47   ` Ville Syrjälä
  3 siblings, 1 reply; 9+ messages in thread
From: Petri Latvala @ 2022-09-16 10:59 UTC (permalink / raw)
  To: Hamza Mahfooz, Ville Syrjälä
  Cc: Development mailing list for IGT GPU Tools

On Tue, Sep 13, 2022 at 11:08:55AM -0400, Hamza Mahfooz wrote:
> Unfortunately, there are too many sources of jitter that can cause the
> vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
> off by more than a scanline for this assert to be useful. So, to allow
> this test to pass consistently degrade the assert to a debug.
> 
> Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
> ---
> v2: fix logic
> ---
>  tests/kms_setmode.c | 12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> index bfa10891..58abfe14 100644
> --- a/tests/kms_setmode.c
> +++ b/tests/kms_setmode.c
> @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
>  	 * See:
>  	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
>  	 */
> -	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
> -		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> -		     expected, mean, stddev,
> -		     fabs(mean - expected),
> -		     fabs(mean - expected) / stddev,
> -		     fabs(mean - expected) / line_time(kmode));
> +	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
> +		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> +			 expected, mean, stddev,
> +			 fabs(mean - expected),
> +			 fabs(mean - expected) / stddev,
> +			 fabs(mean - expected) / line_time(kmode));
>  }


History has taught that having such messages in the logs are generally
useless. Without a mechanism to tell that the output has changed, it's
going to get missed and when you realize something is completely off,
figuring out when it changed is a ghost hunt.

In short, the action "to allow this test to pass" is effectively
deleting the test.

Having said that, deleting the test might be the right move. I said
people will miss changes if it's just a message, but it has been an
assertion failure for i915 CI for some time... But a consistent
one. The difference on SNB looks to be 2.5 scanlines, sometimes 2.1,
but most often 2.5.

Maybe Ville is able to comment? What's up with DRM_IOCTL_WAIT_VBLANK
timings, is this assert obsolete with today's kernel? Or does the test
need some kind of tuning?

-- 
Petri Latvala

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-16 10:59 ` Petri Latvala
@ 2022-09-16 11:47   ` Ville Syrjälä
  2022-09-16 11:56     ` Ville Syrjälä
  0 siblings, 1 reply; 9+ messages in thread
From: Ville Syrjälä @ 2022-09-16 11:47 UTC (permalink / raw)
  To: Petri Latvala; +Cc: Development mailing list for IGT GPU Tools, Hamza Mahfooz

On Fri, Sep 16, 2022 at 01:59:38PM +0300, Petri Latvala wrote:
> On Tue, Sep 13, 2022 at 11:08:55AM -0400, Hamza Mahfooz wrote:
> > Unfortunately, there are too many sources of jitter that can cause the
> > vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
> > off by more than a scanline for this assert to be useful. So, to allow
> > this test to pass consistently degrade the assert to a debug.
> > 
> > Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
> > ---
> > v2: fix logic
> > ---
> >  tests/kms_setmode.c | 12 ++++++------
> >  1 file changed, 6 insertions(+), 6 deletions(-)
> > 
> > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> > index bfa10891..58abfe14 100644
> > --- a/tests/kms_setmode.c
> > +++ b/tests/kms_setmode.c
> > @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> >  	 * See:
> >  	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
> >  	 */
> > -	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
> > -		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > -		     expected, mean, stddev,
> > -		     fabs(mean - expected),
> > -		     fabs(mean - expected) / stddev,
> > -		     fabs(mean - expected) / line_time(kmode));
> > +	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
> > +		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > +			 expected, mean, stddev,
> > +			 fabs(mean - expected),
> > +			 fabs(mean - expected) / stddev,
> > +			 fabs(mean - expected) / line_time(kmode));
> >  }
> 
> 
> History has taught that having such messages in the logs are generally
> useless. Without a mechanism to tell that the output has changed, it's
> going to get missed and when you realize something is completely off,
> figuring out when it changed is a ghost hunt.
> 
> In short, the action "to allow this test to pass" is effectively
> deleting the test.
> 
> Having said that, deleting the test might be the right move. I said
> people will miss changes if it's just a message, but it has been an
> assertion failure for i915 CI for some time... But a consistent
> one. The difference on SNB looks to be 2.5 scanlines, sometimes 2.1,
> but most often 2.5.
> 
> Maybe Ville is able to comment? What's up with DRM_IOCTL_WAIT_VBLANK
> timings, is this assert obsolete with today's kernel? Or does the test
> need some kind of tuning?

It depends on how good of a match to the requested clock the PLL can give
us. Generally for Intel hardware modern platforms tends to have finer
granularity whereas older stuff can be a bit coarse. Though there are
ways (namely clock bending) by which we could increase the accuracy even
old some older platforms, but no one has bothered to implement that so far.

And no, I don't think deleting the test is the right answer. We
certainly want to catch regressions with it on the platforms where
we have accurate clocks. Ideally I guess we'd somehow try to catch
regressions on the less accurate platforms too, but not sure there's
any good way to do that beyond maintaining some database on what the
expected accuracy is for each CI machine. Relaxing the requirements
across the board would risk not catching smaller regressions on the
accurate platforms.

In its current form the test should fail or succeed consistently.
If not then there's likely some kind of problem with the way the
driver or hardware operates.

-- 
Ville Syrjälä
Intel

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-16 11:47   ` Ville Syrjälä
@ 2022-09-16 11:56     ` Ville Syrjälä
  2022-09-16 12:11       ` Ville Syrjälä
  0 siblings, 1 reply; 9+ messages in thread
From: Ville Syrjälä @ 2022-09-16 11:56 UTC (permalink / raw)
  To: Petri Latvala; +Cc: Development mailing list for IGT GPU Tools, Hamza Mahfooz

On Fri, Sep 16, 2022 at 02:47:57PM +0300, Ville Syrjälä wrote:
> On Fri, Sep 16, 2022 at 01:59:38PM +0300, Petri Latvala wrote:
> > On Tue, Sep 13, 2022 at 11:08:55AM -0400, Hamza Mahfooz wrote:
> > > Unfortunately, there are too many sources of jitter that can cause the
> > > vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
> > > off by more than a scanline for this assert to be useful. So, to allow
> > > this test to pass consistently degrade the assert to a debug.
> > > 
> > > Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
> > > ---
> > > v2: fix logic
> > > ---
> > >  tests/kms_setmode.c | 12 ++++++------
> > >  1 file changed, 6 insertions(+), 6 deletions(-)
> > > 
> > > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> > > index bfa10891..58abfe14 100644
> > > --- a/tests/kms_setmode.c
> > > +++ b/tests/kms_setmode.c
> > > @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> > >  	 * See:
> > >  	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
> > >  	 */
> > > -	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
> > > -		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > > -		     expected, mean, stddev,
> > > -		     fabs(mean - expected),
> > > -		     fabs(mean - expected) / stddev,
> > > -		     fabs(mean - expected) / line_time(kmode));
> > > +	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
> > > +		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > > +			 expected, mean, stddev,
> > > +			 fabs(mean - expected),
> > > +			 fabs(mean - expected) / stddev,
> > > +			 fabs(mean - expected) / line_time(kmode));
> > >  }
> > 
> > 
> > History has taught that having such messages in the logs are generally
> > useless. Without a mechanism to tell that the output has changed, it's
> > going to get missed and when you realize something is completely off,
> > figuring out when it changed is a ghost hunt.
> > 
> > In short, the action "to allow this test to pass" is effectively
> > deleting the test.
> > 
> > Having said that, deleting the test might be the right move. I said
> > people will miss changes if it's just a message, but it has been an
> > assertion failure for i915 CI for some time... But a consistent
> > one. The difference on SNB looks to be 2.5 scanlines, sometimes 2.1,
> > but most often 2.5.
> > 
> > Maybe Ville is able to comment? What's up with DRM_IOCTL_WAIT_VBLANK
> > timings, is this assert obsolete with today's kernel? Or does the test
> > need some kind of tuning?
> 
> It depends on how good of a match to the requested clock the PLL can give
> us. Generally for Intel hardware modern platforms tends to have finer
> granularity whereas older stuff can be a bit coarse. Though there are
> ways (namely clock bending) by which we could increase the accuracy even
> old some older platforms, but no one has bothered to implement that so far.
> 
> And no, I don't think deleting the test is the right answer. We
> certainly want to catch regressions with it on the platforms where
> we have accurate clocks. Ideally I guess we'd somehow try to catch
> regressions on the less accurate platforms too, but not sure there's
> any good way to do that beyond maintaining some database on what the
> expected accuracy is for each CI machine. Relaxing the requirements
> across the board would risk not catching smaller regressions on the
> accurate platforms.

One idea would be to make the kernel report back the actual frequency
somehow, and then the test could a) make sure it's reasonably close to
what was requested, and b) the vblank timings are really accurate when
it comes to the reported actual frequency.

> 
> In its current form the test should fail or succeed consistently.
> If not then there's likely some kind of problem with the way the
> driver or hardware operates.
> 
> -- 
> Ville Syrjälä
> Intel

-- 
Ville Syrjälä
Intel

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug
  2022-09-16 11:56     ` Ville Syrjälä
@ 2022-09-16 12:11       ` Ville Syrjälä
  0 siblings, 0 replies; 9+ messages in thread
From: Ville Syrjälä @ 2022-09-16 12:11 UTC (permalink / raw)
  To: Petri Latvala; +Cc: Development mailing list for IGT GPU Tools, Hamza Mahfooz

On Fri, Sep 16, 2022 at 02:56:18PM +0300, Ville Syrjälä wrote:
> On Fri, Sep 16, 2022 at 02:47:57PM +0300, Ville Syrjälä wrote:
> > On Fri, Sep 16, 2022 at 01:59:38PM +0300, Petri Latvala wrote:
> > > On Tue, Sep 13, 2022 at 11:08:55AM -0400, Hamza Mahfooz wrote:
> > > > Unfortunately, there are too many sources of jitter that can cause the
> > > > vblank interval approximation obtained from DRM_IOCTL_WAIT_VBLANK to be
> > > > off by more than a scanline for this assert to be useful. So, to allow
> > > > this test to pass consistently degrade the assert to a debug.
> > > > 
> > > > Signed-off-by: Hamza Mahfooz <hamza.mahfooz@amd.com>
> > > > ---
> > > > v2: fix logic
> > > > ---
> > > >  tests/kms_setmode.c | 12 ++++++------
> > > >  1 file changed, 6 insertions(+), 6 deletions(-)
> > > > 
> > > > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> > > > index bfa10891..58abfe14 100644
> > > > --- a/tests/kms_setmode.c
> > > > +++ b/tests/kms_setmode.c
> > > > @@ -527,12 +527,12 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> > > >  	 * See:
> > > >  	 * https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
> > > >  	 */
> > > > -	igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
> > > > -		     "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > > > -		     expected, mean, stddev,
> > > > -		     fabs(mean - expected),
> > > > -		     fabs(mean - expected) / stddev,
> > > > -		     fabs(mean - expected) / line_time(kmode));
> > > > +	if (fabs(mean - expected) > max(line_time(kmode), 1.718 * stddev))
> > > > +		igt_info("vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
> > > > +			 expected, mean, stddev,
> > > > +			 fabs(mean - expected),
> > > > +			 fabs(mean - expected) / stddev,
> > > > +			 fabs(mean - expected) / line_time(kmode));
> > > >  }
> > > 
> > > 
> > > History has taught that having such messages in the logs are generally
> > > useless. Without a mechanism to tell that the output has changed, it's
> > > going to get missed and when you realize something is completely off,
> > > figuring out when it changed is a ghost hunt.
> > > 
> > > In short, the action "to allow this test to pass" is effectively
> > > deleting the test.
> > > 
> > > Having said that, deleting the test might be the right move. I said
> > > people will miss changes if it's just a message, but it has been an
> > > assertion failure for i915 CI for some time... But a consistent
> > > one. The difference on SNB looks to be 2.5 scanlines, sometimes 2.1,
> > > but most often 2.5.
> > > 
> > > Maybe Ville is able to comment? What's up with DRM_IOCTL_WAIT_VBLANK
> > > timings, is this assert obsolete with today's kernel? Or does the test
> > > need some kind of tuning?
> > 
> > It depends on how good of a match to the requested clock the PLL can give
> > us. Generally for Intel hardware modern platforms tends to have finer
> > granularity whereas older stuff can be a bit coarse. Though there are
> > ways (namely clock bending) by which we could increase the accuracy even
> > old some older platforms, but no one has bothered to implement that so far.
> > 
> > And no, I don't think deleting the test is the right answer. We
> > certainly want to catch regressions with it on the platforms where
> > we have accurate clocks. Ideally I guess we'd somehow try to catch
> > regressions on the less accurate platforms too, but not sure there's
> > any good way to do that beyond maintaining some database on what the
> > expected accuracy is for each CI machine. Relaxing the requirements
> > across the board would risk not catching smaller regressions on the
> > accurate platforms.
> 
> One idea would be to make the kernel report back the actual frequency
> somehow, and then the test could a) make sure it's reasonably close to
> what was requested, and b) the vblank timings are really accurate when
> it comes to the reported actual frequency.

Oh, and I should mention that SSC is the one thing (outside of
broken hw) that can cause variance in the timings, assuming
the driver's vblank timestamp code is otherwise robust.
I even specifically ran this on a machine with SSC enabled at
some point and the effect was certainly visible, but I don't
recall the exact numbers anymore.

Anyways if we start reporting the actual frequency to make the
test really accurate, then we may also need to report whether
SSC is used or not.
 
-- 
Ville Syrjälä
Intel

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2022-09-16 12:11 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-13 15:08 [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
2022-09-13 16:46 ` [igt-dev] ✓ Fi.CI.BAT: success for tests/kms_setmode: degrade assert to debug (rev2) Patchwork
2022-09-14  4:00 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
2022-09-15 16:06 ` [igt-dev] [PATCH i-g-t v2] tests/kms_setmode: degrade assert to debug Hamza Mahfooz
2022-09-15 18:13   ` Rodrigo Siqueira Jordao
2022-09-16 10:59 ` Petri Latvala
2022-09-16 11:47   ` Ville Syrjälä
2022-09-16 11:56     ` Ville Syrjälä
2022-09-16 12:11       ` Ville Syrjälä

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.