[igt-dev] [PATCH i-g-t 1/2] scripts/verify-blacklist: Script for checking blacklist files

[igt-dev] [PATCH i-g-t 1/2] scripts/verify-blacklist: Script for checking blacklist files

[Petri Latvala]

tests/intel-ci/blacklist*.txt files can collect bitrot unless there's
an easy way to check for lines that are no longer needed due to the
tests being renamed or removed. Therefore, a script just for that.

v2: Use long options for readability, exit with 1 if something found

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com> #v1
---
 scripts/verify-blacklist.sh | 52 +++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
 create mode 100755 scripts/verify-blacklist.sh

diff --git a/scripts/verify-blacklist.sh b/scripts/verify-blacklist.sh
new file mode 100755
index 00000000..599711ff
--- /dev/null
+++ b/scripts/verify-blacklist.sh
@@ -0,0 +1,52 @@
+#!/bin/sh
+
+# Verify that all entries in a blacklist file are still valid
+
+usage() {
+    echo "Usage: $0 <path-to-igt-runner> <test-binary-directory> <blacklist-file>"
+    echo
+    echo " path-to-igt-runner: For example build/runner/igt_runner"
+    echo " test-binary-directory: For example build/tests"
+    echo " blacklist-file: For example tests/intel-ci/blacklist.txt"
+    exit 2
+}
+
+if [ $# -ne 3 ]; then
+    usage
+fi
+
+RUNNER="$1"
+BINDIR="$2"
+BLFILE="$3"
+
+if [ ! -x "$RUNNER" ]; then
+    echo "$RUNNER not found"
+    echo
+    usage
+fi
+
+if [ ! -f "$BINDIR/test-list.txt" ]; then
+    echo "$BINDIR doesn't look like a test-binary directory"
+    echo
+    usage
+fi
+
+if [ ! -f "$BLFILE" ]; then
+    echo "$BLFILE not found"
+    echo
+    usage
+fi
+
+STATUS=0
+
+cat "$BLFILE" | while read line; do
+    test=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]')
+    if [ "$test" = "" ]; then continue; fi
+
+    if ! "$RUNNER" --list-all --include-tests "$test" "$BINDIR" >/dev/null 2>/dev/null; then
+        echo Useless blacklist entry: "$test"
+	STATUS=1
+    fi
+done
+
+exit $STATUS
-- 
2.20.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Petri Latvala]

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 .gitlab-ci.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c5a6bd9e..d7fdbfde 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -252,6 +252,12 @@ test:list-undocumented-tests:
     paths:
       - undocumented_tests.txt
 
+test:verify-blacklists:
+  dependencies:
+    - build:tests-fedora
+  stage: test
+  script: for bl in tests/intel-ci/blacklist{,-pre-merge}.txt; do scripts/verify-blacklist.sh build/runner/igt_runner build/tests "$bl" || exit 1; done
+
 ################### DEPLOY #########################
 
 pages:
-- 
2.20.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Petri Latvala]

On Fri, Jun 12, 2020 at 11:51:35AM +0300, Petri Latvala wrote:
> Signed-off-by: Petri Latvala <petri.latvala@intel.com>
> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

FYI: https://gitlab.freedesktop.org/adrinael/igt-gpu-tools/-/jobs/3073434

Verifying took 5 minutes on gitlab, but that's parallelized with the
other test-stage executions.


-- 
Petri Latvala
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/2] scripts/verify-blacklist: Script for checking blacklist files

[Patchwork]

== Series Details ==

Series: series starting with [i-g-t,1/2] scripts/verify-blacklist: Script for checking blacklist files
URL   : https://patchwork.freedesktop.org/series/78253/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_8619 -> IGTPW_4673
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html

Known issues
------------

  Here are the changes found in IGTPW_4673 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@i915_module_load@reload:
    - fi-kbl-soraka:      [PASS][1] -> [DMESG-WARN][2] ([i915#1982])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-kbl-soraka/igt@i915_module_load@reload.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-kbl-soraka/igt@i915_module_load@reload.html
    - fi-icl-guc:         [PASS][3] -> [DMESG-WARN][4] ([i915#1982])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-icl-guc/igt@i915_module_load@reload.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-icl-guc/igt@i915_module_load@reload.html

  * igt@i915_pm_rpm@basic-pci-d3-state:
    - fi-bsw-kefka:       [PASS][5] -> [DMESG-WARN][6] ([i915#1982])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-bsw-kefka/igt@i915_pm_rpm@basic-pci-d3-state.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-bsw-kefka/igt@i915_pm_rpm@basic-pci-d3-state.html

  * igt@kms_cursor_legacy@basic-flip-after-cursor-atomic:
    - fi-icl-u2:          [PASS][7] -> [DMESG-WARN][8] ([i915#1982])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-icl-u2/igt@kms_cursor_legacy@basic-flip-after-cursor-atomic.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-icl-u2/igt@kms_cursor_legacy@basic-flip-after-cursor-atomic.html

  * igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence:
    - fi-tgl-u2:          [PASS][9] -> [DMESG-WARN][10] ([i915#402])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-tgl-u2/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-tgl-u2/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html

  
#### Possible fixes ####

  * igt@i915_module_load@reload:
    - fi-tgl-u2:          [DMESG-WARN][11] ([i915#402]) -> [PASS][12]
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-tgl-u2/igt@i915_module_load@reload.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-tgl-u2/igt@i915_module_load@reload.html

  * igt@i915_pm_rpm@module-reload:
    - fi-byt-j1900:       [DMESG-WARN][13] ([i915#1982]) -> [PASS][14]
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-byt-j1900/igt@i915_pm_rpm@module-reload.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-byt-j1900/igt@i915_pm_rpm@module-reload.html
    - fi-glk-dsi:         [DMESG-WARN][15] ([i915#1982]) -> [PASS][16]
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-glk-dsi/igt@i915_pm_rpm@module-reload.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-glk-dsi/igt@i915_pm_rpm@module-reload.html

  * igt@i915_selftest@live@gt_lrc:
    - fi-tgl-u2:          [DMESG-FAIL][17] ([i915#1233]) -> [PASS][18]
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-tgl-u2/igt@i915_selftest@live@gt_lrc.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-tgl-u2/igt@i915_selftest@live@gt_lrc.html

  * igt@kms_flip@basic-flip-vs-wf_vblank@c-edp1:
    - fi-icl-u2:          [DMESG-WARN][19] ([i915#1982]) -> [PASS][20]
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-icl-u2/igt@kms_flip@basic-flip-vs-wf_vblank@c-edp1.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-icl-u2/igt@kms_flip@basic-flip-vs-wf_vblank@c-edp1.html

  
#### Warnings ####

  * igt@i915_pm_rpm@module-reload:
    - fi-kbl-x1275:       [DMESG-FAIL][21] ([i915#62]) -> [SKIP][22] ([fdo#109271])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-kbl-x1275/igt@i915_pm_rpm@module-reload.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-kbl-x1275/igt@i915_pm_rpm@module-reload.html

  * igt@kms_force_connector_basic@force-edid:
    - fi-kbl-x1275:       [DMESG-WARN][23] ([i915#62] / [i915#92]) -> [DMESG-WARN][24] ([i915#62] / [i915#92] / [i915#95]) +5 similar issues
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-kbl-x1275/igt@kms_force_connector_basic@force-edid.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-kbl-x1275/igt@kms_force_connector_basic@force-edid.html

  * igt@prime_vgem@basic-fence-flip:
    - fi-kbl-x1275:       [DMESG-WARN][25] ([i915#62] / [i915#92] / [i915#95]) -> [DMESG-WARN][26] ([i915#62] / [i915#92]) +5 similar issues
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/fi-kbl-x1275/igt@prime_vgem@basic-fence-flip.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/fi-kbl-x1275/igt@prime_vgem@basic-fence-flip.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [i915#1233]: https://gitlab.freedesktop.org/drm/intel/issues/1233
  [i915#1569]: https://gitlab.freedesktop.org/drm/intel/issues/1569
  [i915#192]: https://gitlab.freedesktop.org/drm/intel/issues/192
  [i915#193]: https://gitlab.freedesktop.org/drm/intel/issues/193
  [i915#194]: https://gitlab.freedesktop.org/drm/intel/issues/194
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402
  [i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
  [i915#92]: https://gitlab.freedesktop.org/drm/intel/issues/92
  [i915#95]: https://gitlab.freedesktop.org/drm/intel/issues/95


Participating hosts (50 -> 43)
------------------------------

  Missing    (7): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-byt-clapper fi-bdw-samus 


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_5706 -> IGTPW_4673

  CI-20190529: 20190529
  CI_DRM_8619: 0f425ca456b45b2bbe1534ecc25c74b657d0645f @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_4673: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html
  IGT_5706: 08fdc87d48dba48fb82f9978e142d57b013ff29f @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✗ Fi.CI.IGT: failure for series starting with [i-g-t,1/2] scripts/verify-blacklist: Script for checking blacklist files

[Patchwork]

== Series Details ==

Series: series starting with [i-g-t,1/2] scripts/verify-blacklist: Script for checking blacklist files
URL   : https://patchwork.freedesktop.org/series/78253/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_8619_full -> IGTPW_4673_full
====================================================

Summary
-------

  **FAILURE**

  Serious unknown changes coming with IGTPW_4673_full absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_4673_full, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html

Possible new issues
-------------------

  Here are the unknown changes that may have been introduced in IGTPW_4673_full:

### IGT changes ###

#### Possible regressions ####

  * igt@gem_exec_endless@dispatch@vecs0:
    - shard-iclb:         [PASS][1] -> [INCOMPLETE][2]
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb7/igt@gem_exec_endless@dispatch@vecs0.html
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb7/igt@gem_exec_endless@dispatch@vecs0.html

  
Known issues
------------

  Here are the changes found in IGTPW_4673_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_isolation@preservation-s3@vcs0:
    - shard-kbl:          [PASS][3] -> [DMESG-WARN][4] ([i915#180]) +2 similar issues
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl3/igt@gem_ctx_isolation@preservation-s3@vcs0.html
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@gem_ctx_isolation@preservation-s3@vcs0.html

  * igt@gem_ctx_shared@disjoint-timelines:
    - shard-kbl:          [PASS][5] -> [DMESG-WARN][6] ([i915#93] / [i915#95]) +48 similar issues
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl1/igt@gem_ctx_shared@disjoint-timelines.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@gem_ctx_shared@disjoint-timelines.html

  * igt@gem_exec_reloc@basic-concurrent0:
    - shard-glk:          [PASS][7] -> [FAIL][8] ([i915#1930])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk9/igt@gem_exec_reloc@basic-concurrent0.html
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk9/igt@gem_exec_reloc@basic-concurrent0.html

  * igt@gem_exec_schedule@implicit-write-read@rcs0:
    - shard-snb:          [PASS][9] -> [INCOMPLETE][10] ([i915#82])
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb4/igt@gem_exec_schedule@implicit-write-read@rcs0.html
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb2/igt@gem_exec_schedule@implicit-write-read@rcs0.html

  * igt@gem_exec_whisper@basic-fds-forked-all:
    - shard-glk:          [PASS][11] -> [DMESG-WARN][12] ([i915#118] / [i915#95])
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk9/igt@gem_exec_whisper@basic-fds-forked-all.html
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk9/igt@gem_exec_whisper@basic-fds-forked-all.html

  * igt@i915_module_load@reload:
    - shard-tglb:         [PASS][13] -> [DMESG-WARN][14] ([i915#402]) +3 similar issues
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb2/igt@i915_module_load@reload.html
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb5/igt@i915_module_load@reload.html

  * igt@i915_pm_dc@dc6-psr:
    - shard-iclb:         [PASS][15] -> [FAIL][16] ([i915#454])
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb4/igt@i915_pm_dc@dc6-psr.html
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb6/igt@i915_pm_dc@dc6-psr.html

  * igt@kms_addfb_basic@invalid-get-prop:
    - shard-snb:          [PASS][17] -> [TIMEOUT][18] ([i915#1958])
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb5/igt@kms_addfb_basic@invalid-get-prop.html
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb6/igt@kms_addfb_basic@invalid-get-prop.html

  * igt@kms_big_fb@linear-32bpp-rotate-180:
    - shard-apl:          [PASS][19] -> [DMESG-WARN][20] ([i915#1982]) +3 similar issues
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl3/igt@kms_big_fb@linear-32bpp-rotate-180.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl4/igt@kms_big_fb@linear-32bpp-rotate-180.html

  * igt@kms_big_fb@x-tiled-64bpp-rotate-0:
    - shard-glk:          [PASS][21] -> [DMESG-FAIL][22] ([i915#118] / [i915#95])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk7/igt@kms_big_fb@x-tiled-64bpp-rotate-0.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk1/igt@kms_big_fb@x-tiled-64bpp-rotate-0.html

  * igt@kms_big_fb@x-tiled-64bpp-rotate-180:
    - shard-glk:          [PASS][23] -> [DMESG-WARN][24] ([i915#1982])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk7/igt@kms_big_fb@x-tiled-64bpp-rotate-180.html
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk2/igt@kms_big_fb@x-tiled-64bpp-rotate-180.html

  * igt@kms_cursor_crc@pipe-a-cursor-256x256-random:
    - shard-kbl:          [PASS][25] -> [DMESG-FAIL][26] ([i915#54] / [i915#95]) +2 similar issues
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl1/igt@kms_cursor_crc@pipe-a-cursor-256x256-random.html
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl2/igt@kms_cursor_crc@pipe-a-cursor-256x256-random.html

  * igt@kms_cursor_crc@pipe-b-cursor-suspend:
    - shard-apl:          [PASS][27] -> [FAIL][28] ([i915#54]) +1 similar issue
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl2/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl8/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
    - shard-kbl:          [PASS][29] -> [INCOMPLETE][30] ([i915#155])
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl2/igt@kms_cursor_crc@pipe-b-cursor-suspend.html
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl1/igt@kms_cursor_crc@pipe-b-cursor-suspend.html

  * igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding:
    - shard-kbl:          [PASS][31] -> [FAIL][32] ([i915#54])
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl4/igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding.html
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl3/igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding.html
    - shard-glk:          [PASS][33] -> [FAIL][34] ([i915#54])
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk9/igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding.html
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk6/igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding.html

  * igt@kms_cursor_crc@pipe-c-cursor-64x21-offscreen:
    - shard-hsw:          [PASS][35] -> [TIMEOUT][36] ([i915#1958]) +2 similar issues
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw1/igt@kms_cursor_crc@pipe-c-cursor-64x21-offscreen.html
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw4/igt@kms_cursor_crc@pipe-c-cursor-64x21-offscreen.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-plflip-blt:
    - shard-iclb:         [PASS][37] -> [DMESG-WARN][38] ([i915#1982])
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb5/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-plflip-blt.html
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb2/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-shrfb-plflip-blt.html

  * igt@kms_lease@lease-uevent:
    - shard-apl:          [PASS][39] -> [DMESG-WARN][40] ([i915#95]) +31 similar issues
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl7/igt@kms_lease@lease-uevent.html
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl4/igt@kms_lease@lease-uevent.html

  * igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence:
    - shard-apl:          [PASS][41] -> [DMESG-FAIL][42] ([i915#95])
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl8/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl3/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html
    - shard-kbl:          [PASS][43] -> [DMESG-FAIL][44] ([i915#95])
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl2/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence.html

  * igt@kms_plane_alpha_blend@pipe-a-coverage-7efc:
    - shard-apl:          [PASS][45] -> [DMESG-FAIL][46] ([fdo#108145] / [i915#95])
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl1/igt@kms_plane_alpha_blend@pipe-a-coverage-7efc.html
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl1/igt@kms_plane_alpha_blend@pipe-a-coverage-7efc.html

  * igt@kms_psr@psr2_cursor_mmap_cpu:
    - shard-iclb:         [PASS][47] -> [SKIP][48] ([fdo#109441]) +2 similar issues
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb2/igt@kms_psr@psr2_cursor_mmap_cpu.html
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb6/igt@kms_psr@psr2_cursor_mmap_cpu.html

  * igt@perf@polling-parameterized:
    - shard-hsw:          [PASS][49] -> [FAIL][50] ([i915#1542])
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw4/igt@perf@polling-parameterized.html
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw8/igt@perf@polling-parameterized.html

  
#### Possible fixes ####

  * igt@gem_exec_schedule@implicit-read-write@rcs0:
    - shard-snb:          [INCOMPLETE][51] ([i915#82]) -> [PASS][52] +1 similar issue
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb6/igt@gem_exec_schedule@implicit-read-write@rcs0.html
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb2/igt@gem_exec_schedule@implicit-read-write@rcs0.html

  * igt@gem_exec_schedule@semaphore-codependency:
    - shard-kbl:          [DMESG-WARN][53] ([i915#93] / [i915#95]) -> [PASS][54] +50 similar issues
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl2/igt@gem_exec_schedule@semaphore-codependency.html
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@gem_exec_schedule@semaphore-codependency.html

  * igt@gem_exec_schedule@smoketest-all:
    - shard-tglb:         [INCOMPLETE][55] ([i915#1318]) -> [PASS][56]
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb8/igt@gem_exec_schedule@smoketest-all.html
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb6/igt@gem_exec_schedule@smoketest-all.html

  * igt@gem_exec_whisper@basic-fds-priority-all:
    - shard-glk:          [DMESG-WARN][57] ([i915#118] / [i915#95]) -> [PASS][58] +1 similar issue
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk1/igt@gem_exec_whisper@basic-fds-priority-all.html
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk8/igt@gem_exec_whisper@basic-fds-priority-all.html

  * igt@i915_suspend@fence-restore-tiled2untiled:
    - shard-apl:          [DMESG-WARN][59] ([i915#180]) -> [PASS][60] +2 similar issues
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl6/igt@i915_suspend@fence-restore-tiled2untiled.html
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl8/igt@i915_suspend@fence-restore-tiled2untiled.html

  * igt@kms_big_fb@x-tiled-64bpp-rotate-0:
    - shard-apl:          [DMESG-WARN][61] ([i915#1982]) -> [PASS][62]
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl2/igt@kms_big_fb@x-tiled-64bpp-rotate-0.html
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl8/igt@kms_big_fb@x-tiled-64bpp-rotate-0.html

  * igt@kms_cursor_crc@pipe-a-cursor-128x128-random:
    - shard-kbl:          [DMESG-FAIL][63] ([i915#54] / [i915#95]) -> [PASS][64] +1 similar issue
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl3/igt@kms_cursor_crc@pipe-a-cursor-128x128-random.html
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl3/igt@kms_cursor_crc@pipe-a-cursor-128x128-random.html

  * igt@kms_cursor_crc@pipe-b-cursor-64x64-random:
    - shard-hsw:          [INCOMPLETE][65] ([i915#61]) -> [PASS][66]
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw2/igt@kms_cursor_crc@pipe-b-cursor-64x64-random.html
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw2/igt@kms_cursor_crc@pipe-b-cursor-64x64-random.html

  * igt@kms_cursor_edge_walk@pipe-b-64x64-bottom-edge:
    - shard-snb:          [TIMEOUT][67] ([i915#1958]) -> [PASS][68]
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb6/igt@kms_cursor_edge_walk@pipe-b-64x64-bottom-edge.html
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb2/igt@kms_cursor_edge_walk@pipe-b-64x64-bottom-edge.html
    - shard-hsw:          [TIMEOUT][69] ([i915#1958]) -> [PASS][70]
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw2/igt@kms_cursor_edge_walk@pipe-b-64x64-bottom-edge.html
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw8/igt@kms_cursor_edge_walk@pipe-b-64x64-bottom-edge.html

  * igt@kms_cursor_edge_walk@pipe-c-128x128-left-edge:
    - shard-glk:          [DMESG-WARN][71] ([i915#1982]) -> [PASS][72]
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-glk5/igt@kms_cursor_edge_walk@pipe-c-128x128-left-edge.html
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-glk1/igt@kms_cursor_edge_walk@pipe-c-128x128-left-edge.html

  * igt@kms_draw_crc@draw-method-xrgb2101010-mmap-wc-untiled:
    - shard-kbl:          [DMESG-WARN][73] ([i915#1982]) -> [PASS][74] +1 similar issue
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl4/igt@kms_draw_crc@draw-method-xrgb2101010-mmap-wc-untiled.html
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl3/igt@kms_draw_crc@draw-method-xrgb2101010-mmap-wc-untiled.html

  * igt@kms_draw_crc@draw-method-xrgb8888-mmap-cpu-untiled:
    - shard-apl:          [DMESG-FAIL][75] ([i915#54] / [i915#95]) -> [PASS][76]
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl1/igt@kms_draw_crc@draw-method-xrgb8888-mmap-cpu-untiled.html
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl6/igt@kms_draw_crc@draw-method-xrgb8888-mmap-cpu-untiled.html

  * igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-untiled:
    - shard-kbl:          [DMESG-FAIL][77] ([fdo#108145] / [i915#54] / [i915#95]) -> [PASS][78]
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl7/igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-untiled.html
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-untiled.html
    - shard-apl:          [DMESG-FAIL][79] ([fdo#108145] / [i915#54] / [i915#95]) -> [PASS][80]
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl8/igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-untiled.html
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl2/igt@kms_draw_crc@draw-method-xrgb8888-mmap-wc-untiled.html

  * igt@kms_draw_crc@fill-fb:
    - shard-apl:          [DMESG-FAIL][81] ([i915#95]) -> [PASS][82]
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl3/igt@kms_draw_crc@fill-fb.html
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl3/igt@kms_draw_crc@fill-fb.html

  * igt@kms_flip@2x-flip-vs-blocking-wf-vblank@ab-vga1-hdmi-a1:
    - shard-hsw:          [DMESG-WARN][83] ([i915#1982]) -> [PASS][84]
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw6/igt@kms_flip@2x-flip-vs-blocking-wf-vblank@ab-vga1-hdmi-a1.html
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw2/igt@kms_flip@2x-flip-vs-blocking-wf-vblank@ab-vga1-hdmi-a1.html

  * igt@kms_flip@flip-vs-suspend@c-dp1:
    - shard-kbl:          [DMESG-WARN][85] ([i915#180]) -> [PASS][86] +3 similar issues
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl7/igt@kms_flip@flip-vs-suspend@c-dp1.html
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@kms_flip@flip-vs-suspend@c-dp1.html

  * igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-wc:
    - shard-iclb:         [DMESG-WARN][87] ([i915#1982]) -> [PASS][88]
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb3/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-wc.html
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb1/igt@kms_frontbuffer_tracking@fbc-1p-offscren-pri-indfb-draw-mmap-wc.html

  * igt@kms_frontbuffer_tracking@fbcpsr-1p-shrfb-fliptrack:
    - shard-tglb:         [DMESG-WARN][89] ([i915#1982]) -> [PASS][90]
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb7/igt@kms_frontbuffer_tracking@fbcpsr-1p-shrfb-fliptrack.html
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb1/igt@kms_frontbuffer_tracking@fbcpsr-1p-shrfb-fliptrack.html

  * igt@kms_hdmi_inject@inject-audio:
    - shard-tglb:         [SKIP][91] ([i915#433]) -> [PASS][92]
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb7/igt@kms_hdmi_inject@inject-audio.html
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb1/igt@kms_hdmi_inject@inject-audio.html

  * igt@kms_psr2_su@frontbuffer:
    - shard-iclb:         [SKIP][93] ([fdo#109642] / [fdo#111068]) -> [PASS][94]
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb3/igt@kms_psr2_su@frontbuffer.html
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb2/igt@kms_psr2_su@frontbuffer.html

  * igt@kms_psr@psr2_cursor_render:
    - shard-iclb:         [SKIP][95] ([fdo#109441]) -> [PASS][96] +2 similar issues
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb4/igt@kms_psr@psr2_cursor_render.html
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb2/igt@kms_psr@psr2_cursor_render.html

  * igt@kms_setmode@basic:
    - shard-kbl:          [FAIL][97] ([i915#31]) -> [PASS][98]
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl7/igt@kms_setmode@basic.html
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl1/igt@kms_setmode@basic.html

  * igt@perf@blocking-parameterized:
    - shard-iclb:         [FAIL][99] ([i915#1542]) -> [PASS][100]
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-iclb3/igt@perf@blocking-parameterized.html
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-iclb6/igt@perf@blocking-parameterized.html

  * igt@perf_pmu@module-unload:
    - shard-tglb:         [DMESG-WARN][101] ([i915#402]) -> [PASS][102] +2 similar issues
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb5/igt@perf_pmu@module-unload.html
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb6/igt@perf_pmu@module-unload.html

  * igt@prime_vgem@basic-fence-flip:
    - shard-apl:          [DMESG-WARN][103] ([i915#95]) -> [PASS][104] +48 similar issues
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl7/igt@prime_vgem@basic-fence-flip.html
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl1/igt@prime_vgem@basic-fence-flip.html

  
#### Warnings ####

  * igt@gem_ppgtt@flink-and-close-vma-leak:
    - shard-hsw:          [TIMEOUT][105] ([i915#1958]) -> [SKIP][106] ([fdo#109271]) +2 similar issues
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw2/igt@gem_ppgtt@flink-and-close-vma-leak.html
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw1/igt@gem_ppgtt@flink-and-close-vma-leak.html

  * igt@i915_pm_dc@dc6-psr:
    - shard-tglb:         [SKIP][107] ([i915#468]) -> [FAIL][108] ([i915#454])
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb2/igt@i915_pm_dc@dc6-psr.html
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb7/igt@i915_pm_dc@dc6-psr.html

  * igt@i915_selftest@live@gt_pm:
    - shard-tglb:         [DMESG-FAIL][109] ([i915#1021]) -> [DMESG-FAIL][110] ([i915#1759])
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-tglb1/igt@i915_selftest@live@gt_pm.html
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-tglb3/igt@i915_selftest@live@gt_pm.html

  * igt@kms_ccs@pipe-d-bad-rotation-90:
    - shard-snb:          [TIMEOUT][111] ([i915#1958]) -> [SKIP][112] ([fdo#109271]) +3 similar issues
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb6/igt@kms_ccs@pipe-d-bad-rotation-90.html
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb2/igt@kms_ccs@pipe-d-bad-rotation-90.html

  * igt@kms_content_protection@atomic:
    - shard-apl:          [TIMEOUT][113] ([i915#1319] / [i915#1635]) -> [TIMEOUT][114] ([i915#1319])
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl3/igt@kms_content_protection@atomic.html
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl3/igt@kms_content_protection@atomic.html

  * igt@kms_content_protection@atomic-dpms:
    - shard-apl:          [DMESG-FAIL][115] ([fdo#110321] / [i915#95]) -> [TIMEOUT][116] ([i915#1319] / [i915#1635])
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl7/igt@kms_content_protection@atomic-dpms.html
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl8/igt@kms_content_protection@atomic-dpms.html

  * igt@kms_content_protection@lic:
    - shard-kbl:          [TIMEOUT][117] ([i915#1319]) -> [TIMEOUT][118] ([i915#1319] / [i915#1958])
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl7/igt@kms_content_protection@lic.html
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl4/igt@kms_content_protection@lic.html

  * igt@kms_content_protection@srm:
    - shard-kbl:          [TIMEOUT][119] ([i915#1319] / [i915#1958]) -> [TIMEOUT][120] ([i915#1319])
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl4/igt@kms_content_protection@srm.html
   [120]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl7/igt@kms_content_protection@srm.html

  * igt@kms_content_protection@uevent:
    - shard-kbl:          [FAIL][121] ([i915#357]) -> [DMESG-FAIL][122] ([i915#357] / [i915#95])
   [121]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl4/igt@kms_content_protection@uevent.html
   [122]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl3/igt@kms_content_protection@uevent.html
    - shard-apl:          [FAIL][123] ([i915#357]) -> [DMESG-FAIL][124] ([i915#357] / [i915#95])
   [123]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl2/igt@kms_content_protection@uevent.html
   [124]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl2/igt@kms_content_protection@uevent.html

  * igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-cpu:
    - shard-snb:          [SKIP][125] ([fdo#109271]) -> [TIMEOUT][126] ([i915#1958]) +4 similar issues
   [125]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-snb4/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-cpu.html
   [126]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-snb6/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-cpu.html

  * igt@kms_frontbuffer_tracking@fbc-suspend:
    - shard-kbl:          [DMESG-WARN][127] ([i915#1982] / [i915#93] / [i915#95]) -> [DMESG-WARN][128] ([i915#93] / [i915#95])
   [127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl7/igt@kms_frontbuffer_tracking@fbc-suspend.html
   [128]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl3/igt@kms_frontbuffer_tracking@fbc-suspend.html

  * igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb:
    - shard-apl:          [FAIL][129] ([fdo#108145] / [i915#265]) -> [DMESG-FAIL][130] ([fdo#108145] / [i915#95])
   [129]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl6/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
   [130]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl3/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
    - shard-kbl:          [FAIL][131] ([fdo#108145] / [i915#265]) -> [DMESG-FAIL][132] ([fdo#108145] / [i915#95])
   [131]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl2/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html
   [132]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl6/igt@kms_plane_alpha_blend@pipe-a-alpha-opaque-fb.html

  * igt@kms_plane_alpha_blend@pipe-a-coverage-vs-premult-vs-constant:
    - shard-hsw:          [SKIP][133] ([fdo#109271]) -> [TIMEOUT][134] ([i915#1958]) +2 similar issues
   [133]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-hsw1/igt@kms_plane_alpha_blend@pipe-a-coverage-vs-premult-vs-constant.html
   [134]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-hsw4/igt@kms_plane_alpha_blend@pipe-a-coverage-vs-premult-vs-constant.html

  * igt@kms_plane_alpha_blend@pipe-b-alpha-7efc:
    - shard-apl:          [DMESG-FAIL][135] ([fdo#108145] / [i915#95]) -> [FAIL][136] ([fdo#108145] / [i915#265]) +1 similar issue
   [135]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-apl4/igt@kms_plane_alpha_blend@pipe-b-alpha-7efc.html
   [136]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-apl8/igt@kms_plane_alpha_blend@pipe-b-alpha-7efc.html
    - shard-kbl:          [DMESG-FAIL][137] ([fdo#108145] / [i915#95]) -> [FAIL][138] ([fdo#108145] / [i915#265]) +1 similar issue
   [137]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8619/shard-kbl1/igt@kms_plane_alpha_blend@pipe-b-alpha-7efc.html
   [138]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/shard-kbl1/igt@kms_plane_alpha_blend@pipe-b-alpha-7efc.html

  
  [fdo#108145]: https://bugs.freedesktop.org/show_bug.cgi?id=108145
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
  [fdo#110321]: https://bugs.freedesktop.org/show_bug.cgi?id=110321
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [i915#1021]: https://gitlab.freedesktop.org/drm/intel/issues/1021
  [i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
  [i915#1318]: https://gitlab.freedesktop.org/drm/intel/issues/1318
  [i915#1319]: https://gitlab.freedesktop.org/drm/intel/issues/1319
  [i915#1542]: https://gitlab.freedesktop.org/drm/intel/issues/1542
  [i915#155]: https://gitlab.freedesktop.org/drm/intel/issues/155
  [i915#1635]: https://gitlab.freedesktop.org/drm/intel/issues/1635
  [i915#1759]: https://gitlab.freedesktop.org/drm/intel/issues/1759
  [i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
  [i915#1930]: https://gitlab.freedesktop.org/drm/intel/issues/1930
  [i915#1958]: https://gitlab.freedesktop.org/drm/intel/issues/1958
  [i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
  [i915#265]: https://gitlab.freedesktop.org/drm/intel/issues/265
  [i915#31]: https://gitlab.freedesktop.org/drm/intel/issues/31
  [i915#357]: https://gitlab.freedesktop.org/drm/intel/issues/357
  [i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402
  [i915#433]: https://gitlab.freedesktop.org/drm/intel/issues/433
  [i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
  [i915#468]: https://gitlab.freedesktop.org/drm/intel/issues/468
  [i915#54]: https://gitlab.freedesktop.org/drm/intel/issues/54
  [i915#61]: https://gitlab.freedesktop.org/drm/intel/issues/61
  [i915#82]: https://gitlab.freedesktop.org/drm/intel/issues/82
  [i915#93]: https://gitlab.freedesktop.org/drm/intel/issues/93
  [i915#95]: https://gitlab.freedesktop.org/drm/intel/issues/95


Participating hosts (11 -> 8)
------------------------------

  Missing    (3): pig-skl-6260u pig-glk-j5005 pig-icl-1065g7 


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_5706 -> IGTPW_4673
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_8619: 0f425ca456b45b2bbe1534ecc25c74b657d0645f @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_4673: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html
  IGT_5706: 08fdc87d48dba48fb82f9978e142d57b013ff29f @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4673/index.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Arkadiusz Hiler]

On Fri, Jun 12, 2020 at 11:57:14AM +0300, Petri Latvala wrote:
> On Fri, Jun 12, 2020 at 11:51:35AM +0300, Petri Latvala wrote:
> > Signed-off-by: Petri Latvala <petri.latvala@intel.com>
> > Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> 
> FYI: https://gitlab.freedesktop.org/adrinael/igt-gpu-tools/-/jobs/3073434
> 
> Verifying took 5 minutes on gitlab, but that's parallelized with the
> other test-stage executions.

But it's also the slowest step of that stage by far. Even the qemu
cross-testing is ~5x faster.

Doing:
> if ! "$RUNNER" --list-all --include-tests "$test" "$BINDIR" >/dev/null 2>/dev/null; then
for each blacklist entry is a bit expensive. It execs all the test
binaries each time.

Something like this should be faster:

TESTLIST="$("$RUNNER" --list-all "$BINDIR")"

cat "$BLFILE" | while read line; do
    blentry=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]')
    if [ "$blentry" = "" ]; then continue; fi

    if ! (echo "$TESTLIST" | grep -q "$blentry") >/dev/null 2>/dev/null; then
        echo Useless blacklist entry: "$blentry"
       STATUS=1
    fi
done

This may be not exactly the same as runner is using glib's regex, but if
you switch it to PCRE then 'grep -P' should be close enough.



Another observation - we don't assert beginning/end off line with ^$,
but yet we have patterns like:
> igt@i915_pm_rpm@universal-planes(-dpms)?

Which will kill any universal-planes-definitely-not-dpms subtest too.

-- 
Cheers,
Arek
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Petri Latvala]

On Fri, Jun 12, 2020 at 03:52:47PM +0300, Arkadiusz Hiler wrote:
> On Fri, Jun 12, 2020 at 11:57:14AM +0300, Petri Latvala wrote:
> > On Fri, Jun 12, 2020 at 11:51:35AM +0300, Petri Latvala wrote:
> > > Signed-off-by: Petri Latvala <petri.latvala@intel.com>
> > > Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> > 
> > FYI: https://gitlab.freedesktop.org/adrinael/igt-gpu-tools/-/jobs/3073434
> > 
> > Verifying took 5 minutes on gitlab, but that's parallelized with the
> > other test-stage executions.
> 
> But it's also the slowest step of that stage by far. Even the qemu
> cross-testing is ~5x faster.
> 
> Doing:
> > if ! "$RUNNER" --list-all --include-tests "$test" "$BINDIR" >/dev/null 2>/dev/null; then
> for each blacklist entry is a bit expensive. It execs all the test
> binaries each time.
> 
> Something like this should be faster:
> 
> TESTLIST="$("$RUNNER" --list-all "$BINDIR")"
> 
> cat "$BLFILE" | while read line; do
>     blentry=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]')
>     if [ "$blentry" = "" ]; then continue; fi
> 
>     if ! (echo "$TESTLIST" | grep -q "$blentry") >/dev/null 2>/dev/null; then
>         echo Useless blacklist entry: "$blentry"
>        STATUS=1
>     fi
> done
> 
> This may be not exactly the same as runner is using glib's regex, but if
> you switch it to PCRE then 'grep -P' should be close enough.


Meh, fair enough. I was doing it this way to make sure we process the
blacklist exactly the same way in the check as in "production", and I
can't remember now what my real fear was. Extending the blacklist
syntax? We're going to catch that on first use on some level of
confidence and even if we don't it's not a big deal.

New revision with new numbers on its way!


-- 
Petri Latvala
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Arkadiusz Hiler]

On Mon, Jun 15, 2020 at 01:25:16PM +0300, Petri Latvala wrote:
> Signed-off-by: Petri Latvala <petri.latvala@intel.com>
> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
> ---
> 
> Now finished in 47 seconds in gitlab.
> 
> 
>  .gitlab-ci.yml | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index c5a6bd9e..d7fdbfde 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -252,6 +252,12 @@ test:list-undocumented-tests:
>      paths:
>        - undocumented_tests.txt
>  
> +test:verify-blacklists:
> +  dependencies:
> +    - build:tests-fedora
> +  stage: test
> +  script: for bl in tests/intel-ci/blacklist{,-pre-merge}.txt; do scripts/verify-blacklist.sh build/runner/igt_runner build/tests "$bl" || exit 1; done
> +
>  ################### DEPLOY #########################
>  
>  pages:
> -- 
> 2.20.1

For both patches,

Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

We still should do something about the assume implicit ^$ that are not
there.
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh

[Petri Latvala]

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---

Now finished in 47 seconds in gitlab.


 .gitlab-ci.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c5a6bd9e..d7fdbfde 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -252,6 +252,12 @@ test:list-undocumented-tests:
     paths:
       - undocumented_tests.txt
 
+test:verify-blacklists:
+  dependencies:
+    - build:tests-fedora
+  stage: test
+  script: for bl in tests/intel-ci/blacklist{,-pre-merge}.txt; do scripts/verify-blacklist.sh build/runner/igt_runner build/tests "$bl" || exit 1; done
+
 ################### DEPLOY #########################
 
 pages:
-- 
2.20.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev