All of lore.kernel.org
 help / color / mirror / Atom feed
* [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed
@ 2018-11-13 13:04 Petri Latvala
  2018-11-13 13:24 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/1] " Patchwork
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: Petri Latvala @ 2018-11-13 13:04 UTC (permalink / raw)
  To: igt-dev; +Cc: Petri Latvala, Martin Peres

When possible, all tests we know we were going to attempt to execute
now appear in the results as "notrun". The only known case where it's
not possible to add an explicit "notrun" is when running in
multiple-mode, because "no subtests" and "run all subtests, we didn't
list them beforehand" are represented the same.

Note: A function call is commented out in resultgen.c because it needs
the function to exist, and that's in a patch that is still in flight.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 .../joblist.txt                               |   4 -
 .../notrun-results-multiple-mode/0/dmesg.txt  |   5 +
 .../notrun-results-multiple-mode/0/err.txt    |   2 +
 .../0/journal.txt                             |   2 +
 .../notrun-results-multiple-mode/0/out.txt    |   3 +
 .../notrun-results-multiple-mode/README.txt   |   4 +
 .../notrun-results-multiple-mode/endtime.txt  |   1 +
 .../notrun-results-multiple-mode/joblist.txt  |   5 +
 .../notrun-results-multiple-mode/metadata.txt |  12 ++
 .../reference.json                            | 102 +++++++++++++++
 .../starttime.txt                             |   1 +
 .../notrun-results-multiple-mode/uname.txt    |   1 +
 .../notrun-results/0/dmesg.txt                |   5 +
 .../json_tests_data/notrun-results/0/err.txt  |   2 +
 .../notrun-results/0/journal.txt              |   2 +
 .../json_tests_data/notrun-results/0/out.txt  |   3 +
 .../json_tests_data/notrun-results/README.txt |   4 +
 .../notrun-results/endtime.txt                |   1 +
 .../notrun-results/joblist.txt                |   5 +
 .../notrun-results/metadata.txt               |  12 ++
 .../notrun-results/reference.json             | 120 ++++++++++++++++++
 .../notrun-results/starttime.txt              |   1 +
 .../json_tests_data/notrun-results/uname.txt  |   1 +
 runner/resultgen.c                            |  43 ++++++-
 runner/runner_json_tests.c                    |   2 +
 25 files changed, 337 insertions(+), 6 deletions(-)
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/0/dmesg.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/0/err.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/0/journal.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/0/out.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/README.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/endtime.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/joblist.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/metadata.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/reference.json
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/starttime.txt
 create mode 100644 runner/json_tests_data/notrun-results-multiple-mode/uname.txt
 create mode 100644 runner/json_tests_data/notrun-results/0/dmesg.txt
 create mode 100644 runner/json_tests_data/notrun-results/0/err.txt
 create mode 100644 runner/json_tests_data/notrun-results/0/journal.txt
 create mode 100644 runner/json_tests_data/notrun-results/0/out.txt
 create mode 100644 runner/json_tests_data/notrun-results/README.txt
 create mode 100644 runner/json_tests_data/notrun-results/endtime.txt
 create mode 100644 runner/json_tests_data/notrun-results/joblist.txt
 create mode 100644 runner/json_tests_data/notrun-results/metadata.txt
 create mode 100644 runner/json_tests_data/notrun-results/reference.json
 create mode 100644 runner/json_tests_data/notrun-results/starttime.txt
 create mode 100644 runner/json_tests_data/notrun-results/uname.txt

diff --git a/runner/json_tests_data/incomplete-before-any-subtests/joblist.txt b/runner/json_tests_data/incomplete-before-any-subtests/joblist.txt
index 31ef8413..81f914a7 100644
--- a/runner/json_tests_data/incomplete-before-any-subtests/joblist.txt
+++ b/runner/json_tests_data/incomplete-before-any-subtests/joblist.txt
@@ -1,5 +1 @@
 successtest first-subtest
-successtest second-subtest
-no-subtests
-skippers skip-one
-skippers skip-two
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/0/dmesg.txt b/runner/json_tests_data/notrun-results-multiple-mode/0/dmesg.txt
new file mode 100644
index 00000000..a189e704
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/0/dmesg.txt
@@ -0,0 +1,5 @@
+6,951,3216186095083,-;Console: switching to colour dummy device 80x25
+14,952,3216186095097,-;[IGT] successtest: executing
+14,953,3216186101115,-;[IGT] successtest: starting subtest first-subtest
+14,954,3216186101160,-;[IGT] successtest: exiting, ret=0
+6,955,3216186101299,-;Console: switching to colour frame buffer device 240x75
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/0/err.txt b/runner/json_tests_data/notrun-results-multiple-mode/0/err.txt
new file mode 100644
index 00000000..5dc78057
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/0/err.txt
@@ -0,0 +1,2 @@
+Starting subtest: first-subtest
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/0/journal.txt b/runner/json_tests_data/notrun-results-multiple-mode/0/journal.txt
new file mode 100644
index 00000000..86a30e07
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/0/journal.txt
@@ -0,0 +1,2 @@
+first-subtest
+exit:0 (0.014s)
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/0/out.txt b/runner/json_tests_data/notrun-results-multiple-mode/0/out.txt
new file mode 100644
index 00000000..5946bf31
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/0/out.txt
@@ -0,0 +1,3 @@
+IGT-Version: 1.23-g0c763bfd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: first-subtest
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/README.txt b/runner/json_tests_data/notrun-results-multiple-mode/README.txt
new file mode 100644
index 00000000..fedb6cf5
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/README.txt
@@ -0,0 +1,4 @@
+Tests that are not executed are expected to be present in results.json
+when possible. If run with --multiple-mode, we can't distinguish
+tests without subtests from tests where we attempt to execute all
+subtests.
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/endtime.txt b/runner/json_tests_data/notrun-results-multiple-mode/endtime.txt
new file mode 100644
index 00000000..635f6ae9
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/endtime.txt
@@ -0,0 +1 @@
+1539953735.172373
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/joblist.txt b/runner/json_tests_data/notrun-results-multiple-mode/joblist.txt
new file mode 100644
index 00000000..31ef8413
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/joblist.txt
@@ -0,0 +1,5 @@
+successtest first-subtest
+successtest second-subtest
+no-subtests
+skippers skip-one
+skippers skip-two
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/metadata.txt b/runner/json_tests_data/notrun-results-multiple-mode/metadata.txt
new file mode 100644
index 00000000..4f027f72
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/metadata.txt
@@ -0,0 +1,12 @@
+abort_on_error : 0
+name : normal-run
+dry_run : 0
+sync : 0
+log_level : 0
+overwrite : 0
+multiple_mode : 1
+inactivity_timeout : 0
+use_watchdog : 0
+piglit_style_dmesg : 0
+test_root : /path/does/not/exist
+results_path : /path/does/not/exist
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/reference.json b/runner/json_tests_data/notrun-results-multiple-mode/reference.json
new file mode 100644
index 00000000..b953b4a2
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/reference.json
@@ -0,0 +1,102 @@
+{
+  "__type__":"TestrunResult",
+  "results_version":9,
+  "name":"normal-run",
+  "uname":"Linux hostname 4.18.0-1-amd64 #1 SMP Debian 4.18.6-1 (2018-09-06) x86_64",
+  "time_elapsed":{
+    "__type__":"TimeAttribute",
+    "start":1539953735.1110389,
+    "end":1539953735.1723731
+  },
+  "tests":{
+    "igt@successtest@first-subtest":{
+      "out":"Starting subtest: first-subtest\nSubtest first-subtest: SUCCESS (0.000s)\n",
+      "igt-version":"IGT-Version: 1.23-g0c763bfd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)",
+      "result":"pass",
+      "time":{
+        "__type__":"TimeAttribute",
+        "start":0,
+        "end":0
+      },
+      "err":"Starting subtest: first-subtest\nSubtest first-subtest: SUCCESS (0.000s)\n",
+      "dmesg":"<6> [3216186.095083] Console: switching to colour dummy device 80x25\n<6> [3216186.095097] [IGT] successtest: executing\n<6> [3216186.101115] [IGT] successtest: starting subtest first-subtest\n<6> [3216186.101160] [IGT] successtest: exiting, ret=0\n<6> [3216186.101299] Console: switching to colour frame buffer device 240x75\n"
+    },
+    "igt@successtest@second-subtest":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+    },
+    "igt@skippers@skip-one":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+      },
+    "igt@skippers@skip-two":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+    }
+  },
+  "totals":{
+    "":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":3,
+      "fail":0,
+      "warn":0
+    },
+    "root":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":3,
+      "fail":0,
+      "warn":0
+    },
+    "igt@successtest":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":1,
+      "fail":0,
+      "warn":0
+    },
+    "igt@skippers":{
+      "crash":0,
+      "pass":0,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":2,
+      "fail":0,
+      "warn":0
+    }
+  },
+  "runtimes":{
+    "igt@successtest":{
+      "time":{
+        "__type__":"TimeAttribute",
+        "start":0,
+        "end":0.014
+      }
+    },
+  }
+}
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/starttime.txt b/runner/json_tests_data/notrun-results-multiple-mode/starttime.txt
new file mode 100644
index 00000000..ae038f18
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/starttime.txt
@@ -0,0 +1 @@
+1539953735.111039
diff --git a/runner/json_tests_data/notrun-results-multiple-mode/uname.txt b/runner/json_tests_data/notrun-results-multiple-mode/uname.txt
new file mode 100644
index 00000000..a7aef6f7
--- /dev/null
+++ b/runner/json_tests_data/notrun-results-multiple-mode/uname.txt
@@ -0,0 +1 @@
+Linux hostname 4.18.0-1-amd64 #1 SMP Debian 4.18.6-1 (2018-09-06) x86_64
diff --git a/runner/json_tests_data/notrun-results/0/dmesg.txt b/runner/json_tests_data/notrun-results/0/dmesg.txt
new file mode 100644
index 00000000..a189e704
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/0/dmesg.txt
@@ -0,0 +1,5 @@
+6,951,3216186095083,-;Console: switching to colour dummy device 80x25
+14,952,3216186095097,-;[IGT] successtest: executing
+14,953,3216186101115,-;[IGT] successtest: starting subtest first-subtest
+14,954,3216186101160,-;[IGT] successtest: exiting, ret=0
+6,955,3216186101299,-;Console: switching to colour frame buffer device 240x75
diff --git a/runner/json_tests_data/notrun-results/0/err.txt b/runner/json_tests_data/notrun-results/0/err.txt
new file mode 100644
index 00000000..5dc78057
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/0/err.txt
@@ -0,0 +1,2 @@
+Starting subtest: first-subtest
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/notrun-results/0/journal.txt b/runner/json_tests_data/notrun-results/0/journal.txt
new file mode 100644
index 00000000..86a30e07
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/0/journal.txt
@@ -0,0 +1,2 @@
+first-subtest
+exit:0 (0.014s)
diff --git a/runner/json_tests_data/notrun-results/0/out.txt b/runner/json_tests_data/notrun-results/0/out.txt
new file mode 100644
index 00000000..5946bf31
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/0/out.txt
@@ -0,0 +1,3 @@
+IGT-Version: 1.23-g0c763bfd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: first-subtest
+Subtest first-subtest: SUCCESS (0.000s)
diff --git a/runner/json_tests_data/notrun-results/README.txt b/runner/json_tests_data/notrun-results/README.txt
new file mode 100644
index 00000000..4355aebd
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/README.txt
@@ -0,0 +1,4 @@
+Tests that are not executed are expected to be present in results.json
+when possible. If run without --multiple-mode, we can distinguish
+tests without subtests from tests where we attempt to execute all
+subtests.
diff --git a/runner/json_tests_data/notrun-results/endtime.txt b/runner/json_tests_data/notrun-results/endtime.txt
new file mode 100644
index 00000000..635f6ae9
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/endtime.txt
@@ -0,0 +1 @@
+1539953735.172373
diff --git a/runner/json_tests_data/notrun-results/joblist.txt b/runner/json_tests_data/notrun-results/joblist.txt
new file mode 100644
index 00000000..31ef8413
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/joblist.txt
@@ -0,0 +1,5 @@
+successtest first-subtest
+successtest second-subtest
+no-subtests
+skippers skip-one
+skippers skip-two
diff --git a/runner/json_tests_data/notrun-results/metadata.txt b/runner/json_tests_data/notrun-results/metadata.txt
new file mode 100644
index 00000000..1316560d
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/metadata.txt
@@ -0,0 +1,12 @@
+abort_on_error : 0
+name : normal-run
+dry_run : 0
+sync : 0
+log_level : 0
+overwrite : 0
+multiple_mode : 0
+inactivity_timeout : 0
+use_watchdog : 0
+piglit_style_dmesg : 0
+test_root : /path/does/not/exist
+results_path : /path/does/not/exist
diff --git a/runner/json_tests_data/notrun-results/reference.json b/runner/json_tests_data/notrun-results/reference.json
new file mode 100644
index 00000000..4c9a5ce2
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/reference.json
@@ -0,0 +1,120 @@
+{
+  "__type__":"TestrunResult",
+  "results_version":9,
+  "name":"normal-run",
+  "uname":"Linux hostname 4.18.0-1-amd64 #1 SMP Debian 4.18.6-1 (2018-09-06) x86_64",
+  "time_elapsed":{
+    "__type__":"TimeAttribute",
+    "start":1539953735.1110389,
+    "end":1539953735.1723731
+  },
+  "tests":{
+    "igt@successtest@first-subtest":{
+      "out":"Starting subtest: first-subtest\nSubtest first-subtest: SUCCESS (0.000s)\n",
+      "igt-version":"IGT-Version: 1.23-g0c763bfd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)",
+      "result":"pass",
+      "time":{
+        "__type__":"TimeAttribute",
+        "start":0,
+        "end":0
+      },
+      "err":"Starting subtest: first-subtest\nSubtest first-subtest: SUCCESS (0.000s)\n",
+      "dmesg":"<6> [3216186.095083] Console: switching to colour dummy device 80x25\n<6> [3216186.095097] [IGT] successtest: executing\n<6> [3216186.101115] [IGT] successtest: starting subtest first-subtest\n<6> [3216186.101160] [IGT] successtest: exiting, ret=0\n<6> [3216186.101299] Console: switching to colour frame buffer device 240x75\n"
+    },
+    "igt@successtest@second-subtest":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+    },
+    "igt@no-subtests":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+    },
+    "igt@skippers@skip-one":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+      },
+    "igt@skippers@skip-two":{
+      "result":"notrun",
+	"out":"",
+	"err":"",
+      "dmesg":""
+    }
+  },
+  "totals":{
+    "":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":4,
+      "fail":0,
+      "warn":0
+    },
+    "root":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":4,
+      "fail":0,
+      "warn":0
+    },
+    "igt@successtest":{
+      "crash":0,
+      "pass":1,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":1,
+      "fail":0,
+      "warn":0
+    },
+    "igt@no-subtests":{
+      "crash":0,
+      "pass":0,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":1,
+      "fail":0,
+      "warn":0
+    },
+    "igt@skippers":{
+      "crash":0,
+      "pass":0,
+      "dmesg-fail":0,
+      "dmesg-warn":0,
+      "skip":0,
+      "incomplete":0,
+      "timeout":0,
+      "notrun":2,
+      "fail":0,
+      "warn":0
+    }
+  },
+  "runtimes":{
+    "igt@successtest":{
+      "time":{
+        "__type__":"TimeAttribute",
+        "start":0,
+        "end":0.014
+      }
+    },
+  }
+}
diff --git a/runner/json_tests_data/notrun-results/starttime.txt b/runner/json_tests_data/notrun-results/starttime.txt
new file mode 100644
index 00000000..ae038f18
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/starttime.txt
@@ -0,0 +1 @@
+1539953735.111039
diff --git a/runner/json_tests_data/notrun-results/uname.txt b/runner/json_tests_data/notrun-results/uname.txt
new file mode 100644
index 00000000..a7aef6f7
--- /dev/null
+++ b/runner/json_tests_data/notrun-results/uname.txt
@@ -0,0 +1 @@
+Linux hostname 4.18.0-1-amd64 #1 SMP Debian 4.18.6-1 (2018-09-06) x86_64
diff --git a/runner/resultgen.c b/runner/resultgen.c
index a62e400e..b7e41ec1 100644
--- a/runner/resultgen.c
+++ b/runner/resultgen.c
@@ -977,6 +977,45 @@ static bool parse_test_directory(int dirfd,
 	return true;
 }
 
+static void try_add_notrun_results(const struct job_list_entry *entry,
+				   const struct settings *settings,
+				   struct results *results)
+{
+	struct subtests subtests = {};
+	struct json_object *current_test;
+	size_t i;
+
+	if (entry->subtest_count == 0) {
+		char piglit_name[256];
+
+		/* We cannot distinguish no-subtests from run-all-subtests in multiple-mode */
+		if (settings->multiple_mode)
+			return;
+		generate_piglit_name(entry->binary, NULL, piglit_name, sizeof(piglit_name));
+		current_test = get_or_create_json_object(results->tests, piglit_name);
+		json_object_object_add(current_test, "out", json_object_new_string(""));
+		json_object_object_add(current_test, "err", json_object_new_string(""));
+		json_object_object_add(current_test, "dmesg", json_object_new_string(""));
+		json_object_object_add(current_test, "result", json_object_new_string("notrun"));
+	}
+
+	for (i = 0; i < entry->subtest_count; i++) {
+		char piglit_name[256];
+
+		generate_piglit_name(entry->binary, entry->subtests[i], piglit_name, sizeof(piglit_name));
+		current_test = get_or_create_json_object(results->tests, piglit_name);
+		json_object_object_add(current_test, "out", json_object_new_string(""));
+		json_object_object_add(current_test, "err", json_object_new_string(""));
+		json_object_object_add(current_test, "dmesg", json_object_new_string(""));
+		json_object_object_add(current_test, "result", json_object_new_string("notrun"));
+		add_subtest(&subtests, strdup(entry->subtests[i]));
+	}
+
+	add_to_totals(entry->binary, &subtests, results);
+	// This function is appearing from another patch that is currently in review.
+	// free_subtests(&subtests);
+}
+
 static void create_result_root_nodes(struct json_object *root,
 				     struct results *results)
 {
@@ -1066,8 +1105,8 @@ struct json_object *generate_results_json(int dirfd)
 
 		snprintf(name, 16, "%zd", i);
 		if ((testdirfd = openat(dirfd, name, O_DIRECTORY | O_RDONLY)) < 0) {
-			fprintf(stderr, "Warning: Cannot open result directory %s\n", name);
-			break;
+			try_add_notrun_results(&job_list.entries[i], &settings, &results);
+			continue;
 		}
 
 		if (!parse_test_directory(testdirfd, &job_list.entries[i], &settings, &results)) {
diff --git a/runner/runner_json_tests.c b/runner/runner_json_tests.c
index 758700d4..4584ab5f 100644
--- a/runner/runner_json_tests.c
+++ b/runner/runner_json_tests.c
@@ -154,6 +154,8 @@ static const char *dirnames[] = {
 	"piglit-style-dmesg",
 	"incomplete-before-any-subtests",
 	"dmesg-results",
+	"notrun-results",
+	"notrun-results-multiple-mode",
 };
 
 igt_main
-- 
2.18.0

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply related	[flat|nested] 7+ messages in thread
* [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
@ 2018-11-13 13:24 ` Patchwork
  2018-11-13 13:30 ` [igt-dev] [PATCH i-g-t 1/1] " Chris Wilson
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 7+ messages in thread
From: Patchwork @ 2018-11-13 13:24 UTC (permalink / raw)
  To: Petri Latvala; +Cc: igt-dev

== Series Details ==

Series: series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
URL   : https://patchwork.freedesktop.org/series/52418/
State : failure

== Summary ==

= CI Bug Log - changes from IGT_4714 -> IGTPW_2054 =

== Summary - FAILURE ==

  Serious unknown changes coming with IGTPW_2054 absolutely need to be
  verified manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_2054, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/52418/revisions/1/mbox/

== Possible new issues ==

  Here are the unknown changes that may have been introduced in IGTPW_2054:

  === IGT changes ===

    ==== Possible regressions ====

    igt@kms_chamelium@dp-crc-fast:
      fi-kbl-7500u:       PASS -> FAIL +2

    
    ==== Warnings ====

    igt@kms_chamelium@common-hpd-after-suspend:
      fi-kbl-7500u:       DMESG-WARN (fdo#105079, fdo#105602, fdo#102505) -> FAIL

    igt@kms_flip@basic-flip-vs-modeset:
      fi-kbl-x1275:       SKIP -> PASS +36

    igt@prime_vgem@basic-fence-flip:
      fi-cfl-s3:          SKIP -> PASS

    
== Known issues ==

  Here are the changes found in IGTPW_2054 that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@drv_selftest@live_contexts:
      fi-cfl-8109u:       PASS -> DMESG-FAIL (fdo#108698)
      fi-icl-u:           NOTRUN -> DMESG-FAIL (fdo#108569)

    
    ==== Possible fixes ====

    igt@drv_module_reload@basic-reload:
      fi-blb-e6850:       INCOMPLETE (fdo#107718) -> PASS

    igt@kms_pipe_crc_basic@suspend-read-crc-pipe-b:
      fi-byt-clapper:     FAIL (fdo#107362, fdo#103191) -> PASS

    
  fdo#102505 https://bugs.freedesktop.org/show_bug.cgi?id=102505
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#105079 https://bugs.freedesktop.org/show_bug.cgi?id=105079
  fdo#105602 https://bugs.freedesktop.org/show_bug.cgi?id=105602
  fdo#107362 https://bugs.freedesktop.org/show_bug.cgi?id=107362
  fdo#107718 https://bugs.freedesktop.org/show_bug.cgi?id=107718
  fdo#108569 https://bugs.freedesktop.org/show_bug.cgi?id=108569
  fdo#108698 https://bugs.freedesktop.org/show_bug.cgi?id=108698


== Participating hosts (53 -> 46) ==

  Additional (1): fi-icl-u 
  Missing    (8): fi-kbl-soraka fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-icl-u2 fi-bsw-cyan fi-snb-2520m fi-ctg-p8600 


== Build changes ==

    * IGT: IGT_4714 -> IGTPW_2054

  CI_DRM_5106: 852b9329fbb6ea8bdbb3dac78328aae73d919305 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2054: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/
  IGT_4714: cab148ca3ec904a94d0cd43476cf7e1f8663f906 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
  2018-11-13 13:24 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/1] " Patchwork
@ 2018-11-13 13:30 ` Chris Wilson
  2018-11-14 11:33   ` Petri Latvala
  2018-11-13 13:52 ` [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/1] " Patchwork
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 7+ messages in thread
From: Chris Wilson @ 2018-11-13 13:30 UTC (permalink / raw)
  To: Petri Latvala, igt-dev; +Cc: Petri Latvala, Martin Peres

Quoting Petri Latvala (2018-11-13 13:04:30)
> When possible, all tests we know we were going to attempt to execute
> now appear in the results as "notrun". The only known case where it's
> not possible to add an explicit "notrun" is when running in
> multiple-mode, because "no subtests" and "run all subtests, we didn't
> list them beforehand" are represented the same.

Sounds strange for a runner with which we can integrate with the igt
framework not know whether or not a test started. Although the easiest
procedure would seem to be that all tests should include at least one
igt_subtest().

Just my 2c,
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
  2018-11-13 13:24 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/1] " Patchwork
  2018-11-13 13:30 ` [igt-dev] [PATCH i-g-t 1/1] " Chris Wilson
@ 2018-11-13 13:52 ` Patchwork
  2018-11-13 15:40 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
  2018-12-05 14:51 ` [igt-dev] [PATCH i-g-t 1/1] " Martin Peres
  4 siblings, 0 replies; 7+ messages in thread
From: Patchwork @ 2018-11-13 13:52 UTC (permalink / raw)
  To: Petri Latvala; +Cc: igt-dev

== Series Details ==

Series: series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
URL   : https://patchwork.freedesktop.org/series/52418/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4714 -> IGTPW_2054 =

== Summary - WARNING ==

  Minor unknown changes coming with IGTPW_2054 need to be verified
  manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_2054, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/52418/revisions/1/mbox/

== Possible new issues ==

  Here are the unknown changes that may have been introduced in IGTPW_2054:

  === IGT changes ===

    ==== Warnings ====

    igt@kms_flip@basic-flip-vs-modeset:
      fi-kbl-x1275:       SKIP -> PASS +36

    igt@prime_vgem@basic-fence-flip:
      fi-cfl-s3:          SKIP -> PASS

    
== Known issues ==

  Here are the changes found in IGTPW_2054 that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@drv_selftest@live_contexts:
      fi-cfl-8109u:       PASS -> DMESG-FAIL (fdo#108698)
      fi-icl-u:           NOTRUN -> DMESG-FAIL (fdo#108569)

    igt@kms_chamelium@dp-crc-fast:
      fi-kbl-7500u:       PASS -> FAIL (fdo#108725) +1

    igt@kms_chamelium@dp-hpd-fast:
      fi-kbl-7500u:       PASS -> FAIL (fdo#108724)

    
    ==== Possible fixes ====

    igt@drv_module_reload@basic-reload:
      fi-blb-e6850:       INCOMPLETE (fdo#107718) -> PASS

    igt@kms_pipe_crc_basic@suspend-read-crc-pipe-b:
      fi-byt-clapper:     FAIL (fdo#107362, fdo#103191) -> PASS

    
    ==== Warnings ====

    igt@kms_chamelium@common-hpd-after-suspend:
      fi-kbl-7500u:       DMESG-WARN (fdo#102505, fdo#105079, fdo#105602) -> FAIL (fdo#108724)

    
  fdo#102505 https://bugs.freedesktop.org/show_bug.cgi?id=102505
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#105079 https://bugs.freedesktop.org/show_bug.cgi?id=105079
  fdo#105602 https://bugs.freedesktop.org/show_bug.cgi?id=105602
  fdo#107362 https://bugs.freedesktop.org/show_bug.cgi?id=107362
  fdo#107718 https://bugs.freedesktop.org/show_bug.cgi?id=107718
  fdo#108569 https://bugs.freedesktop.org/show_bug.cgi?id=108569
  fdo#108698 https://bugs.freedesktop.org/show_bug.cgi?id=108698
  fdo#108724 https://bugs.freedesktop.org/show_bug.cgi?id=108724
  fdo#108725 https://bugs.freedesktop.org/show_bug.cgi?id=108725


== Participating hosts (53 -> 46) ==

  Additional (1): fi-icl-u 
  Missing    (8): fi-kbl-soraka fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-icl-u2 fi-bsw-cyan fi-snb-2520m fi-ctg-p8600 


== Build changes ==

    * IGT: IGT_4714 -> IGTPW_2054

  CI_DRM_5106: 852b9329fbb6ea8bdbb3dac78328aae73d919305 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2054: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/
  IGT_4714: cab148ca3ec904a94d0cd43476cf7e1f8663f906 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
* [igt-dev] ✓ Fi.CI.IGT: success for series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
                   ` (2 preceding siblings ...)
  2018-11-13 13:52 ` [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/1] " Patchwork
@ 2018-11-13 15:40 ` Patchwork
  2018-12-05 14:51 ` [igt-dev] [PATCH i-g-t 1/1] " Martin Peres
  4 siblings, 0 replies; 7+ messages in thread
From: Patchwork @ 2018-11-13 15:40 UTC (permalink / raw)
  To: Petri Latvala; +Cc: igt-dev

== Series Details ==

Series: series starting with [i-g-t,1/1] runner: Add explicit "notrun" results for tests that were not executed
URL   : https://patchwork.freedesktop.org/series/52418/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4714_full -> IGTPW_2054_full =

== Summary - SUCCESS ==

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/52418/revisions/1/mbox/

== Known issues ==

  Here are the changes found in IGTPW_2054_full that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@kms_busy@extended-modeset-hang-newfb-with-reset-render-c:
      shard-kbl:          NOTRUN -> DMESG-WARN (fdo#107956) +1

    igt@kms_busy@extended-pageflip-modeset-hang-oldfb-render-a:
      shard-apl:          PASS -> DMESG-WARN (fdo#107956)

    igt@kms_busy@extended-pageflip-modeset-hang-oldfb-render-c:
      shard-glk:          NOTRUN -> DMESG-WARN (fdo#107956) +1

    igt@kms_flip@dpms-vs-vblank-race:
      shard-glk:          NOTRUN -> FAIL (fdo#103060)

    igt@kms_flip@flip-vs-expired-vblank:
      shard-glk:          NOTRUN -> FAIL (fdo#105363, fdo#102887)

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-mmap-cpu:
      shard-apl:          PASS -> FAIL (fdo#103167) +1

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-move:
      shard-glk:          NOTRUN -> FAIL (fdo#103167) +4

    igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-draw-mmap-wc:
      shard-glk:          PASS -> FAIL (fdo#103167) +4

    igt@kms_plane@plane-panning-bottom-right-suspend-pipe-b-planes:
      shard-glk:          PASS -> INCOMPLETE (fdo#103359, k.org#198133)

    igt@kms_plane_alpha_blend@pipe-c-alpha-transparant-fb:
      shard-glk:          NOTRUN -> FAIL (fdo#108145) +8

    igt@kms_plane_multiple@atomic-pipe-a-tiling-yf:
      shard-glk:          NOTRUN -> FAIL (fdo#103166)

    igt@kms_plane_multiple@atomic-pipe-b-tiling-x:
      shard-apl:          PASS -> FAIL (fdo#103166) +1

    igt@kms_properties@connector-properties-atomic:
      shard-glk:          NOTRUN -> FAIL (fdo#108642)
      shard-hsw:          NOTRUN -> FAIL (fdo#108642)

    igt@kms_vblank@pipe-c-ts-continuation-suspend:
      shard-hsw:          PASS -> FAIL (fdo#104894)

    
    ==== Possible fixes ====

    igt@drv_suspend@forcewake:
      shard-kbl:          INCOMPLETE (fdo#103665) -> PASS

    igt@drv_suspend@shrink:
      shard-snb:          INCOMPLETE (fdo#106886, fdo#105411) -> PASS

    igt@gem_ppgtt@blt-vs-render-ctx0:
      shard-kbl:          INCOMPLETE (fdo#103665, fdo#106023, fdo#106887) -> PASS

    igt@kms_available_modes_crc@available_mode_test_crc:
      shard-apl:          FAIL (fdo#106641) -> PASS

    igt@kms_cursor_crc@cursor-128x42-sliding:
      shard-kbl:          FAIL (fdo#103232) -> PASS +3

    igt@kms_cursor_crc@cursor-256x85-random:
      shard-glk:          FAIL (fdo#103232) -> PASS +2

    igt@kms_cursor_crc@cursor-64x21-sliding:
      shard-apl:          FAIL (fdo#103232) -> PASS +2

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-cur-indfb-draw-pwrite:
      shard-apl:          FAIL (fdo#103167) -> PASS +1

    igt@kms_frontbuffer_tracking@fbc-2p-primscrn-spr-indfb-onoff:
      shard-glk:          FAIL (fdo#103167) -> PASS +3

    igt@kms_plane_multiple@atomic-pipe-b-tiling-y:
      shard-apl:          FAIL (fdo#103166) -> PASS +1

    igt@kms_plane_multiple@atomic-pipe-b-tiling-yf:
      shard-kbl:          FAIL (fdo#103166) -> PASS

    igt@kms_setmode@basic:
      shard-kbl:          FAIL (fdo#99912) -> PASS

    igt@kms_universal_plane@universal-plane-pipe-a-functional:
      shard-glk:          FAIL (fdo#103166) -> PASS +3

    igt@kms_vblank@pipe-a-query-busy:
      shard-snb:          INCOMPLETE (fdo#105411) -> PASS

    igt@perf_pmu@busy-start-vcs0:
      shard-apl:          DMESG-WARN (fdo#103558, fdo#105602) -> PASS

    
  fdo#102887 https://bugs.freedesktop.org/show_bug.cgi?id=102887
  fdo#103060 https://bugs.freedesktop.org/show_bug.cgi?id=103060
  fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
  fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
  fdo#103232 https://bugs.freedesktop.org/show_bug.cgi?id=103232
  fdo#103359 https://bugs.freedesktop.org/show_bug.cgi?id=103359
  fdo#103558 https://bugs.freedesktop.org/show_bug.cgi?id=103558
  fdo#103665 https://bugs.freedesktop.org/show_bug.cgi?id=103665
  fdo#104894 https://bugs.freedesktop.org/show_bug.cgi?id=104894
  fdo#105363 https://bugs.freedesktop.org/show_bug.cgi?id=105363
  fdo#105411 https://bugs.freedesktop.org/show_bug.cgi?id=105411
  fdo#105602 https://bugs.freedesktop.org/show_bug.cgi?id=105602
  fdo#106023 https://bugs.freedesktop.org/show_bug.cgi?id=106023
  fdo#106641 https://bugs.freedesktop.org/show_bug.cgi?id=106641
  fdo#106886 https://bugs.freedesktop.org/show_bug.cgi?id=106886
  fdo#106887 https://bugs.freedesktop.org/show_bug.cgi?id=106887
  fdo#107956 https://bugs.freedesktop.org/show_bug.cgi?id=107956
  fdo#108145 https://bugs.freedesktop.org/show_bug.cgi?id=108145
  fdo#108642 https://bugs.freedesktop.org/show_bug.cgi?id=108642
  fdo#99912 https://bugs.freedesktop.org/show_bug.cgi?id=99912
  k.org#198133 https://bugzilla.kernel.org/show_bug.cgi?id=198133


== Participating hosts (6 -> 5) ==

  Missing    (1): shard-skl 


== Build changes ==

    * IGT: IGT_4714 -> IGTPW_2054

  CI_DRM_5106: 852b9329fbb6ea8bdbb3dac78328aae73d919305 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_2054: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/
  IGT_4714: cab148ca3ec904a94d0cd43476cf7e1f8663f906 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2054/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:30 ` [igt-dev] [PATCH i-g-t 1/1] " Chris Wilson
@ 2018-11-14 11:33   ` Petri Latvala
  0 siblings, 0 replies; 7+ messages in thread
From: Petri Latvala @ 2018-11-14 11:33 UTC (permalink / raw)
  To: Chris Wilson; +Cc: igt-dev, Martin Peres

On Tue, Nov 13, 2018 at 01:30:26PM +0000, Chris Wilson wrote:
> Quoting Petri Latvala (2018-11-13 13:04:30)
> > When possible, all tests we know we were going to attempt to execute
> > now appear in the results as "notrun". The only known case where it's
> > not possible to add an explicit "notrun" is when running in
> > multiple-mode, because "no subtests" and "run all subtests, we didn't
> > list them beforehand" are represented the same.
> 
> Sounds strange for a runner with which we can integrate with the igt
> framework not know whether or not a test started. Although the easiest
> procedure would seem to be that all tests should include at least one
> igt_subtest().

That wouldn't work for the case I'm describing. Although it seems I
didn't describe it in enough detail. And there's a different case
lurking there too.

When running in multiple-mode, if you specified e.g. -t kms_flip, the
job list creation sees that the binary "kms_flip" is enough to match
that filter, ergo, we run all subtests of that binary, we don't need
to execute kms_flip --list-subtests to get the subtest list
explicitly. The job list will contain:

 kms_flip

If we also specified e.g. -t testdisplay, the job list will contain:

 kms_flip
 testdisplay

Now, the purpose of this patch was for Martin to know from the results
what our pass rate is, and for that he needs the list of all tests,
and we can get there by explicitly getting a "notrun" for tests that
didn't get executed (due to an incomplete). We can't know what
kms_flip subtests to get the result for, because we didn't list
them. Knowing that it _has_ subtests doesn't help.

For Intel CI, this is a moot point because we use testlists
everywhere. Even with multiple-mode, we get the full list of subtests
explicitly stated and we can fill in "notrun" for them.


-- 
Petri Latvala
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed
  2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
                   ` (3 preceding siblings ...)
  2018-11-13 15:40 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
@ 2018-12-05 14:51 ` Martin Peres
  4 siblings, 0 replies; 7+ messages in thread
From: Martin Peres @ 2018-12-05 14:51 UTC (permalink / raw)
  To: Petri Latvala, igt-dev

On 13/11/2018 15:04, Petri Latvala wrote:
> When possible, all tests we know we were going to attempt to execute
> now appear in the results as "notrun". The only known case where it's
> not possible to add an explicit "notrun" is when running in
> multiple-mode, because "no subtests" and "run all subtests, we didn't
> list them beforehand" are represented the same.
> 
> Note: A function call is commented out in resultgen.c because it needs
> the function to exist, and that's in a patch that is still in flight.

Thanks for doing this!

I checked out the code and tests, and this looks good. However, since I
did not review much code before, I will demote my R-b to:

Acked-by: Martin Peres <martin.peres@linux.intel.com>
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-12-05 14:51 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-13 13:04 [igt-dev] [PATCH i-g-t 1/1] runner: Add explicit "notrun" results for tests that were not executed Petri Latvala
2018-11-13 13:24 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/1] " Patchwork
2018-11-13 13:30 ` [igt-dev] [PATCH i-g-t 1/1] " Chris Wilson
2018-11-14 11:33   ` Petri Latvala
2018-11-13 13:52 ` [igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/1] " Patchwork
2018-11-13 15:40 ` [igt-dev] ✓ Fi.CI.IGT: " Patchwork
2018-12-05 14:51 ` [igt-dev] [PATCH i-g-t 1/1] " Martin Peres

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.