[igt-dev] [PATCH i-g-t 1/2] runner: Don't try to execute an empty set of subtests

[igt-dev] [PATCH i-g-t 1/2] runner: Don't try to execute an empty set of subtests

[Petri Latvala]

If we resume a test run with igt_resume, or if resume is done
automatically from a test timeout, the runner will execute the last
attempted test with the subtest selection set to original set minus
the subtests already journaled to have started. If this results in an
empty set, we get a harmless but misleading message from the test
saying

"igt_core-WARNING: Unknown subtest: subtest-name,!subtest-name"

If the journal already contains as many subtests as we have requested
(when we know the set), assume we have them all already and move to
the next job list entry instead.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Martin Peres <martin.peres@linux.intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 runner/executor.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/runner/executor.c b/runner/executor.c
index 4552b02c..8b87a421 100644
--- a/runner/executor.c
+++ b/runner/executor.c
@@ -140,7 +140,8 @@ static bool prune_from_journal(struct job_list_entry *entry, int fd)
 {
 	char *subtest;
 	FILE *f;
-	bool any_pruned = false;
+	size_t pruned = 0;
+	size_t old_count = entry->subtest_count;
 
 	/*
 	 * Each journal line is a subtest that has been started, or
@@ -169,11 +170,19 @@ static bool prune_from_journal(struct job_list_entry *entry, int fd)
 		prune_subtest(entry, subtest);
 
 		free(subtest);
-		any_pruned = true;
+		pruned++;
 	}
 
 	fclose(f);
-	return any_pruned;
+
+	/*
+	 * If we know the subtests we originally wanted to run, check
+	 * if we got an equal amount already.
+	 */
+	if (old_count > 0 && pruned >= old_count)
+		entry->binary[0] = '\0';
+
+	return pruned > 0;
 }
 
 static const char *filenames[_F_LAST] = {
-- 
2.18.0

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] [PATCH i-g-t 2/2] runner: Add unit test for resume when enough subtests are already run

[Petri Latvala]

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>
---
 runner/runner_tests.c | 54 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/runner/runner_tests.c b/runner/runner_tests.c
index 37d27123..b18af3a0 100644
--- a/runner/runner_tests.c
+++ b/runner/runner_tests.c
@@ -800,6 +800,60 @@ igt_main
 		}
 	}
 
+	igt_subtest_group {
+		char dirname[] = "tmpdirXXXXXX";
+		struct job_list list;
+		int dirfd = -1, subdirfd = -1, fd = -1;
+
+		igt_fixture {
+			init_job_list(&list);
+			igt_require(mkdtemp(dirname) != NULL);
+		}
+
+		igt_subtest("execute-initialize-all-subtests-started") {
+			struct execute_state state;
+			char *argv[] = { "runner",
+					 "--multiple-mode",
+					 "-t", "successtest@first-subtest",
+					 "-t", "successtest@second-subtest",
+					 testdatadir,
+					 dirname,
+			};
+			char journaltext[] = "first-subtest\nsecond-subtest\n";
+
+			igt_assert(parse_options(ARRAY_SIZE(argv), argv, &settings));
+			igt_assert(create_job_list(&list, &settings));
+			igt_assert(list.size == 1);
+			igt_assert(list.entries[0].subtest_count == 2);
+
+			igt_assert(serialize_settings(&settings));
+			igt_assert(serialize_job_list(&list, &settings));
+
+			igt_assert((dirfd = open(dirname, O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert(mkdirat(dirfd, "0", 0770) == 0);
+			igt_assert((subdirfd = openat(dirfd, "0", O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert((fd = openat(subdirfd, "journal.txt", O_CREAT | O_WRONLY | O_EXCL, 0660)) >= 0);
+			igt_assert(write(fd, journaltext, strlen(journaltext)) == strlen(journaltext));
+
+			free_job_list(&list);
+			free_settings(&settings);
+			igt_assert(initialize_execute_state_from_resume(dirfd, &state, &settings, &list));
+
+			/* All subtests are in journal, the entry should be considered completed */
+			igt_assert_eq(state.next, 1);
+			igt_assert_eq(list.size, 1);
+			igt_assert_eq(list.entries[0].subtest_count, 4);
+		}
+
+		igt_fixture {
+			close(fd);
+			close(subdirfd);
+			close(dirfd);
+			clear_directory(dirname);
+			free_job_list(&list);
+		}
+	}
+
 	igt_subtest_group {
 		char dirname[] = "tmpdirXXXXXX";
 		struct job_list list;
-- 
2.18.0

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,1/2] runner: Don't try to execute an empty set of subtests

[Patchwork]

== Series Details ==

Series: series starting with [i-g-t,1/2] runner: Don't try to execute an empty set of subtests
URL   : https://patchwork.freedesktop.org/series/50503/
State : success

== Summary ==

= CI Bug Log - changes from CI_DRM_4918 -> IGTPW_1902 =

== Summary - SUCCESS ==

  No regressions found.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/50503/revisions/1/mbox/

== Known issues ==

  Here are the changes found in IGTPW_1902 that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@amdgpu/amd_cs_nop@fork-gfx0:
      fi-kbl-8809g:       PASS -> DMESG-WARN (fdo#107762)

    igt@drv_module_reload@basic-reload:
      fi-glk-j4005:       PASS -> DMESG-WARN (fdo#106248, fdo#106725)

    igt@kms_flip@basic-flip-vs-wf_vblank:
      fi-glk-j4005:       PASS -> FAIL (fdo#100368)

    igt@kms_frontbuffer_tracking@basic:
      fi-byt-clapper:     PASS -> FAIL (fdo#103167)

    igt@kms_pipe_crc_basic@read-crc-pipe-a-frame-sequence:
      fi-skl-6700k2:      PASS -> FAIL (fdo#103191)

    igt@kms_pipe_crc_basic@suspend-read-crc-pipe-a:
      fi-bdw-samus:       PASS -> INCOMPLETE (fdo#107773)

    igt@pm_rpm@module-reload:
      fi-glk-j4005:       PASS -> DMESG-WARN (fdo#107726)

    
  fdo#100368 https://bugs.freedesktop.org/show_bug.cgi?id=100368
  fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#106248 https://bugs.freedesktop.org/show_bug.cgi?id=106248
  fdo#106725 https://bugs.freedesktop.org/show_bug.cgi?id=106725
  fdo#107726 https://bugs.freedesktop.org/show_bug.cgi?id=107726
  fdo#107762 https://bugs.freedesktop.org/show_bug.cgi?id=107762
  fdo#107773 https://bugs.freedesktop.org/show_bug.cgi?id=107773


== Participating hosts (48 -> 44) ==

  Additional (1): fi-cfl-guc 
  Missing    (5): fi-bsw-cyan fi-byt-squawks fi-icl-u2 fi-snb-2520m fi-hsw-4200u 


== Build changes ==

    * IGT: IGT_4662 -> IGTPW_1902

  CI_DRM_4918: f595aba3a6e2f6972bb158eb8434b58d22d0e5f0 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_1902: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1902/
  IGT_4662: ebf6a1dd1795e2f014ff3c47fe2eb4d5255845bd @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1902/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.IGT: success for series starting with [i-g-t,1/2] runner: Don't try to execute an empty set of subtests

[Patchwork]

== Series Details ==

Series: series starting with [i-g-t,1/2] runner: Don't try to execute an empty set of subtests
URL   : https://patchwork.freedesktop.org/series/50503/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4662_full -> IGTPW_1902_full =

== Summary - WARNING ==

  Minor unknown changes coming with IGTPW_1902_full need to be verified
  manually.
  
  If you think the reported changes have nothing to do with the changes
  introduced in IGTPW_1902_full, please notify your bug team to allow them
  to document this new failure mode, which will reduce false positives in CI.

  External URL: https://patchwork.freedesktop.org/api/1.0/series/50503/revisions/1/mbox/

== Possible new issues ==

  Here are the unknown changes that may have been introduced in IGTPW_1902_full:

  === IGT changes ===

    ==== Warnings ====

    igt@pm_rc6_residency@rc6-accuracy:
      shard-kbl:          PASS -> SKIP

    
== Known issues ==

  Here are the changes found in IGTPW_1902_full that come from known issues:

  === IGT changes ===

    ==== Issues hit ====

    igt@gem_ppgtt@blt-vs-render-ctxn:
      shard-kbl:          PASS -> INCOMPLETE (fdo#103665, fdo#106023)

    igt@kms_available_modes_crc@available_mode_test_crc:
      shard-apl:          PASS -> FAIL (fdo#106641)

    igt@kms_color@pipe-a-ctm-max:
      shard-apl:          PASS -> FAIL (fdo#108147)

    igt@kms_cursor_crc@cursor-256x256-random:
      shard-kbl:          PASS -> FAIL (fdo#103232) +1

    igt@kms_cursor_crc@cursor-64x21-sliding:
      shard-apl:          PASS -> FAIL (fdo#103232) +2

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-fullscreen:
      shard-apl:          PASS -> FAIL (fdo#103167)
      shard-kbl:          PASS -> FAIL (fdo#103167)

    igt@kms_frontbuffer_tracking@fbc-suspend:
      shard-kbl:          PASS -> INCOMPLETE (fdo#103665, fdo#105959)

    igt@kms_plane_multiple@atomic-pipe-a-tiling-x:
      shard-snb:          PASS -> FAIL (fdo#103166)
      shard-apl:          PASS -> FAIL (fdo#103166) +1

    igt@kms_setmode@basic:
      shard-apl:          PASS -> FAIL (fdo#99912)
      shard-kbl:          PASS -> FAIL (fdo#99912)

    igt@perf@blocking:
      shard-hsw:          PASS -> FAIL (fdo#102252)

    
    ==== Possible fixes ====

    igt@kms_busy@extended-modeset-hang-newfb-with-reset-render-b:
      shard-hsw:          DMESG-WARN (fdo#107956) -> PASS

    igt@kms_color@pipe-a-degamma:
      shard-apl:          FAIL (fdo#108145) -> PASS

    igt@kms_cursor_crc@cursor-128x128-suspend:
      shard-apl:          FAIL (fdo#103232, fdo#103191) -> PASS
      shard-kbl:          FAIL (fdo#103232, fdo#103191) -> PASS

    igt@kms_cursor_crc@cursor-128x42-onscreen:
      shard-apl:          FAIL (fdo#103232) -> PASS +3

    igt@kms_cursor_crc@cursor-64x21-onscreen:
      shard-kbl:          FAIL (fdo#103232) -> PASS +1

    igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-draw-pwrite:
      shard-apl:          FAIL (fdo#103167) -> PASS +1
      shard-kbl:          FAIL (fdo#103167) -> PASS

    igt@kms_plane_multiple@atomic-pipe-c-tiling-yf:
      shard-apl:          FAIL (fdo#103166) -> PASS +2

    igt@kms_rotation_crc@sprite-rotation-180:
      shard-snb:          FAIL (fdo#103925) -> PASS

    igt@pm_rpm@system-suspend-modeset:
      shard-kbl:          INCOMPLETE (fdo#103665, fdo#107807) -> PASS

    
  fdo#102252 https://bugs.freedesktop.org/show_bug.cgi?id=102252
  fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
  fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
  fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
  fdo#103232 https://bugs.freedesktop.org/show_bug.cgi?id=103232
  fdo#103665 https://bugs.freedesktop.org/show_bug.cgi?id=103665
  fdo#103925 https://bugs.freedesktop.org/show_bug.cgi?id=103925
  fdo#105959 https://bugs.freedesktop.org/show_bug.cgi?id=105959
  fdo#106023 https://bugs.freedesktop.org/show_bug.cgi?id=106023
  fdo#106641 https://bugs.freedesktop.org/show_bug.cgi?id=106641
  fdo#107807 https://bugs.freedesktop.org/show_bug.cgi?id=107807
  fdo#107956 https://bugs.freedesktop.org/show_bug.cgi?id=107956
  fdo#108145 https://bugs.freedesktop.org/show_bug.cgi?id=108145
  fdo#108147 https://bugs.freedesktop.org/show_bug.cgi?id=108147
  fdo#99912 https://bugs.freedesktop.org/show_bug.cgi?id=99912


== Participating hosts (6 -> 4) ==

  Missing    (2): shard-skl shard-glk 


== Build changes ==

    * IGT: IGT_4662 -> IGTPW_1902
    * Linux: CI_DRM_4915 -> CI_DRM_4918

  CI_DRM_4915: 26e7a7d954a9c28b97af8ca7813f430fd9117232 @ git://anongit.freedesktop.org/gfx-ci/linux
  CI_DRM_4918: f595aba3a6e2f6972bb158eb8434b58d22d0e5f0 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_1902: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1902/
  IGT_4662: ebf6a1dd1795e2f014ff3c47fe2eb4d5255845bd @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1902/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [PATCH i-g-t 1/2] runner: Don't try to execute an empty set of subtests

[Arkadiusz Hiler]

On Wed, Oct 03, 2018 at 04:08:45PM +0300, Petri Latvala wrote:
> If we resume a test run with igt_resume, or if resume is done
> automatically from a test timeout, the runner will execute the last
> attempted test with the subtest selection set to original set minus
> the subtests already journaled to have started. If this results in an
> empty set, we get a harmless but misleading message from the test
> saying
> 
> "igt_core-WARNING: Unknown subtest: subtest-name,!subtest-name"
> 
> If the journal already contains as many subtests as we have requested
> (when we know the set), assume we have them all already and move to
> the next job list entry instead.
> 
> Signed-off-by: Petri Latvala <petri.latvala@intel.com>
> Cc: Martin Peres <martin.peres@linux.intel.com>
> Cc: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

Reviewed-by: Arkadiusz Hiler <arkadiusz.hiler@intel.com>

for both this and the test
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev