[PATCH i-g-t] tests/tools_test: Make sure l3_parity is supported

[PATCH i-g-t] tests/tools_test: Make sure l3_parity is supported

[Abdiel Janulgue]

Check support before executing test.
v2: Skip test only if intel_l3_parity tool tells us to skip. (Petri)

bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101650
Cc: Petri Latvala <petri.latvala@intel.com>
Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
---
 tests/tools_test.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/tools_test.c b/tests/tools_test.c
index ccd165d..ebd4a9d 100644
--- a/tests/tools_test.c
+++ b/tests/tools_test.c
@@ -89,7 +89,8 @@ igt_main
 		igt_system_cmd(exec_return,
 			       "../tools/intel_l3_parity -r 0 -b 0 "
 			       "-s 0 -e");
-		igt_assert(exec_return == IGT_EXIT_SUCCESS);
+		igt_skip_on_f(exec_return == IGT_EXIT_SKIP,
+			      "intel_l3_parity not supported\n");
 
 		igt_system_cmd(exec_return,
 			       "../tools/intel_l3_parity -l | "
@@ -101,13 +102,14 @@ igt_main
 					       &val);
 			igt_assert(val == 1);
 		} else {
-			igt_fail(IGT_EXIT_FAILURE);
+			igt_skip("intel_l3_parity not supported\n");
 		}
 
 		igt_system_cmd(exec_return,
 			       "../tools/intel_l3_parity -r 0 -b 0 "
 			       "-s 0 -e");
-		igt_assert(exec_return == IGT_EXIT_SUCCESS);
+		igt_skip_on_f(exec_return == IGT_EXIT_SKIP,
+			      "intel_l3_parity not supported\n");
 
 		/* Check that we can clear remaps */
 		igt_system_cmd(exec_return,
@@ -119,7 +121,7 @@ igt_main
 					       &val);
 			igt_assert(val == 1);
 		} else {
-			igt_fail(IGT_EXIT_FAILURE);
+			igt_skip("intel_l3_parity not supported\n");
 		}
 	}
 
-- 
2.9.3

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.BAT: success for tests/tools_test: Make sure l3_parity is supported

[Patchwork]

== Series Details ==

Series: tests/tools_test: Make sure l3_parity is supported
URL   : https://patchwork.freedesktop.org/series/29926/
State : success

== Summary ==

IGT patchset tested on top of latest successful build
4a1c8daff2005e2cbfe980d63bc0a0fb09cb017d igt/gem_ringfill: Prime execbuf before measuring ring size

with latest DRM-Tip kernel build CI_DRM_3051
d987a6e6aed6 drm-tip: 2017y-09m-06d-23h-36m-35s UTC integration manifest

Test kms_cursor_legacy:
        Subgroup basic-busy-flip-before-cursor-legacy:
                fail       -> PASS       (fi-snb-2600) fdo#100215
Test pm_rpm:
        Subgroup basic-pci-d3-state:
                pass       -> SKIP       (fi-cfl-s) fdo#102294

fdo#100215 https://bugs.freedesktop.org/show_bug.cgi?id=100215
fdo#102294 https://bugs.freedesktop.org/show_bug.cgi?id=102294

fi-bdw-5557u     total:289  pass:268  dwarn:0   dfail:0   fail:0   skip:21  time:462s
fi-bdw-gvtdvm    total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:445s
fi-blb-e6850     total:289  pass:224  dwarn:1   dfail:0   fail:0   skip:64  time:362s
fi-bsw-n3050     total:289  pass:243  dwarn:0   dfail:0   fail:0   skip:46  time:558s
fi-bwr-2160      total:289  pass:184  dwarn:0   dfail:0   fail:0   skip:105 time:254s
fi-bxt-j4205     total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:527s
fi-byt-j1900     total:289  pass:254  dwarn:1   dfail:0   fail:0   skip:34  time:528s
fi-byt-n2820     total:289  pass:250  dwarn:1   dfail:0   fail:0   skip:38  time:525s
fi-cfl-s         total:289  pass:249  dwarn:4   dfail:0   fail:0   skip:36  time:468s
fi-elk-e7500     total:289  pass:230  dwarn:0   dfail:0   fail:0   skip:59  time:441s
fi-glk-2a        total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:614s
fi-hsw-4770      total:289  pass:263  dwarn:0   dfail:0   fail:0   skip:26  time:452s
fi-hsw-4770r     total:289  pass:263  dwarn:0   dfail:0   fail:0   skip:26  time:432s
fi-ilk-650       total:289  pass:229  dwarn:0   dfail:0   fail:0   skip:60  time:427s
fi-ivb-3520m     total:289  pass:261  dwarn:0   dfail:0   fail:0   skip:28  time:515s
fi-ivb-3770      total:289  pass:261  dwarn:0   dfail:0   fail:0   skip:28  time:483s
fi-kbl-7500u     total:289  pass:264  dwarn:1   dfail:0   fail:0   skip:24  time:516s
fi-kbl-7560u     total:289  pass:270  dwarn:0   dfail:0   fail:0   skip:19  time:605s
fi-kbl-r         total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:603s
fi-pnv-d510      total:289  pass:223  dwarn:1   dfail:0   fail:0   skip:65  time:535s
fi-skl-6260u     total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:482s
fi-skl-6700k     total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:543s
fi-skl-6770hq    total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:517s
fi-skl-gvtdvm    total:289  pass:266  dwarn:0   dfail:0   fail:0   skip:23  time:451s
fi-skl-x1585l    total:289  pass:268  dwarn:0   dfail:0   fail:0   skip:21  time:503s
fi-snb-2520m     total:289  pass:251  dwarn:0   dfail:0   fail:0   skip:38  time:561s
fi-snb-2600      total:289  pass:250  dwarn:0   dfail:0   fail:0   skip:39  time:413s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_155/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✗ Fi.CI.IGT: failure for tests/tools_test: Make sure l3_parity is supported

[Patchwork]

== Series Details ==

Series: tests/tools_test: Make sure l3_parity is supported
URL   : https://patchwork.freedesktop.org/series/29926/
State : failure

== Summary ==

Test kms_setmode:
        Subgroup basic:
                fail       -> PASS       (shard-hsw) fdo#99912
Test kms_flip:
        Subgroup modeset-vs-vblank-race-interruptible:
                fail       -> PASS       (shard-hsw)
        Subgroup flip-vs-fences-interruptible:
                pass       -> FAIL       (shard-hsw)
        Subgroup wf_vblank-vs-modeset:
                pass       -> DMESG-WARN (shard-hsw)
Test tools_test:
        Subgroup sysfs_l3_parity:
                fail       -> SKIP       (shard-hsw) fdo#102255
Test perf:
        Subgroup blocking:
                pass       -> FAIL       (shard-hsw) fdo#102252
Test kms_atomic_transition:
        Subgroup plane-use-after-nonblocking-unbind:
                incomplete -> FAIL       (shard-hsw) fdo#101847

fdo#99912 https://bugs.freedesktop.org/show_bug.cgi?id=99912
fdo#102255 https://bugs.freedesktop.org/show_bug.cgi?id=102255
fdo#102252 https://bugs.freedesktop.org/show_bug.cgi?id=102252
fdo#101847 https://bugs.freedesktop.org/show_bug.cgi?id=101847

shard-hsw        total:2265 pass:1234 dwarn:1   dfail:0   fail:13  skip:1017 time:9618s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_155/shards.html
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH i-g-t] tests/tools_test: Make sure l3_parity is supported

[Petri Latvala]

On Thu, Sep 07, 2017 at 09:49:42AM +0300, Abdiel Janulgue wrote:
> Check support before executing test.
> v2: Skip test only if intel_l3_parity tool tells us to skip. (Petri)
> 
> bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=101650
> Cc: Petri Latvala <petri.latvala@intel.com>
> Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
> ---
>  tests/tools_test.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/tools_test.c b/tests/tools_test.c
> index ccd165d..ebd4a9d 100644
> --- a/tests/tools_test.c
> +++ b/tests/tools_test.c
> @@ -89,7 +89,8 @@ igt_main
>  		igt_system_cmd(exec_return,
>  			       "../tools/intel_l3_parity -r 0 -b 0 "
>  			       "-s 0 -e");
> -		igt_assert(exec_return == IGT_EXIT_SUCCESS);
> +		igt_skip_on_f(exec_return == IGT_EXIT_SKIP,
> +			      "intel_l3_parity not supported\n");
>

Asserting that exec_return equals IGT_EXIT_SUCCESS would be nice here,
after the skip_on_f.

Btw, use igt_assert_eq for that assert instead of using == so we get
the values in the failure log.


>  		igt_system_cmd(exec_return,
>  			       "../tools/intel_l3_parity -l | "
> @@ -101,13 +102,14 @@ igt_main
>  					       &val);
>  			igt_assert(val == 1);
>  		} else {
> -			igt_fail(IGT_EXIT_FAILURE);
> +			igt_skip("intel_l3_parity not supported\n");
>  		}
>


This part still causes problems that are misreported.

In this pipeline

  #  intel_l3_parity -l | grep whatever

the exit status of the shell is the exit status of grep, not
intel_l3_parity. Not to mention we don't get to see what
intel_l3_parity even printed here.

Make that call to igt_system_cmd just call intel_l3_parity -l, and
check for the "Row blah, Bank blah" string in your check_cmd_return
value function (needs to also be renamed and the comment explaining it
reworded, that makes no sense to me now and will make less sense with
that change). Make sure you account for the log buffer possibly having
more than just that one line, maybe like this:

-- begin untested code --

static bool find_the_line(const char *haystack, void *data)
{
	int *val = data;
	const char *needle = "Row blah etc";

	if (strstr(haystack, needle)) {
		*val = 1;
		return false;
	}

	return true;
}

/* pseudo in the subtest */
igt_system_cmd(exec_return, "path/intel_l3_parity -l");
if (succeeded) {
	igt_log_buffer_inspect(find_the_line, &val);
	igt_assert_eq(val, 1);
}

-- end untested code --

Bonus points for making the find_the_line function more reusable by
having the needle be specifiable.



>  		igt_system_cmd(exec_return,
>  			       "../tools/intel_l3_parity -r 0 -b 0 "
>  			       "-s 0 -e");
> -		igt_assert(exec_return == IGT_EXIT_SUCCESS);
> +		igt_skip_on_f(exec_return == IGT_EXIT_SKIP,
> +			      "intel_l3_parity not supported\n");
>

The "intel_l3_parity not supported" should be reworded everywhere to
state that the particular attempted operation is not supported.

Same here for asserting EXIT_SUCCESS after checking for EXIT_SKIP



-- 
Petri Latvala
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx