* [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results
@ 2018-11-28 22:52 Chris Wilson
2018-11-28 22:52 ` [igt-dev] [PATCH i-g-t 2/2] lib: Only require we have i915.reset module parameter for allow-hang Chris Wilson
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Chris Wilson @ 2018-11-28 22:52 UTC (permalink / raw)
To: igt-dev
If an igt_fixture causes an independent subgroup to fail/skip and that
subgroup is not part of the execution set (--run-subtest) as no subtests
themselves failed or skipped, it should not count towards the overall
test exitcode.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108891
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
lib/igt_core.c | 57 ++++++++++++++++++++++++++++++--------------------
1 file changed, 34 insertions(+), 23 deletions(-)
diff --git a/lib/igt_core.c b/lib/igt_core.c
index 64883d640..1d47e171f 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -291,6 +291,10 @@ enum {
static int igt_exitcode = IGT_EXIT_SUCCESS;
static const char *command_str;
+static bool skipped_one = false;
+static bool succeeded_one = false;
+static bool failed_one = false;
+
static char* igt_log_domain_filter;
static struct {
char *entries[256];
@@ -919,15 +923,28 @@ bool __igt_run_subtest(const char *subtest_name)
}
if (skip_subtests_henceforth) {
+ const char *result;
+
+ if (skip_subtests_henceforth == FAIL) {
+ if (!failed_one) {
+ igt_exitcode = FAIL;
+ failed_one = true;
+ }
+ result = "FAIL";
+ } else {
+ skipped_one = true;
+ result = "SKIP";
+ }
+
printf("%sSubtest %s: %s%s\n",
- (!__igt_plain_output) ? "\x1b[1m" : "", subtest_name,
- skip_subtests_henceforth == SKIP ?
- "SKIP" : "FAIL", (!__igt_plain_output) ? "\x1b[0m" : "");
+ !__igt_plain_output ? "\x1b[1m" : "",
+ subtest_name, result,
+ !__igt_plain_output ? "\x1b[0m" : "");
fflush(stdout);
if (stderr_needs_sentinel)
- fprintf(stderr, "Subtest %s: %s\n", subtest_name,
- skip_subtests_henceforth == SKIP ?
- "SKIP" : "FAIL");
+ fprintf(stderr, "Subtest %s: %s\n",
+ subtest_name, result);
+
return false;
}
@@ -976,10 +993,6 @@ void __igt_subtest_group_restore(int save)
skip_subtests_henceforth = save;
}
-static bool skipped_one = false;
-static bool succeeded_one = false;
-static bool failed_one = false;
-
static void exit_subtest(const char *) __attribute__((noreturn));
static void exit_subtest(const char *result)
{
@@ -1018,7 +1031,6 @@ static void exit_subtest(const char *result)
void igt_skip(const char *f, ...)
{
va_list args;
- skipped_one = true;
assert(!test_child);
@@ -1029,6 +1041,7 @@ void igt_skip(const char *f, ...)
}
if (in_subtest) {
+ skipped_one = true;
exit_subtest("SKIP");
} else if (test_with_subtests) {
skip_subtests_henceforth = SKIP;
@@ -1116,11 +1129,6 @@ void igt_fail(int exitcode)
if (in_atexit_handler)
_exit(IGT_EXIT_FAILURE);
- if (!failed_one)
- igt_exitcode = exitcode;
-
- failed_one = true;
-
/* Silent exit, parent will do the yelling. */
if (test_child)
exit(exitcode);
@@ -1128,18 +1136,21 @@ void igt_fail(int exitcode)
_igt_log_buffer_dump();
if (in_subtest) {
+ if (!failed_one) {
+ igt_exitcode = exitcode;
+ failed_one = true;
+ }
+
if (exitcode == IGT_EXIT_TIMEOUT)
exit_subtest("TIMEOUT");
else
exit_subtest("FAIL");
+ } else if (test_with_subtests) {
+ skip_subtests_henceforth = FAIL;
+ assert(in_fixture);
+ __igt_fixture_end();
} else {
- assert(igt_can_fail());
-
- if (in_fixture) {
- skip_subtests_henceforth = FAIL;
- __igt_fixture_end();
- }
-
+ igt_exitcode = exitcode;
igt_exit();
}
}
--
2.20.0.rc1
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [igt-dev] [PATCH i-g-t 2/2] lib: Only require we have i915.reset module parameter for allow-hang
2018-11-28 22:52 [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results Chris Wilson
@ 2018-11-28 22:52 ` Chris Wilson
2018-11-29 1:25 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/2] lib: Defer application of igt_fixture results Patchwork
2018-11-29 17:16 ` [igt-dev] [PATCH i-g-t 1/2] " Daniel Vetter
2 siblings, 0 replies; 6+ messages in thread
From: Chris Wilson @ 2018-11-28 22:52 UTC (permalink / raw)
To: igt-dev
To control hang detection, we manipulate the i915.reset module
parameter. However, to be nice we should SKIP if we cannot modify the
parameter as opposed to outright FAILing.
References: https://bugs.freedesktop.org/show_bug.cgi?id=108891
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
lib/igt_gt.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/igt_gt.c b/lib/igt_gt.c
index a20619246..9683b3e5a 100644
--- a/lib/igt_gt.c
+++ b/lib/igt_gt.c
@@ -170,8 +170,8 @@ igt_hang_t igt_allow_hang(int fd, unsigned ctx, unsigned flags)
*/
igt_require_gem(fd);
- igt_assert(igt_sysfs_set_parameter
- (fd, "reset", "%d", INT_MAX /* any reset method */));
+ igt_require(igt_sysfs_set_parameter
+ (fd, "reset", "%d", INT_MAX /* any reset method */));
if (!igt_check_boolean_env_var("IGT_HANG", true))
igt_skip("hang injection disabled by user");
--
2.20.0.rc1
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/2] lib: Defer application of igt_fixture results
2018-11-28 22:52 [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results Chris Wilson
2018-11-28 22:52 ` [igt-dev] [PATCH i-g-t 2/2] lib: Only require we have i915.reset module parameter for allow-hang Chris Wilson
@ 2018-11-29 1:25 ` Patchwork
2018-11-29 17:16 ` [igt-dev] [PATCH i-g-t 1/2] " Daniel Vetter
2 siblings, 0 replies; 6+ messages in thread
From: Patchwork @ 2018-11-29 1:25 UTC (permalink / raw)
To: Chris Wilson; +Cc: igt-dev
== Series Details ==
Series: series starting with [i-g-t,1/2] lib: Defer application of igt_fixture results
URL : https://patchwork.freedesktop.org/series/53196/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_5219 -> IGTPW_2106
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_2106 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in IGTPW_2106, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://patchwork.freedesktop.org/api/1.0/series/53196/revisions/1/mbox/
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_2106:
### IGT changes ###
#### Possible regressions ####
* igt@i915_module_load@reload-with-fault-injection:
- fi-byt-clapper: PASS -> WARN
Known issues
------------
Here are the changes found in IGTPW_2106 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_ctx_create@basic-files:
- fi-bsw-n3050: PASS -> INCOMPLETE [fdo#108714]
* igt@i915_selftest@live_hangcheck:
- fi-kbl-7560u: PASS -> INCOMPLETE [fdo#108044]
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-b:
- fi-blb-e6850: PASS -> INCOMPLETE [fdo#107718]
* igt@pm_rpm@module-reload:
- fi-byt-clapper: PASS -> FAIL [fdo#108675]
* igt@vgem_basic@sysfs:
- fi-cfl-8109u: PASS -> DMESG-WARN [fdo#106107]
#### Possible fixes ####
* igt@kms_chamelium@hdmi-hpd-fast:
- {fi-kbl-7500u}: FAIL [fdo#108769] -> PASS
* igt@kms_pipe_crc_basic@read-crc-pipe-b:
- fi-byt-clapper: FAIL [fdo#107362] -> PASS
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-b:
- fi-byt-clapper: FAIL [fdo#103191] / [fdo#107362] -> PASS
* igt@prime_vgem@basic-fence-flip:
- fi-ilk-650: FAIL [fdo#104008] -> PASS
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#103191]: https://bugs.freedesktop.org/show_bug.cgi?id=103191
[fdo#104008]: https://bugs.freedesktop.org/show_bug.cgi?id=104008
[fdo#106107]: https://bugs.freedesktop.org/show_bug.cgi?id=106107
[fdo#107362]: https://bugs.freedesktop.org/show_bug.cgi?id=107362
[fdo#107718]: https://bugs.freedesktop.org/show_bug.cgi?id=107718
[fdo#108044]: https://bugs.freedesktop.org/show_bug.cgi?id=108044
[fdo#108675]: https://bugs.freedesktop.org/show_bug.cgi?id=108675
[fdo#108714]: https://bugs.freedesktop.org/show_bug.cgi?id=108714
[fdo#108769]: https://bugs.freedesktop.org/show_bug.cgi?id=108769
Participating hosts (50 -> 44)
------------------------------
Missing (6): fi-kbl-soraka fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-ctg-p8600
Build changes
-------------
* IGT: IGT_4735 -> IGTPW_2106
CI_DRM_5219: e8e52c8a114aa27eba27c6d962de37b01af3758d @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_2106: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2106/
IGT_4735: b05c028ccdb6ac8e8d8499a041bb14dfe358ee26 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_2106/
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results
2018-11-28 22:52 [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results Chris Wilson
2018-11-28 22:52 ` [igt-dev] [PATCH i-g-t 2/2] lib: Only require we have i915.reset module parameter for allow-hang Chris Wilson
2018-11-29 1:25 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/2] lib: Defer application of igt_fixture results Patchwork
@ 2018-11-29 17:16 ` Daniel Vetter
2018-11-29 17:17 ` Daniel Vetter
2 siblings, 1 reply; 6+ messages in thread
From: Daniel Vetter @ 2018-11-29 17:16 UTC (permalink / raw)
To: Chris Wilson; +Cc: igt-dev
On Wed, Nov 28, 2018 at 10:52:30PM +0000, Chris Wilson wrote:
> If an igt_fixture causes an independent subgroup to fail/skip and that
> subgroup is not part of the execution set (--run-subtest) as no subtests
> themselves failed or skipped, it should not count towards the overall
> test exitcode.
>
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108891
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
I'm honestly a bit lost on what's going on here. Could you please write a
new library testcase for this one here, as demonstrator? For stuff like
this I really want to lock behaviour down with compile-time testcases.
-Daniel
> ---
> lib/igt_core.c | 57 ++++++++++++++++++++++++++++++--------------------
> 1 file changed, 34 insertions(+), 23 deletions(-)
>
> diff --git a/lib/igt_core.c b/lib/igt_core.c
> index 64883d640..1d47e171f 100644
> --- a/lib/igt_core.c
> +++ b/lib/igt_core.c
> @@ -291,6 +291,10 @@ enum {
> static int igt_exitcode = IGT_EXIT_SUCCESS;
> static const char *command_str;
>
> +static bool skipped_one = false;
> +static bool succeeded_one = false;
> +static bool failed_one = false;
> +
> static char* igt_log_domain_filter;
> static struct {
> char *entries[256];
> @@ -919,15 +923,28 @@ bool __igt_run_subtest(const char *subtest_name)
> }
>
> if (skip_subtests_henceforth) {
> + const char *result;
> +
> + if (skip_subtests_henceforth == FAIL) {
> + if (!failed_one) {
> + igt_exitcode = FAIL;
> + failed_one = true;
> + }
> + result = "FAIL";
> + } else {
> + skipped_one = true;
> + result = "SKIP";
> + }
> +
> printf("%sSubtest %s: %s%s\n",
> - (!__igt_plain_output) ? "\x1b[1m" : "", subtest_name,
> - skip_subtests_henceforth == SKIP ?
> - "SKIP" : "FAIL", (!__igt_plain_output) ? "\x1b[0m" : "");
> + !__igt_plain_output ? "\x1b[1m" : "",
> + subtest_name, result,
> + !__igt_plain_output ? "\x1b[0m" : "");
> fflush(stdout);
> if (stderr_needs_sentinel)
> - fprintf(stderr, "Subtest %s: %s\n", subtest_name,
> - skip_subtests_henceforth == SKIP ?
> - "SKIP" : "FAIL");
> + fprintf(stderr, "Subtest %s: %s\n",
> + subtest_name, result);
> +
> return false;
> }
>
> @@ -976,10 +993,6 @@ void __igt_subtest_group_restore(int save)
> skip_subtests_henceforth = save;
> }
>
> -static bool skipped_one = false;
> -static bool succeeded_one = false;
> -static bool failed_one = false;
> -
> static void exit_subtest(const char *) __attribute__((noreturn));
> static void exit_subtest(const char *result)
> {
> @@ -1018,7 +1031,6 @@ static void exit_subtest(const char *result)
> void igt_skip(const char *f, ...)
> {
> va_list args;
> - skipped_one = true;
>
> assert(!test_child);
>
> @@ -1029,6 +1041,7 @@ void igt_skip(const char *f, ...)
> }
>
> if (in_subtest) {
> + skipped_one = true;
> exit_subtest("SKIP");
> } else if (test_with_subtests) {
> skip_subtests_henceforth = SKIP;
> @@ -1116,11 +1129,6 @@ void igt_fail(int exitcode)
> if (in_atexit_handler)
> _exit(IGT_EXIT_FAILURE);
>
> - if (!failed_one)
> - igt_exitcode = exitcode;
> -
> - failed_one = true;
> -
> /* Silent exit, parent will do the yelling. */
> if (test_child)
> exit(exitcode);
> @@ -1128,18 +1136,21 @@ void igt_fail(int exitcode)
> _igt_log_buffer_dump();
>
> if (in_subtest) {
> + if (!failed_one) {
> + igt_exitcode = exitcode;
> + failed_one = true;
> + }
> +
> if (exitcode == IGT_EXIT_TIMEOUT)
> exit_subtest("TIMEOUT");
> else
> exit_subtest("FAIL");
> + } else if (test_with_subtests) {
> + skip_subtests_henceforth = FAIL;
> + assert(in_fixture);
> + __igt_fixture_end();
> } else {
> - assert(igt_can_fail());
> -
> - if (in_fixture) {
> - skip_subtests_henceforth = FAIL;
> - __igt_fixture_end();
> - }
> -
> + igt_exitcode = exitcode;
> igt_exit();
> }
> }
> --
> 2.20.0.rc1
>
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results
2018-11-29 17:16 ` [igt-dev] [PATCH i-g-t 1/2] " Daniel Vetter
@ 2018-11-29 17:17 ` Daniel Vetter
2019-01-28 16:05 ` Katarzyna Dec
0 siblings, 1 reply; 6+ messages in thread
From: Daniel Vetter @ 2018-11-29 17:17 UTC (permalink / raw)
To: Chris Wilson; +Cc: igt-dev
On Thu, Nov 29, 2018 at 06:16:48PM +0100, Daniel Vetter wrote:
> On Wed, Nov 28, 2018 at 10:52:30PM +0000, Chris Wilson wrote:
> > If an igt_fixture causes an independent subgroup to fail/skip and that
> > subgroup is not part of the execution set (--run-subtest) as no subtests
> > themselves failed or skipped, it should not count towards the overall
> > test exitcode.
> >
> > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108891
> > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
>
> I'm honestly a bit lost on what's going on here. Could you please write a
> new library testcase for this one here, as demonstrator? For stuff like
> this I really want to lock behaviour down with compile-time testcases.
Forgot to add: They're in lib/tests/
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results
2018-11-29 17:17 ` Daniel Vetter
@ 2019-01-28 16:05 ` Katarzyna Dec
0 siblings, 0 replies; 6+ messages in thread
From: Katarzyna Dec @ 2019-01-28 16:05 UTC (permalink / raw)
To: Daniel Vetter, Chris Wilson; +Cc: igt-dev
On Thu, Nov 29, 2018 at 06:17:41PM +0100, Daniel Vetter wrote:
> On Thu, Nov 29, 2018 at 06:16:48PM +0100, Daniel Vetter wrote:
> > On Wed, Nov 28, 2018 at 10:52:30PM +0000, Chris Wilson wrote:
> > > If an igt_fixture causes an independent subgroup to fail/skip and that
> > > subgroup is not part of the execution set (--run-subtest) as no subtests
> > > themselves failed or skipped, it should not count towards the overall
> > > test exitcode.
> > >
> > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=108891
> > > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> >
> > I'm honestly a bit lost on what's going on here. Could you please write a
> > new library testcase for this one here, as demonstrator? For stuff like
> > this I really want to lock behaviour down with compile-time testcases.
>
> Forgot to add: They're in lib/tests/
> -Daniel
Looks like this issue is observed when running single subtest, e.g.:
gem_busy --r busy-render
Code prepared by Chris doesn't help with all issues I am observing, but at least
I see the direction to follow.
I will debug the issue more and let you know.
Kasia :)
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-01-28 16:05 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-28 22:52 [igt-dev] [PATCH i-g-t 1/2] lib: Defer application of igt_fixture results Chris Wilson
2018-11-28 22:52 ` [igt-dev] [PATCH i-g-t 2/2] lib: Only require we have i915.reset module parameter for allow-hang Chris Wilson
2018-11-29 1:25 ` [igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,1/2] lib: Defer application of igt_fixture results Patchwork
2018-11-29 17:16 ` [igt-dev] [PATCH i-g-t 1/2] " Daniel Vetter
2018-11-29 17:17 ` Daniel Vetter
2019-01-28 16:05 ` Katarzyna Dec
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.