* [igt-dev] [PATCH i-g-t] benchmarks/gem_blt: fix baseline estimation
@ 2022-05-19 7:41 Mauro Carvalho Chehab
2022-05-19 8:36 ` [igt-dev] ✗ Fi.CI.BAT: failure for " Patchwork
2022-05-19 12:53 ` [igt-dev] [PATCH i-g-t] " Kamil Konieczny
0 siblings, 2 replies; 3+ messages in thread
From: Mauro Carvalho Chehab @ 2022-05-19 7:41 UTC (permalink / raw)
To: igt-dev, Petri Latvala; +Cc: Chris Wilson
From: Mauro Carvalho Chehab <mchehab@kernel.org>
This test is expected to run under a certain time defined by a command
line parameter (by default, 2s).
In order to achive the specified time, the baseline() function tries to
estimate the value of a counter that will get the minimal amount of
interaction to wait for > 0.1s.
However, currently, the baseline estimation is broken, returning a too high
number, as it is measuring the memcpy() time, instead of actually measuring
each loop's interaction.
Due to that, a default test without passing any command line parameter
would take more than 1.5 years to output the first benchmark data!
Fix it by using the same logic that it is inside run() at the baseline
time estimation.
On ADL-P, before this patch, baseline would return 399242693. After
it, it now return 20 - meaning an interval of about 120 ms, which
should be good enough to ensure that the tests won't take too long.
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
---
benchmarks/gem_blt.c | 18 +++++-------------
1 file changed, 5 insertions(+), 13 deletions(-)
diff --git a/benchmarks/gem_blt.c b/benchmarks/gem_blt.c
index 424ce8e75913..bd8264b4e896 100644
--- a/benchmarks/gem_blt.c
+++ b/benchmarks/gem_blt.c
@@ -57,29 +57,21 @@ elapsed(const struct timespec *start, const struct timespec *end)
return (end->tv_sec - start->tv_sec) + 1e-9*(end->tv_nsec - start->tv_nsec);
}
-static int baseline(uint64_t bytes, int milliseconds)
+static int baseline(int fd, struct drm_i915_gem_execbuffer2 execbuf, int milliseconds)
{
struct timespec start, end;
- const int size = 64*1024*1024;
int count = 0;
- void *mem;
-
- mem = malloc(size);
- if (mem == NULL)
- return 1;
clock_gettime(CLOCK_MONOTONIC, &start);
do {
- memset(mem, count, size);
+ gem_execbuf(fd, &execbuf);
count++;
clock_gettime(CLOCK_MONOTONIC, &end);
- if (elapsed(&start, &end) > 0.1)
+ if (elapsed(&start, &end) > (milliseconds / 1000.))
break;
} while (1);
- free(mem);
-
- return ceil(1e-3*milliseconds/elapsed(&start, &end) * count * size / bytes);
+ return count;
}
static int gem_linear_blt(int fd,
@@ -260,7 +252,7 @@ static int run(int object, int batch, int time, int reps, int ncpus, unsigned fl
execbuf.flags |= I915_EXEC_NO_RELOC;
/* Guess how many loops we need for 0.1s */
- count = baseline((uint64_t)object * batch, 100) / ncpus;
+ count = baseline(fd, execbuf, 100) / ncpus;
if (flags & SYNC) {
time *= count / 2;
count = 1;
--
2.36.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [igt-dev] ✗ Fi.CI.BAT: failure for benchmarks/gem_blt: fix baseline estimation
2022-05-19 7:41 [igt-dev] [PATCH i-g-t] benchmarks/gem_blt: fix baseline estimation Mauro Carvalho Chehab
@ 2022-05-19 8:36 ` Patchwork
2022-05-19 12:53 ` [igt-dev] [PATCH i-g-t] " Kamil Konieczny
1 sibling, 0 replies; 3+ messages in thread
From: Patchwork @ 2022-05-19 8:36 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: igt-dev
[-- Attachment #1: Type: text/plain, Size: 11486 bytes --]
== Series Details ==
Series: benchmarks/gem_blt: fix baseline estimation
URL : https://patchwork.freedesktop.org/series/104163/
State : failure
== Summary ==
CI Bug Log - changes from IGT_6481 -> IGTPW_7141
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_7141 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in IGTPW_7141, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
Participating hosts (45 -> 45)
------------------------------
Additional (2): fi-icl-u2 fi-pnv-d510
Missing (2): bat-dg2-8 bat-dg2-9
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_7141:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@workarounds:
- fi-bdw-5557u: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-bdw-5557u/igt@i915_selftest@live@workarounds.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bdw-5557u/igt@i915_selftest@live@workarounds.html
* igt@i915_suspend@basic-s3-without-i915:
- fi-cfl-8109u: [PASS][3] -> [INCOMPLETE][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@i915_suspend@basic-s3-without-i915.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@i915_suspend@basic-s3-without-i915.html
Known issues
------------
Here are the changes found in IGTPW_7141 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_huc_copy@huc-copy:
- fi-icl-u2: NOTRUN -> [SKIP][5] ([i915#2190])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@gem_huc_copy@huc-copy.html
* igt@gem_lmem_swapping@parallel-random-engines:
- fi-icl-u2: NOTRUN -> [SKIP][6] ([i915#4613]) +3 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@gem_lmem_swapping@parallel-random-engines.html
* igt@i915_pm_rpm@basic-rte:
- fi-cfl-8109u: [PASS][7] -> [DMESG-WARN][8] ([i915#1888] / [i915#62])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@i915_pm_rpm@basic-rte.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@i915_pm_rpm@basic-rte.html
* igt@i915_pm_rpm@module-reload:
- fi-cfl-8109u: [PASS][9] -> [DMESG-FAIL][10] ([i915#62]) +1 similar issue
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@i915_pm_rpm@module-reload.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@i915_pm_rpm@module-reload.html
* igt@i915_pm_rps@basic-api:
- fi-glk-j4005: [PASS][11] -> [DMESG-WARN][12] ([i915#118])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-glk-j4005/igt@i915_pm_rps@basic-api.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-glk-j4005/igt@i915_pm_rps@basic-api.html
* igt@i915_selftest@live@hangcheck:
- fi-hsw-g3258: [PASS][13] -> [INCOMPLETE][14] ([i915#4785])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-hsw-g3258/igt@i915_selftest@live@hangcheck.html
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-hsw-g3258/igt@i915_selftest@live@hangcheck.html
* igt@i915_selftest@live@requests:
- fi-pnv-d510: NOTRUN -> [DMESG-FAIL][15] ([i915#4528])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@i915_selftest@live@requests.html
* igt@i915_suspend@basic-s2idle-without-i915:
- bat-dg1-5: NOTRUN -> [INCOMPLETE][16] ([i915#6011])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-dg1-5/igt@i915_suspend@basic-s2idle-without-i915.html
* igt@i915_suspend@basic-s3-without-i915:
- fi-icl-u2: NOTRUN -> [SKIP][17] ([i915#5903])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@i915_suspend@basic-s3-without-i915.html
* igt@kms_busy@basic@flip:
- fi-tgl-u2: [PASS][18] -> [DMESG-WARN][19] ([i915#402]) +2 similar issues
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-tgl-u2/igt@kms_busy@basic@flip.html
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-tgl-u2/igt@kms_busy@basic@flip.html
* igt@kms_chamelium@common-hpd-after-suspend:
- fi-bsw-nick: NOTRUN -> [SKIP][20] ([fdo#109271] / [fdo#111827])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@kms_chamelium@common-hpd-after-suspend.html
* igt@kms_chamelium@hdmi-hpd-fast:
- fi-icl-u2: NOTRUN -> [SKIP][21] ([fdo#111827]) +8 similar issues
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@kms_chamelium@hdmi-hpd-fast.html
* igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy:
- fi-icl-u2: NOTRUN -> [SKIP][22] ([fdo#109278]) +2 similar issues
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-legacy.html
* igt@kms_flip@basic-flip-vs-modeset@b-edp1:
- bat-adlp-4: [PASS][23] -> [DMESG-WARN][24] ([i915#3576])
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-adlp-4/igt@kms_flip@basic-flip-vs-modeset@b-edp1.html
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-adlp-4/igt@kms_flip@basic-flip-vs-modeset@b-edp1.html
* igt@kms_force_connector_basic@force-load-detect:
- fi-icl-u2: NOTRUN -> [SKIP][25] ([fdo#109285])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@kms_force_connector_basic@force-load-detect.html
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b:
- fi-cfl-8109u: [PASS][26] -> [DMESG-WARN][27] ([i915#62]) +12 similar issues
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-cfl-8109u/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b.html
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b.html
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-a:
- fi-bsw-nick: NOTRUN -> [SKIP][28] ([fdo#109271])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-a.html
* igt@kms_setmode@basic-clone-single-crtc:
- fi-icl-u2: NOTRUN -> [SKIP][29] ([i915#3555])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@kms_setmode@basic-clone-single-crtc.html
* igt@prime_vgem@basic-userptr:
- fi-pnv-d510: NOTRUN -> [SKIP][30] ([fdo#109271]) +39 similar issues
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@prime_vgem@basic-userptr.html
- fi-icl-u2: NOTRUN -> [SKIP][31] ([i915#3301])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-icl-u2/igt@prime_vgem@basic-userptr.html
* igt@runner@aborted:
- fi-pnv-d510: NOTRUN -> [FAIL][32] ([fdo#109271] / [i915#2403] / [i915#4312])
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-pnv-d510/igt@runner@aborted.html
- fi-cfl-8109u: NOTRUN -> [FAIL][33] ([i915#4312])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-cfl-8109u/igt@runner@aborted.html
- fi-hsw-g3258: NOTRUN -> [FAIL][34] ([fdo#109271] / [i915#4312])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-hsw-g3258/igt@runner@aborted.html
#### Possible fixes ####
* igt@i915_selftest@live@execlists:
- fi-bsw-nick: [INCOMPLETE][35] ([i915#5801]) -> [PASS][36]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-bsw-nick/igt@i915_selftest@live@execlists.html
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-bsw-nick/igt@i915_selftest@live@execlists.html
* igt@i915_selftest@live@gt_engines:
- bat-dg1-5: [INCOMPLETE][37] ([i915#4418]) -> [PASS][38]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-dg1-5/igt@i915_selftest@live@gt_engines.html
* igt@kms_flip@basic-flip-vs-modeset@a-edp1:
- {bat-adlp-6}: [DMESG-WARN][39] ([i915#3576]) -> [PASS][40] +1 similar issue
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/bat-adlp-6/igt@kms_flip@basic-flip-vs-modeset@a-edp1.html
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/bat-adlp-6/igt@kms_flip@basic-flip-vs-modeset@a-edp1.html
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
- fi-tgl-u2: [DMESG-WARN][41] ([i915#402]) -> [PASS][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_6481/fi-tgl-u2/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/fi-tgl-u2/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
[fdo#109285]: https://bugs.freedesktop.org/show_bug.cgi?id=109285
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#118]: https://gitlab.freedesktop.org/drm/intel/issues/118
[i915#1888]: https://gitlab.freedesktop.org/drm/intel/issues/1888
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
[i915#2403]: https://gitlab.freedesktop.org/drm/intel/issues/2403
[i915#2411]: https://gitlab.freedesktop.org/drm/intel/issues/2411
[i915#3301]: https://gitlab.freedesktop.org/drm/intel/issues/3301
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#3576]: https://gitlab.freedesktop.org/drm/intel/issues/3576
[i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402
[i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
[i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
[i915#4418]: https://gitlab.freedesktop.org/drm/intel/issues/4418
[i915#4528]: https://gitlab.freedesktop.org/drm/intel/issues/4528
[i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
[i915#4785]: https://gitlab.freedesktop.org/drm/intel/issues/4785
[i915#5801]: https://gitlab.freedesktop.org/drm/intel/issues/5801
[i915#5903]: https://gitlab.freedesktop.org/drm/intel/issues/5903
[i915#5950]: https://gitlab.freedesktop.org/drm/intel/issues/5950
[i915#6011]: https://gitlab.freedesktop.org/drm/intel/issues/6011
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_6481 -> IGTPW_7141
CI-20190529: 20190529
CI_DRM_11675: c31582c636f4c6ac8c0f822c39b250db2c5cd437 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_7141: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
IGT_6481: f2a9c2e6f6f7aa97e5d92274f20aa698087359c4 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_7141/index.html
[-- Attachment #2: Type: text/html, Size: 13281 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [igt-dev] [PATCH i-g-t] benchmarks/gem_blt: fix baseline estimation
2022-05-19 7:41 [igt-dev] [PATCH i-g-t] benchmarks/gem_blt: fix baseline estimation Mauro Carvalho Chehab
2022-05-19 8:36 ` [igt-dev] ✗ Fi.CI.BAT: failure for " Patchwork
@ 2022-05-19 12:53 ` Kamil Konieczny
1 sibling, 0 replies; 3+ messages in thread
From: Kamil Konieczny @ 2022-05-19 12:53 UTC (permalink / raw)
To: igt-dev
Hi Mauro,
small nitpicks on commit message.
On 2022-05-19 at 09:41:10 +0200, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <mchehab@kernel.org>
>
> This test is expected to run under a certain time defined by a command
> line parameter (by default, 2s).
>
> In order to achive the specified time, the baseline() function tries to
> estimate the value of a counter that will get the minimal amount of
> interaction to wait for > 0.1s.
>
> However, currently, the baseline estimation is broken, returning a too high
------------------------------------------------------------------------ ^
s/high/big/
> number, as it is measuring the memcpy() time, instead of actually measuring
> each loop's interaction.
/each loop's interaction./gem_execbuf()./
>
> Due to that, a default test without passing any command line parameter
> would take more than 1.5 years to output the first benchmark data!
>
> Fix it by using the same logic that it is inside run() at the baseline
> time estimation.
>
> On ADL-P, before this patch, baseline would return 399242693. After
s/On ADL-P, before/Before/
s/would/could/
> it, it now return 20 - meaning an interval of about 120 ms, which
Well, both numbers means the same interval _if_ used for the
same syscall.
With that fixed you can add my r-b tag.
--
Kamil
> should be good enough to ensure that the tests won't take too long.
>
> Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
> ---
> benchmarks/gem_blt.c | 18 +++++-------------
> 1 file changed, 5 insertions(+), 13 deletions(-)
>
> diff --git a/benchmarks/gem_blt.c b/benchmarks/gem_blt.c
> index 424ce8e75913..bd8264b4e896 100644
> --- a/benchmarks/gem_blt.c
> +++ b/benchmarks/gem_blt.c
> @@ -57,29 +57,21 @@ elapsed(const struct timespec *start, const struct timespec *end)
> return (end->tv_sec - start->tv_sec) + 1e-9*(end->tv_nsec - start->tv_nsec);
> }
>
> -static int baseline(uint64_t bytes, int milliseconds)
> +static int baseline(int fd, struct drm_i915_gem_execbuffer2 execbuf, int milliseconds)
> {
> struct timespec start, end;
> - const int size = 64*1024*1024;
> int count = 0;
> - void *mem;
> -
> - mem = malloc(size);
> - if (mem == NULL)
> - return 1;
>
> clock_gettime(CLOCK_MONOTONIC, &start);
> do {
> - memset(mem, count, size);
> + gem_execbuf(fd, &execbuf);
> count++;
> clock_gettime(CLOCK_MONOTONIC, &end);
> - if (elapsed(&start, &end) > 0.1)
> + if (elapsed(&start, &end) > (milliseconds / 1000.))
> break;
> } while (1);
>
> - free(mem);
> -
> - return ceil(1e-3*milliseconds/elapsed(&start, &end) * count * size / bytes);
> + return count;
> }
>
> static int gem_linear_blt(int fd,
> @@ -260,7 +252,7 @@ static int run(int object, int batch, int time, int reps, int ncpus, unsigned fl
> execbuf.flags |= I915_EXEC_NO_RELOC;
>
> /* Guess how many loops we need for 0.1s */
> - count = baseline((uint64_t)object * batch, 100) / ncpus;
> + count = baseline(fd, execbuf, 100) / ncpus;
> if (flags & SYNC) {
> time *= count / 2;
> count = 1;
> --
> 2.36.1
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-05-19 12:53 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-19 7:41 [igt-dev] [PATCH i-g-t] benchmarks/gem_blt: fix baseline estimation Mauro Carvalho Chehab
2022-05-19 8:36 ` [igt-dev] ✗ Fi.CI.BAT: failure for " Patchwork
2022-05-19 12:53 ` [igt-dev] [PATCH i-g-t] " Kamil Konieczny
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.