[PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

We need to use absolute tolerance when asserting on percentages. Relative
tolerance in this case is unfair and inaccurate since it's strictness
varies with relative target busyness.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
---
 tests/perf_pmu.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
index 9ebffc64d1f1..8e547338b47c 100644
--- a/tests/perf_pmu.c
+++ b/tests/perf_pmu.c
@@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
        __sync_synchronize();
 }
 
-#define div_round_up(a, b) (((a) + (b) - 1) / (b))
+#define __assert_within(x, ref, tol_up, tol_down) \
+	igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
+		     (double)(x) >= ((double)(ref) - (tol_down)), \
+		     "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
+		     (double)(x), (double)(tol_up), (double)(tol_down), \
+		     (double)(ref), #x, #ref)
+
+#define assert_within(x, ref, tolerance) \
+	__assert_within(x, ref, tolerance, tolerance)
 
 static void
 accuracy(int gem_fd, const struct intel_execution_engine2 *e,
@@ -1571,7 +1579,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 
 	/* Let the child run. */
 	read(link[0], &expected, sizeof(expected));
-	assert_within_epsilon(expected, target_busy_pct/100., 0.05);
+	assert_within(100.0 * expected, target_busy_pct, 5);
 
 	/* Collect engine busyness for an interesting part of child runtime. */
 	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
@@ -1590,8 +1598,11 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
 		 __error(busy_r, expected), 100 * busy_r, 100 * expected);
 
-	assert_within_epsilon(busy_r, expected, 0.15);
-	assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
+	busy_r *= 100.0;
+	expected *= 100.0;
+
+	assert_within(busy_r, expected, 2);
+	assert_within(100.0 - busy_r, 100.0 - expected, 2);
 }
 
 igt_main
-- 
2.14.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[igt-dev] [PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

We need to use absolute tolerance when asserting on percentages. Relative
tolerance in this case is unfair and inaccurate since it's strictness
varies with relative target busyness.

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
---
 tests/perf_pmu.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
index 9ebffc64d1f1..8e547338b47c 100644
--- a/tests/perf_pmu.c
+++ b/tests/perf_pmu.c
@@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
        __sync_synchronize();
 }
 
-#define div_round_up(a, b) (((a) + (b) - 1) / (b))
+#define __assert_within(x, ref, tol_up, tol_down) \
+	igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
+		     (double)(x) >= ((double)(ref) - (tol_down)), \
+		     "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
+		     (double)(x), (double)(tol_up), (double)(tol_down), \
+		     (double)(ref), #x, #ref)
+
+#define assert_within(x, ref, tolerance) \
+	__assert_within(x, ref, tolerance, tolerance)
 
 static void
 accuracy(int gem_fd, const struct intel_execution_engine2 *e,
@@ -1571,7 +1579,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 
 	/* Let the child run. */
 	read(link[0], &expected, sizeof(expected));
-	assert_within_epsilon(expected, target_busy_pct/100., 0.05);
+	assert_within(100.0 * expected, target_busy_pct, 5);
 
 	/* Collect engine busyness for an interesting part of child runtime. */
 	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
@@ -1590,8 +1598,11 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
 		 __error(busy_r, expected), 100 * busy_r, 100 * expected);
 
-	assert_within_epsilon(busy_r, expected, 0.15);
-	assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
+	busy_r *= 100.0;
+	expected *= 100.0;
+
+	assert_within(busy_r, expected, 2);
+	assert_within(100.0 - busy_r, 100.0 - expected, 2);
 }
 
 igt_main
-- 
2.14.1

_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-07 11:11:19)
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> We need to use absolute tolerance when asserting on percentages. Relative
> tolerance in this case is unfair and inaccurate since it's strictness
> varies with relative target busyness.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> ---
>  tests/perf_pmu.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
> index 9ebffc64d1f1..8e547338b47c 100644
> --- a/tests/perf_pmu.c
> +++ b/tests/perf_pmu.c
> @@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
>         __sync_synchronize();
>  }
>  
> -#define div_round_up(a, b) (((a) + (b) - 1) / (b))
> +#define __assert_within(x, ref, tol_up, tol_down) \
> +       igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
> +                    (double)(x) >= ((double)(ref) - (tol_down)), \
> +                    "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
> +                    (double)(x), (double)(tol_up), (double)(tol_down), \
> +                    (double)(ref), #x, #ref)
> +
> +#define assert_within(x, ref, tolerance) \
> +       __assert_within(x, ref, tolerance, tolerance)
>  
>  static void
>  accuracy(int gem_fd, const struct intel_execution_engine2 *e,
> @@ -1571,7 +1579,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>  
>         /* Let the child run. */
>         read(link[0], &expected, sizeof(expected));
> -       assert_within_epsilon(expected, target_busy_pct/100., 0.05);
> +       assert_within(100.0 * expected, target_busy_pct, 5);
>  
>         /* Collect engine busyness for an interesting part of child runtime. */
>         fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
> @@ -1590,8 +1598,11 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>         igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
>                  __error(busy_r, expected), 100 * busy_r, 100 * expected);
>  
> -       assert_within_epsilon(busy_r, expected, 0.15);
> -       assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
> +       busy_r *= 100.0;
> +       expected *= 100.0;
> +
> +       assert_within(busy_r, expected, 2);
> +       assert_within(100.0 - busy_r, 100.0 - expected, 2);

The advantage of switching to absolute here is that we only need the
single test. Ok, using a factor of 100 here should make the output more
readable.

Kill the extra assert_within,
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

But I suspect we may need to relax the target for kasan, we will see in
a few days.
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [igt-dev] [PATCH i-g-t] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-07 11:11:19)
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> We need to use absolute tolerance when asserting on percentages. Relative
> tolerance in this case is unfair and inaccurate since it's strictness
> varies with relative target busyness.
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> ---
>  tests/perf_pmu.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
> index 9ebffc64d1f1..8e547338b47c 100644
> --- a/tests/perf_pmu.c
> +++ b/tests/perf_pmu.c
> @@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
>         __sync_synchronize();
>  }
>  
> -#define div_round_up(a, b) (((a) + (b) - 1) / (b))
> +#define __assert_within(x, ref, tol_up, tol_down) \
> +       igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
> +                    (double)(x) >= ((double)(ref) - (tol_down)), \
> +                    "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
> +                    (double)(x), (double)(tol_up), (double)(tol_down), \
> +                    (double)(ref), #x, #ref)
> +
> +#define assert_within(x, ref, tolerance) \
> +       __assert_within(x, ref, tolerance, tolerance)
>  
>  static void
>  accuracy(int gem_fd, const struct intel_execution_engine2 *e,
> @@ -1571,7 +1579,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>  
>         /* Let the child run. */
>         read(link[0], &expected, sizeof(expected));
> -       assert_within_epsilon(expected, target_busy_pct/100., 0.05);
> +       assert_within(100.0 * expected, target_busy_pct, 5);
>  
>         /* Collect engine busyness for an interesting part of child runtime. */
>         fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
> @@ -1590,8 +1598,11 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>         igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
>                  __error(busy_r, expected), 100 * busy_r, 100 * expected);
>  
> -       assert_within_epsilon(busy_r, expected, 0.15);
> -       assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
> +       busy_r *= 100.0;
> +       expected *= 100.0;
> +
> +       assert_within(busy_r, expected, 2);
> +       assert_within(100.0 - busy_r, 100.0 - expected, 2);

The advantage of switching to absolute here is that we only need the
single test. Ok, using a factor of 100 here should make the output more
readable.

Kill the extra assert_within,
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

But I suspect we may need to relax the target for kasan, we will see in
a few days.
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[PATCH i-g-t v2] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

We need to use absolute tolerance when asserting on percentages. Relative
tolerance in this case is unfair and inaccurate since it's strictness
varies with relative target busyness.

v2:
 * Do not include spin batch edit and submit into measured time.
 * Open PMU before child is in test PWM phase.
 * No need to emit test PWM for twice as long with the new explicit
   synchroniazation via pipe.
 * Log test duration in ms for better readability.
 * Drop inverse assert. (Chris Wilson)

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> # v1
---
 tests/perf_pmu.c | 33 +++++++++++++++++++--------------
 1 file changed, 19 insertions(+), 14 deletions(-)

diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
index 9ebffc64d1f1..ff9f71540ee4 100644
--- a/tests/perf_pmu.c
+++ b/tests/perf_pmu.c
@@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
        __sync_synchronize();
 }
 
-#define div_round_up(a, b) (((a) + (b) - 1) / (b))
+#define __assert_within(x, ref, tol_up, tol_down) \
+	igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
+		     (double)(x) >= ((double)(ref) - (tol_down)), \
+		     "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
+		     (double)(x), (double)(tol_up), (double)(tol_down), \
+		     (double)(ref), #x, #ref)
+
+#define assert_within(x, ref, tolerance) \
+	__assert_within(x, ref, tolerance, tolerance)
 
 static void
 accuracy(int gem_fd, const struct intel_execution_engine2 *e,
@@ -1493,8 +1501,8 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	while (test_us < min_test_us)
 		test_us += busy_us + idle_us;
 
-	igt_info("calibration=%luus, test=%luus; ratio=%.2f%% (%luus/%luus)\n",
-		 pwm_calibration_us, test_us,
+	igt_info("calibration=%lums, test=%lums; ratio=%.2f%% (%luus/%luus)\n",
+		 pwm_calibration_us / 1000, test_us / 1000,
 		 (double)busy_us / (busy_us + idle_us) * 100.0,
 		 busy_us, idle_us);
 
@@ -1507,7 +1515,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_fork(child, 1) {
 		struct sched_param rt = { .sched_priority = 99 };
 		const unsigned long timeout[] = {
-			pwm_calibration_us * 1000, test_us * 2 * 1000
+			pwm_calibration_us * 1000, test_us * 1000
 		};
 		struct drm_i915_gem_exec_object2 obj = {};
 		uint64_t total_busy_ns = 0, total_idle_ns = 0;
@@ -1537,19 +1545,16 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 
 			igt_nsec_elapsed(&test_start);
 			do {
-				struct timespec t_busy = { };
-				unsigned int target_idle_us;
-
-				igt_nsec_elapsed(&t_busy);
+				unsigned int target_idle_us, t_busy;
 
 				/* Restart the spinbatch. */
 				__rearm_spin_batch(spin);
 				__submit_spin_batch(gem_fd, &obj, e);
-				measured_usleep(busy_us);
+				t_busy = measured_usleep(busy_us);
 				igt_spin_batch_end(spin);
 				gem_sync(gem_fd, obj.handle);
 
-				total_busy_ns += igt_nsec_elapsed(&t_busy);
+				total_busy_ns += t_busy;
 
 				target_idle_us =
 					(100 * total_busy_ns / target_busy_pct - (total_busy_ns + total_idle_ns)) / 1000;
@@ -1569,12 +1574,13 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 		igt_spin_batch_free(gem_fd, spin);
 	}
 
+	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
+
 	/* Let the child run. */
 	read(link[0], &expected, sizeof(expected));
-	assert_within_epsilon(expected, target_busy_pct/100., 0.05);
+	assert_within(100.0 * expected, target_busy_pct, 5);
 
 	/* Collect engine busyness for an interesting part of child runtime. */
-	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
 	val[0] = __pmu_read_single(fd, &ts[0]);
 	read(link[0], &expected, sizeof(expected));
 	val[1] = __pmu_read_single(fd, &ts[1]);
@@ -1590,8 +1596,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
 		 __error(busy_r, expected), 100 * busy_r, 100 * expected);
 
-	assert_within_epsilon(busy_r, expected, 0.15);
-	assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
+	assert_within(100.0 * busy_r, 100.0 * expected, 2);
 }
 
 igt_main
-- 
2.14.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[Intel-gfx] [PATCH i-g-t v2] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Tvrtko Ursulin]

From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

We need to use absolute tolerance when asserting on percentages. Relative
tolerance in this case is unfair and inaccurate since it's strictness
varies with relative target busyness.

v2:
 * Do not include spin batch edit and submit into measured time.
 * Open PMU before child is in test PWM phase.
 * No need to emit test PWM for twice as long with the new explicit
   synchroniazation via pipe.
 * Log test duration in ms for better readability.
 * Drop inverse assert. (Chris Wilson)

Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> # v1
---
 tests/perf_pmu.c | 33 +++++++++++++++++++--------------
 1 file changed, 19 insertions(+), 14 deletions(-)

diff --git a/tests/perf_pmu.c b/tests/perf_pmu.c
index 9ebffc64d1f1..ff9f71540ee4 100644
--- a/tests/perf_pmu.c
+++ b/tests/perf_pmu.c
@@ -1459,7 +1459,15 @@ static void __rearm_spin_batch(igt_spin_t *spin)
        __sync_synchronize();
 }
 
-#define div_round_up(a, b) (((a) + (b) - 1) / (b))
+#define __assert_within(x, ref, tol_up, tol_down) \
+	igt_assert_f((double)(x) <= ((double)(ref) + (tol_up)) && \
+		     (double)(x) >= ((double)(ref) - (tol_down)), \
+		     "%f not within +%f/-%f of %f! ('%s' vs '%s')\n", \
+		     (double)(x), (double)(tol_up), (double)(tol_down), \
+		     (double)(ref), #x, #ref)
+
+#define assert_within(x, ref, tolerance) \
+	__assert_within(x, ref, tolerance, tolerance)
 
 static void
 accuracy(int gem_fd, const struct intel_execution_engine2 *e,
@@ -1493,8 +1501,8 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	while (test_us < min_test_us)
 		test_us += busy_us + idle_us;
 
-	igt_info("calibration=%luus, test=%luus; ratio=%.2f%% (%luus/%luus)\n",
-		 pwm_calibration_us, test_us,
+	igt_info("calibration=%lums, test=%lums; ratio=%.2f%% (%luus/%luus)\n",
+		 pwm_calibration_us / 1000, test_us / 1000,
 		 (double)busy_us / (busy_us + idle_us) * 100.0,
 		 busy_us, idle_us);
 
@@ -1507,7 +1515,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_fork(child, 1) {
 		struct sched_param rt = { .sched_priority = 99 };
 		const unsigned long timeout[] = {
-			pwm_calibration_us * 1000, test_us * 2 * 1000
+			pwm_calibration_us * 1000, test_us * 1000
 		};
 		struct drm_i915_gem_exec_object2 obj = {};
 		uint64_t total_busy_ns = 0, total_idle_ns = 0;
@@ -1537,19 +1545,16 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 
 			igt_nsec_elapsed(&test_start);
 			do {
-				struct timespec t_busy = { };
-				unsigned int target_idle_us;
-
-				igt_nsec_elapsed(&t_busy);
+				unsigned int target_idle_us, t_busy;
 
 				/* Restart the spinbatch. */
 				__rearm_spin_batch(spin);
 				__submit_spin_batch(gem_fd, &obj, e);
-				measured_usleep(busy_us);
+				t_busy = measured_usleep(busy_us);
 				igt_spin_batch_end(spin);
 				gem_sync(gem_fd, obj.handle);
 
-				total_busy_ns += igt_nsec_elapsed(&t_busy);
+				total_busy_ns += t_busy;
 
 				target_idle_us =
 					(100 * total_busy_ns / target_busy_pct - (total_busy_ns + total_idle_ns)) / 1000;
@@ -1569,12 +1574,13 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 		igt_spin_batch_free(gem_fd, spin);
 	}
 
+	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
+
 	/* Let the child run. */
 	read(link[0], &expected, sizeof(expected));
-	assert_within_epsilon(expected, target_busy_pct/100., 0.05);
+	assert_within(100.0 * expected, target_busy_pct, 5);
 
 	/* Collect engine busyness for an interesting part of child runtime. */
-	fd = open_pmu(I915_PMU_ENGINE_BUSY(e->class, e->instance));
 	val[0] = __pmu_read_single(fd, &ts[0]);
 	read(link[0], &expected, sizeof(expected));
 	val[1] = __pmu_read_single(fd, &ts[1]);
@@ -1590,8 +1596,7 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
 	igt_info("error=%.2f%% (%.2f%% vs %.2f%%)\n",
 		 __error(busy_r, expected), 100 * busy_r, 100 * expected);
 
-	assert_within_epsilon(busy_r, expected, 0.15);
-	assert_within_epsilon(1 - busy_r, 1 - expected, 0.15);
+	assert_within(100.0 * busy_r, 100.0 * expected, 2);
 }
 
 igt_main
-- 
2.14.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH i-g-t v2] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-09 11:54:13)
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> We need to use absolute tolerance when asserting on percentages. Relative
> tolerance in this case is unfair and inaccurate since it's strictness
> varies with relative target busyness.
> 
> v2:
>  * Do not include spin batch edit and submit into measured time.
>  * Open PMU before child is in test PWM phase.
>  * No need to emit test PWM for twice as long with the new explicit
>    synchroniazation via pipe.
>  * Log test duration in ms for better readability.
>  * Drop inverse assert. (Chris Wilson)
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> # v1
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

Would be nice to add a comment now we have a reasonable suspicion:

> @@ -1537,19 +1545,16 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>  
>                         igt_nsec_elapsed(&test_start);
>                         do {
> -                               struct timespec t_busy = { };
> -                               unsigned int target_idle_us;
> -
> -                               igt_nsec_elapsed(&t_busy);
> +                               unsigned int target_idle_us, t_busy;
>  
>                                 /* Restart the spinbatch. */
>                                 __rearm_spin_batch(spin);
>                                 __submit_spin_batch(gem_fd, &obj, e);

/*
 * Note that the submission may be delayed to a tasklet (ksoftirqd)
 * which cannot run until we sleep as we hog the cpu (we are RT).
 */

> -                               measured_usleep(busy_us);
> +                               t_busy = measured_usleep(busy_us);
>                                 igt_spin_batch_end(spin);
>                                 gem_sync(gem_fd, obj.handle);

And back to thinking how we can kick the tasklet, or kick the habit.
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [igt-dev] [PATCH i-g-t v2] tests/perf_pmu: Use absolute tolerance in accuracy tests

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-09 11:54:13)
> From: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> 
> We need to use absolute tolerance when asserting on percentages. Relative
> tolerance in this case is unfair and inaccurate since it's strictness
> varies with relative target busyness.
> 
> v2:
>  * Do not include spin batch edit and submit into measured time.
>  * Open PMU before child is in test PWM phase.
>  * No need to emit test PWM for twice as long with the new explicit
>    synchroniazation via pipe.
>  * Log test duration in ms for better readability.
>  * Drop inverse assert. (Chris Wilson)
> 
> Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: Chris Wilson <chris@chris-wilson.co.uk>
> Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> # v1
Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>

Would be nice to add a comment now we have a reasonable suspicion:

> @@ -1537,19 +1545,16 @@ accuracy(int gem_fd, const struct intel_execution_engine2 *e,
>  
>                         igt_nsec_elapsed(&test_start);
>                         do {
> -                               struct timespec t_busy = { };
> -                               unsigned int target_idle_us;
> -
> -                               igt_nsec_elapsed(&t_busy);
> +                               unsigned int target_idle_us, t_busy;
>  
>                                 /* Restart the spinbatch. */
>                                 __rearm_spin_batch(spin);
>                                 __submit_spin_batch(gem_fd, &obj, e);

/*
 * Note that the submission may be delayed to a tasklet (ksoftirqd)
 * which cannot run until we sleep as we hog the cpu (we are RT).
 */

> -                               measured_usleep(busy_us);
> +                               t_busy = measured_usleep(busy_us);
>                                 igt_spin_batch_end(spin);
>                                 gem_sync(gem_fd, obj.handle);

And back to thinking how we can kick the tasklet, or kick the habit.
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for tests/perf_pmu: Use absolute tolerance in accuracy tests

[Patchwork]

== Series Details ==

Series: tests/perf_pmu: Use absolute tolerance in accuracy tests
URL   : https://patchwork.freedesktop.org/series/39518/
State : success

== Summary ==

IGT patchset tested on top of latest successful build
b4689dce36d0fbd9aec70d5a4b077c43a6b9c254 igt: Remove gen7_forcewake_mt

with latest DRM-Tip kernel build CI_DRM_3885
8a4eb4556f66 drm-tip: 2018y-03m-06d-22h-59m-29s UTC integration manifest

No testlist changes.

---- Known issues:

Test debugfs_test:
        Subgroup read_all_entries:
                pass       -> INCOMPLETE (fi-snb-2520m) fdo#103713
Test prime_vgem:
        Subgroup basic-fence-flip:
                fail       -> PASS       (fi-ilk-650) fdo#104008

fdo#103713 https://bugs.freedesktop.org/show_bug.cgi?id=103713
fdo#104008 https://bugs.freedesktop.org/show_bug.cgi?id=104008

fi-bdw-5557u     total:288  pass:267  dwarn:0   dfail:0   fail:0   skip:21  time:423s
fi-bdw-gvtdvm    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:426s
fi-blb-e6850     total:288  pass:223  dwarn:1   dfail:0   fail:0   skip:64  time:374s
fi-bsw-n3050     total:288  pass:242  dwarn:0   dfail:0   fail:0   skip:46  time:507s
fi-bwr-2160      total:288  pass:183  dwarn:0   dfail:0   fail:0   skip:105 time:280s
fi-bxt-j4205     total:288  pass:259  dwarn:0   dfail:0   fail:0   skip:29  time:496s
fi-byt-j1900     total:288  pass:253  dwarn:0   dfail:0   fail:0   skip:35  time:485s
fi-byt-n2820     total:288  pass:249  dwarn:0   dfail:0   fail:0   skip:39  time:477s
fi-cfl-8700k     total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:404s
fi-cfl-s2        total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:580s
fi-elk-e7500     total:288  pass:229  dwarn:0   dfail:0   fail:0   skip:59  time:420s
fi-gdg-551       total:288  pass:179  dwarn:0   dfail:0   fail:1   skip:108 time:290s
fi-glk-1         total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:520s
fi-hsw-4770      total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:398s
fi-ilk-650       total:288  pass:228  dwarn:0   dfail:0   fail:0   skip:60  time:415s
fi-ivb-3520m     total:288  pass:259  dwarn:0   dfail:0   fail:0   skip:29  time:458s
fi-ivb-3770      total:288  pass:255  dwarn:0   dfail:0   fail:0   skip:33  time:424s
fi-kbl-7500u     total:288  pass:263  dwarn:1   dfail:0   fail:0   skip:24  time:471s
fi-kbl-7560u     total:108  pass:104  dwarn:0   dfail:0   fail:0   skip:3  
fi-kbl-7567u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:459s
fi-kbl-r         total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:510s
fi-pnv-d510      total:288  pass:222  dwarn:1   dfail:0   fail:0   skip:65  time:588s
fi-skl-6260u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:434s
fi-skl-6600u     total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:527s
fi-skl-6700hq    total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:537s
fi-skl-6700k2    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:498s
fi-skl-6770hq    total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:481s
fi-skl-guc       total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:422s
fi-skl-gvtdvm    total:288  pass:265  dwarn:0   dfail:0   fail:0   skip:23  time:434s
fi-snb-2520m     total:3    pass:2    dwarn:0   dfail:0   fail:0   skip:0  
fi-snb-2600      total:288  pass:248  dwarn:0   dfail:0   fail:0   skip:40  time:399s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1074/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✗ Fi.CI.IGT: warning for tests/perf_pmu: Use absolute tolerance in accuracy tests

[Patchwork]

== Series Details ==

Series: tests/perf_pmu: Use absolute tolerance in accuracy tests
URL   : https://patchwork.freedesktop.org/series/39518/
State : warning

== Summary ==

---- Possible new issues:

Test gem_linear_blits:
        Subgroup normal:
                pass       -> SKIP       (shard-apl)

---- Known issues:

Test gem_softpin:
        Subgroup noreloc-s3:
                skip       -> PASS       (shard-snb) fdo#103375
Test kms_chv_cursor_fail:
        Subgroup pipe-b-128x128-right-edge:
                pass       -> DMESG-WARN (shard-snb) fdo#105185 +4
Test kms_flip:
        Subgroup 2x-flip-vs-blocking-wf-vblank:
                pass       -> FAIL       (shard-hsw) fdo#100368 +2
Test kms_rotation_crc:
        Subgroup primary-rotation-180:
                fail       -> PASS       (shard-snb) fdo#103925
Test perf:
        Subgroup blocking:
                fail       -> PASS       (shard-hsw) fdo#102252 +1
Test pm_lpsp:
        Subgroup screens-disabled:
                fail       -> PASS       (shard-hsw) fdo#104941

fdo#103375 https://bugs.freedesktop.org/show_bug.cgi?id=103375
fdo#105185 https://bugs.freedesktop.org/show_bug.cgi?id=105185
fdo#100368 https://bugs.freedesktop.org/show_bug.cgi?id=100368
fdo#103925 https://bugs.freedesktop.org/show_bug.cgi?id=103925
fdo#102252 https://bugs.freedesktop.org/show_bug.cgi?id=102252
fdo#104941 https://bugs.freedesktop.org/show_bug.cgi?id=104941

shard-apl        total:3467 pass:1822 dwarn:1   dfail:0   fail:10  skip:1633 time:12344s
shard-hsw        total:3467 pass:1772 dwarn:1   dfail:0   fail:2   skip:1691 time:11981s
shard-snb        total:3467 pass:1363 dwarn:3   dfail:0   fail:1   skip:2100 time:7138s
Blacklisted hosts:
shard-kbl        total:3365 pass:1880 dwarn:1   dfail:0   fail:9   skip:1472 time:8609s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1074/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] ✗ Fi.CI.IGT: warning for tests/perf_pmu: Use absolute tolerance in accuracy tests

[Tvrtko Ursulin]

On 07/03/2018 12:35, Patchwork wrote:
> == Series Details ==
> 
> Series: tests/perf_pmu: Use absolute tolerance in accuracy tests
> URL   : https://patchwork.freedesktop.org/series/39518/
> State : warning
> 
> == Summary ==
> 
> ---- Possible new issues:
> 
> Test gem_linear_blits:
>          Subgroup normal:
>                  pass       -> SKIP       (shard-apl)
> 
> ---- Known issues:
> 
> Test gem_softpin:
>          Subgroup noreloc-s3:
>                  skip       -> PASS       (shard-snb) fdo#103375
> Test kms_chv_cursor_fail:
>          Subgroup pipe-b-128x128-right-edge:
>                  pass       -> DMESG-WARN (shard-snb) fdo#105185 +4
> Test kms_flip:
>          Subgroup 2x-flip-vs-blocking-wf-vblank:
>                  pass       -> FAIL       (shard-hsw) fdo#100368 +2
> Test kms_rotation_crc:
>          Subgroup primary-rotation-180:
>                  fail       -> PASS       (shard-snb) fdo#103925
> Test perf:
>          Subgroup blocking:
>                  fail       -> PASS       (shard-hsw) fdo#102252 +1
> Test pm_lpsp:
>          Subgroup screens-disabled:
>                  fail       -> PASS       (shard-hsw) fdo#104941

It doesn't say here but 50% tests are failing on shards. By the look of 
it around 3-4% off the target. So strange.. I need to figure out why 
would 50% be so much worse than 2% and 98% loads.

Regards,

Tvrtko

> fdo#103375 https://bugs.freedesktop.org/show_bug.cgi?id=103375
> fdo#105185 https://bugs.freedesktop.org/show_bug.cgi?id=105185
> fdo#100368 https://bugs.freedesktop.org/show_bug.cgi?id=100368
> fdo#103925 https://bugs.freedesktop.org/show_bug.cgi?id=103925
> fdo#102252 https://bugs.freedesktop.org/show_bug.cgi?id=102252
> fdo#104941 https://bugs.freedesktop.org/show_bug.cgi?id=104941
> 
> shard-apl        total:3467 pass:1822 dwarn:1   dfail:0   fail:10  skip:1633 time:12344s
> shard-hsw        total:3467 pass:1772 dwarn:1   dfail:0   fail:2   skip:1691 time:11981s
> shard-snb        total:3467 pass:1363 dwarn:3   dfail:0   fail:1   skip:2100 time:7138s
> Blacklisted hosts:
> shard-kbl        total:3365 pass:1880 dwarn:1   dfail:0   fail:9   skip:1472 time:8609s
> 
> == Logs ==
> 
> For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1074/shards.html
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
> 
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✓ Fi.CI.BAT: success for tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)

[Patchwork]

== Series Details ==

Series: tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)
URL   : https://patchwork.freedesktop.org/series/39518/
State : success

== Summary ==

IGT patchset tested on top of latest successful build
b4689dce36d0fbd9aec70d5a4b077c43a6b9c254 igt: Remove gen7_forcewake_mt

with latest DRM-Tip kernel build CI_DRM_3904
074e834cb3cc drm-tip: 2018y-03m-09d-10h-30m-56s UTC integration manifest

No testlist changes.

---- Known issues:

Test kms_frontbuffer_tracking:
        Subgroup basic:
                fail       -> PASS       (fi-cnl-y3) fdo#103167
Test kms_pipe_crc_basic:
        Subgroup suspend-read-crc-pipe-a:
                dmesg-warn -> PASS       (fi-skl-6700k2) fdo#103191
Test prime_vgem:
        Subgroup basic-fence-flip:
                pass       -> FAIL       (fi-ilk-650) fdo#104008

fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
fdo#103191 https://bugs.freedesktop.org/show_bug.cgi?id=103191
fdo#104008 https://bugs.freedesktop.org/show_bug.cgi?id=104008

fi-bdw-5557u     total:288  pass:267  dwarn:0   dfail:0   fail:0   skip:21  time:425s
fi-bdw-gvtdvm    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:429s
fi-blb-e6850     total:288  pass:223  dwarn:1   dfail:0   fail:0   skip:64  time:373s
fi-bsw-n3050     total:288  pass:242  dwarn:0   dfail:0   fail:0   skip:46  time:504s
fi-bwr-2160      total:288  pass:183  dwarn:0   dfail:0   fail:0   skip:105 time:279s
fi-bxt-dsi       total:288  pass:258  dwarn:0   dfail:0   fail:0   skip:30  time:493s
fi-bxt-j4205     total:288  pass:259  dwarn:0   dfail:0   fail:0   skip:29  time:490s
fi-byt-j1900     total:288  pass:253  dwarn:0   dfail:0   fail:0   skip:35  time:483s
fi-byt-n2820     total:288  pass:249  dwarn:0   dfail:0   fail:0   skip:39  time:472s
fi-cfl-8700k     total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:406s
fi-cfl-s2        total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:576s
fi-cnl-y3        total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:582s
fi-elk-e7500     total:288  pass:229  dwarn:0   dfail:0   fail:0   skip:59  time:415s
fi-gdg-551       total:288  pass:179  dwarn:0   dfail:0   fail:1   skip:108 time:290s
fi-glk-1         total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:517s
fi-hsw-4770      total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:400s
fi-ilk-650       total:288  pass:227  dwarn:0   dfail:0   fail:1   skip:60  time:414s
fi-ivb-3520m     total:288  pass:259  dwarn:0   dfail:0   fail:0   skip:29  time:464s
fi-ivb-3770      total:288  pass:255  dwarn:0   dfail:0   fail:0   skip:33  time:418s
fi-kbl-7500u     total:288  pass:263  dwarn:1   dfail:0   fail:0   skip:24  time:478s
fi-kbl-7567u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:469s
fi-kbl-r         total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:506s
fi-pnv-d510      total:288  pass:222  dwarn:1   dfail:0   fail:0   skip:65  time:593s
fi-skl-6260u     total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:430s
fi-skl-6600u     total:288  pass:261  dwarn:0   dfail:0   fail:0   skip:27  time:523s
fi-skl-6700hq    total:288  pass:262  dwarn:0   dfail:0   fail:0   skip:26  time:535s
fi-skl-6700k2    total:288  pass:264  dwarn:0   dfail:0   fail:0   skip:24  time:501s
fi-skl-6770hq    total:288  pass:268  dwarn:0   dfail:0   fail:0   skip:20  time:481s
fi-skl-guc       total:288  pass:260  dwarn:0   dfail:0   fail:0   skip:28  time:423s
fi-snb-2520m     total:288  pass:248  dwarn:0   dfail:0   fail:0   skip:40  time:515s
fi-snb-2600      total:288  pass:248  dwarn:0   dfail:0   fail:0   skip:40  time:391s
Blacklisted hosts:
fi-cnl-drrs      total:288  pass:257  dwarn:3   dfail:0   fail:0   skip:19  time:521s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1094/issues.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

[igt-dev] ✗ Fi.CI.IGT: failure for tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)

[Patchwork]

== Series Details ==

Series: tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)
URL   : https://patchwork.freedesktop.org/series/39518/
State : failure

== Summary ==

---- Possible new issues:

Test kms_cursor_crc:
        Subgroup cursor-64x64-random:
                pass       -> INCOMPLETE (shard-hsw)

---- Known issues:

Test gem_eio:
        Subgroup in-flight:
                incomplete -> PASS       (shard-apl) fdo#105341
Test kms_cursor_crc:
        Subgroup cursor-256x256-suspend:
                incomplete -> PASS       (shard-hsw) fdo#103375 +1
Test kms_vblank:
        Subgroup pipe-a-ts-continuation-dpms-suspend:
                incomplete -> PASS       (shard-hsw) fdo#103540

fdo#105341 https://bugs.freedesktop.org/show_bug.cgi?id=105341
fdo#103375 https://bugs.freedesktop.org/show_bug.cgi?id=103375
fdo#103540 https://bugs.freedesktop.org/show_bug.cgi?id=103540

shard-apl        total:3467 pass:1825 dwarn:1   dfail:0   fail:8   skip:1632 time:12322s
shard-hsw        total:3421 pass:1754 dwarn:1   dfail:0   fail:1   skip:1663 time:10826s
shard-snb        total:3467 pass:1364 dwarn:1   dfail:0   fail:2   skip:2100 time:7013s
Blacklisted hosts:
shard-kbl        total:3281 pass:1838 dwarn:9   dfail:0   fail:9   skip:1423 time:8774s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1094/shards.html
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] ✗ Fi.CI.IGT: failure for tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)

[Tvrtko Ursulin]

Much better accuracy with these tweaks.

Looks like WC writes and ioctls were slow and were affecting the 
self-calibration. Although I don't have the explanation on why were the 
50% tests most affected, especially compared with 2% ones. Shrug.

Will it be good enough for KASAN enabled runs I don't know.

Regards,

Tvrtko

On 09/03/2018 16:20, Patchwork wrote:
> == Series Details ==
> 
> Series: tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)
> URL   : https://patchwork.freedesktop.org/series/39518/
> State : failure
> 
> == Summary ==
> 
> ---- Possible new issues:
> 
> Test kms_cursor_crc:
>          Subgroup cursor-64x64-random:
>                  pass       -> INCOMPLETE (shard-hsw)
> 
> ---- Known issues:
> 
> Test gem_eio:
>          Subgroup in-flight:
>                  incomplete -> PASS       (shard-apl) fdo#105341
> Test kms_cursor_crc:
>          Subgroup cursor-256x256-suspend:
>                  incomplete -> PASS       (shard-hsw) fdo#103375 +1
> Test kms_vblank:
>          Subgroup pipe-a-ts-continuation-dpms-suspend:
>                  incomplete -> PASS       (shard-hsw) fdo#103540
> 
> fdo#105341 https://bugs.freedesktop.org/show_bug.cgi?id=105341
> fdo#103375 https://bugs.freedesktop.org/show_bug.cgi?id=103375
> fdo#103540 https://bugs.freedesktop.org/show_bug.cgi?id=103540
> 
> shard-apl        total:3467 pass:1825 dwarn:1   dfail:0   fail:8   skip:1632 time:12322s
> shard-hsw        total:3421 pass:1754 dwarn:1   dfail:0   fail:1   skip:1663 time:10826s
> shard-snb        total:3467 pass:1364 dwarn:1   dfail:0   fail:2   skip:2100 time:7013s
> Blacklisted hosts:
> shard-kbl        total:3281 pass:1838 dwarn:9   dfail:0   fail:9   skip:1423 time:8774s
> 
> == Logs ==
> 
> For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1094/shards.html
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
> 
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] ✗ Fi.CI.IGT: failure for tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)

[Chris Wilson]

Quoting Tvrtko Ursulin (2018-03-09 16:45:31)
> 
> Much better accuracy with these tweaks.
> 
> Looks like WC writes and ioctls were slow and were affecting the 
> self-calibration. Although I don't have the explanation on why were the 
> 50% tests most affected, especially compared with 2% ones. Shrug.

loop duration for 2%: 2500 + 122500 = 125000us
loop duration for 50%: 2500 + 2500 = 5000us

So 25x more ioctls at 50%.

Bound to be the ioctls, scary. We need to track down the cause of that.
A latency histogram just to see the distribution? kcov tracing for the
outliers? Though ftrace is probably better, if we assume that it's
likely outside forces (the path through i915 should be pretty static --
or is it???).

Something like capture an ftrace snippet for each ioctl (having
reproduced something that show the spikes or whatever); then throw away
all traces that lie within the normal distribution and look for patterns
in the outliers?

Hmm, my guess would be ksoftirqd. If the submit wasn't immediate then it
would only be run when the RT calibration thread slept (the submit will
be from the same cpu because it's a tasklet). Ho hum, that sounds very
plausible.
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] ✗ Fi.CI.IGT: failure for tests/perf_pmu: Use absolute tolerance in accuracy tests (rev2)

[Tvrtko Ursulin]

On 09/03/2018 17:09, Chris Wilson wrote:
> Quoting Tvrtko Ursulin (2018-03-09 16:45:31)
>>
>> Much better accuracy with these tweaks.
>>
>> Looks like WC writes and ioctls were slow and were affecting the
>> self-calibration. Although I don't have the explanation on why were the
>> 50% tests most affected, especially compared with 2% ones. Shrug.
> 
> loop duration for 2%: 2500 + 122500 = 125000us
> loop duration for 50%: 2500 + 2500 = 5000us
> 
> So 25x more ioctls at 50%.
> 
> Bound to be the ioctls, scary. We need to track down the cause of that.
> A latency histogram just to see the distribution? kcov tracing for the
> outliers? Though ftrace is probably better, if we assume that it's
> likely outside forces (the path through i915 should be pretty static --
> or is it???).
> 
> Something like capture an ftrace snippet for each ioctl (having
> reproduced something that show the spikes or whatever); then throw away
> all traces that lie within the normal distribution and look for patterns
> in the outliers?
> 
> Hmm, my guess would be ksoftirqd. If the submit wasn't immediate then it
> would only be run when the RT calibration thread slept (the submit will
> be from the same cpu because it's a tasklet). Ho hum, that sounds very
> plausible.

Yes tasklet delays.. I even realized, well suspected, that while 
tweaking the test. By the time shard results came in it was long 
forgotten. :(

Huge difference in number of execbufs did not strike me though, well 
spotted. It is fewest towards the edges, and most in the middle. So it 
makes perfect sense. (98% comes up as 160000us busy, 3264us idle.)

Regards,

Tvrtko
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev