* [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
@ 2020-11-02 9:08 Anand Moon
2020-11-03 10:49 ` Chris Wilson
0 siblings, 1 reply; 7+ messages in thread
From: Anand Moon @ 2020-11-02 9:08 UTC (permalink / raw)
To: igt-dev; +Cc: Anand Moon
Using confidence interval formula accuracy is estimates approximately
at 95% when {X} is the average of a sample of size n.
Use this confidence interval formula as used in statistics
to estimate the accuracy.
Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
---
See below link.
https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulative_distribution_function
---
tests/kms_setmode.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
index 92f3ead2..58814a13 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
* which depend on timing, giving the lowest acceptable MTBF of 5.6s
* for 60Hz sampling rate.
*/
- accuracy = 3. * stddev;
+ accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
igt_info("Expected frametime: %.0fus; measured %.1fus +- %.3fus accuracy %.2f%% [%.2f scanlines]\n",
expected, mean, stddev,
--
2.25.1
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
2020-11-02 9:08 [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval Anand Moon
@ 2020-11-03 10:49 ` Chris Wilson
2020-11-03 17:58 ` Ram Moon, AnandX
0 siblings, 1 reply; 7+ messages in thread
From: Chris Wilson @ 2020-11-03 10:49 UTC (permalink / raw)
To: Anand Moon, igt-dev; +Cc: Anand Moon
Quoting Anand Moon (2020-11-02 09:08:36)
> Using confidence interval formula accuracy is estimates approximately
> at 95% when {X} is the average of a sample of size n.
> Use this confidence interval formula as used in statistics
> to estimate the accuracy.
>
> Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
> ---
> See below link.
> https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulative_distribution_function
> ---
> tests/kms_setmode.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
> index 92f3ead2..58814a13 100644
> --- a/tests/kms_setmode.c
> +++ b/tests/kms_setmode.c
> @@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> * which depend on timing, giving the lowest acceptable MTBF of 5.6s
> * for 60Hz sampling rate.
> */
> - accuracy = 3. * stddev;
> + accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
Oops, yes, this should be standard error rather than deviation. Could you
add igt_stats_get_std_error() so we don't hardcode N?
-Chris
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
2020-11-03 10:49 ` Chris Wilson
@ 2020-11-03 17:58 ` Ram Moon, AnandX
2020-11-20 9:12 ` Ram Moon, AnandX
0 siblings, 1 reply; 7+ messages in thread
From: Ram Moon, AnandX @ 2020-11-03 17:58 UTC (permalink / raw)
To: Chris Wilson, igt-dev
Hi Chris,
Thanks for your review comment.
-----Original Message-----
From: Chris Wilson <chris@chris-wilson.co.uk>
Sent: Tuesday, November 3, 2020 4:19 PM
To: Ram Moon, AnandX <anandx.ram.moon@intel.com>; igt-dev@lists.freedesktop.org
Cc: Ram Moon, AnandX <anandx.ram.moon@intel.com>
Subject: Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
Quoting Anand Moon (2020-11-02 09:08:36)
> Using confidence interval formula accuracy is estimates approximately
> at 95% when {X} is the average of a sample of size n.
> Use this confidence interval formula as used in statistics to estimate
> the accuracy.
>
> Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
> ---
> See below link.
> https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulati
> ve_distribution_function
> ---
> tests/kms_setmode.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c index
> 92f3ead2..58814a13 100644
> --- a/tests/kms_setmode.c
> +++ b/tests/kms_setmode.c
> @@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> * which depend on timing, giving the lowest acceptable MTBF of 5.6s
> * for 60Hz sampling rate.
> */
> - accuracy = 3. * stddev;
> + accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
Oops, yes, this should be standard error rather than deviation. Could you add igt_stats_get_std_error() so we don't hardcode N?
-Chris
Do you want me to rename accuracy to std_error and use stats.n_values instead of CALIBRATE_TS_STEPS
I could not find the definition of igt_stats_get_std_error function.
-Anand
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
2020-11-03 17:58 ` Ram Moon, AnandX
@ 2020-11-20 9:12 ` Ram Moon, AnandX
2020-11-20 15:05 ` Chris Wilson
0 siblings, 1 reply; 7+ messages in thread
From: Ram Moon, AnandX @ 2020-11-20 9:12 UTC (permalink / raw)
To: Chris Wilson, igt-dev
Hi Chris,
-----Original Message-----
From: Ram Moon, AnandX
Sent: Tuesday, November 3, 2020 11:28 PM
To: Chris Wilson <chris@chris-wilson.co.uk>; igt-dev@lists.freedesktop.org
Subject: RE: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
Hi Chris,
Thanks for your review comment.
-----Original Message-----
From: Chris Wilson <chris@chris-wilson.co.uk>
Sent: Tuesday, November 3, 2020 4:19 PM
To: Ram Moon, AnandX <anandx.ram.moon@intel.com>; igt-dev@lists.freedesktop.org
Cc: Ram Moon, AnandX <anandx.ram.moon@intel.com>
Subject: Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
Quoting Anand Moon (2020-11-02 09:08:36)
> Using confidence interval formula accuracy is estimates approximately
> at 95% when {X} is the average of a sample of size n.
> Use this confidence interval formula as used in statistics to estimate
> the accuracy.
>
> Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
> ---
> See below link.
> https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulati
> ve_distribution_function
> ---
> tests/kms_setmode.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c index
> 92f3ead2..58814a13 100644
> --- a/tests/kms_setmode.c
> +++ b/tests/kms_setmode.c
> @@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> * which depend on timing, giving the lowest acceptable MTBF of 5.6s
> * for 60Hz sampling rate.
> */
> - accuracy = 3. * stddev;
> + accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
Oops, yes, this should be standard error rather than deviation. Could you add igt_stats_get_std_error() so we don't hardcode N?
-Chris
Do you want me to rename accuracy to std_error and use stats.n_values instead of CALIBRATE_TS_STEPS I could not find the definition of igt_stats_get_std_error function.
-Anand
Thanks for fixing the code changes.
Actually above changes fix the issue to improve the accuracy calculations.
But now I am able to reproduce the original bug using 4K monitors on 5.8.x kernel, See below logs.
CRTC[98] [Pipe A] Mode: 3840x2160@60Hz Connectors: DP-2[330] Expected frametime: 16668us; measured 16672.3us +- 1.805us accuracy 0.00% [0.07 scanlines]
(kms_setmode:14315) CRITICAL: Test assertion failure function check_timings, file ../tests/kms_setmode.c:544:
(kms_setmode:14315) CRITICAL: Failed assertion: fabs(mean - expected) < 1.718 * stddev
(kms_setmode:14315) CRITICAL: vblank interval differs from modeline! expected 16667.6us, measured 16672us +- 1.805us, difference 4.7us (2.6 sigma) Stack trace:
#0 ../lib/igt_core.c:1734 __igt_fail_assert()
#1 ../tests/kms_setmode.c:544 test_one_combination()
#2 ../tests/kms_setmode.c:834 __real_main883()
#3 ../tests/kms_setmode.c:883 main()
#4 ../csu/libc-start.c:342 __libc_start_main()
#5 [_start+0x2e]
Subtest basic failed.
**** DEBUG ****
So after in digging into the below comments it turns out that *confidence interval* with respect to standard deviation is calculated with *Cumulative distribution function* (formula)
As per the Wikipedia table
For various values of z, the percentage of values expected to lie in and outside the symmetric interval, CI = (−zσ, zσ), are as follows:
[0] https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
It turns out that *3 σ* gives around proportion percentage 99.7300204% which is according to the CI = (−zσ, zσ) table.
I would like to submit a below patch as this changes fix the overall bug.
Please share your thought and suggestion on any alternative approach I should try.
Thanks
Anand
naveen@naveen-GB-Z390-Garuda:~/anand/igt-gpu-tools.internal-rebasing$ git diff diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c index 928a5a8cf..58da36985 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -526,7 +526,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
* See:
* https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
*/
- igt_assert_f(fabs(mean - expected) < 1.718 * stddev,
+ igt_assert_f(fabs(mean - expected) < 3. * stddev,
"vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma)\n",
expected, mean, stddev,
fabs(mean - expected), fabs(mean - expected) / stddev);
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
2020-11-20 9:12 ` Ram Moon, AnandX
@ 2020-11-20 15:05 ` Chris Wilson
2020-11-23 13:57 ` Ram Moon, AnandX
0 siblings, 1 reply; 7+ messages in thread
From: Chris Wilson @ 2020-11-20 15:05 UTC (permalink / raw)
To: Ram Moon, AnandX, igt-dev
Quoting Ram Moon, AnandX (2020-11-20 09:12:40)
> Hi Chris,
>
> -----Original Message-----
> From: Ram Moon, AnandX
> Sent: Tuesday, November 3, 2020 11:28 PM
> To: Chris Wilson <chris@chris-wilson.co.uk>; igt-dev@lists.freedesktop.org
> Subject: RE: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
>
> Hi Chris,
>
> Thanks for your review comment.
>
> -----Original Message-----
> From: Chris Wilson <chris@chris-wilson.co.uk>
> Sent: Tuesday, November 3, 2020 4:19 PM
> To: Ram Moon, AnandX <anandx.ram.moon@intel.com>; igt-dev@lists.freedesktop.org
> Cc: Ram Moon, AnandX <anandx.ram.moon@intel.com>
> Subject: Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
>
> Quoting Anand Moon (2020-11-02 09:08:36)
> > Using confidence interval formula accuracy is estimates approximately
> > at 95% when {X} is the average of a sample of size n.
> > Use this confidence interval formula as used in statistics to estimate
> > the accuracy.
> >
> > Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
> > ---
> > See below link.
> > https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulati
> > ve_distribution_function
> > ---
> > tests/kms_setmode.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c index
> > 92f3ead2..58814a13 100644
> > --- a/tests/kms_setmode.c
> > +++ b/tests/kms_setmode.c
> > @@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> > * which depend on timing, giving the lowest acceptable MTBF of 5.6s
> > * for 60Hz sampling rate.
> > */
> > - accuracy = 3. * stddev;
> > + accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
>
> Oops, yes, this should be standard error rather than deviation. Could you add igt_stats_get_std_error() so we don't hardcode N?
> -Chris
>
> Do you want me to rename accuracy to std_error and use stats.n_values instead of CALIBRATE_TS_STEPS I could not find the definition of igt_stats_get_std_error function.
>
> -Anand
>
> Thanks for fixing the code changes.
>
> Actually above changes fix the issue to improve the accuracy calculations.
> But now I am able to reproduce the original bug using 4K monitors on 5.8.x kernel, See below logs.
>
> CRTC[98] [Pipe A] Mode: 3840x2160@60Hz Connectors: DP-2[330] Expected frametime: 16668us; measured 16672.3us +- 1.805us accuracy 0.00% [0.07 scanlines]
> (kms_setmode:14315) CRITICAL: Test assertion failure function check_timings, file ../tests/kms_setmode.c:544:
> (kms_setmode:14315) CRITICAL: Failed assertion: fabs(mean - expected) < 1.718 * stddev
> (kms_setmode:14315) CRITICAL: vblank interval differs from modeline! expected 16667.6us, measured 16672us +- 1.805us, difference 4.7us (2.6 sigma) Stack trace:
> #0 ../lib/igt_core.c:1734 __igt_fail_assert()
> #1 ../tests/kms_setmode.c:544 test_one_combination()
> #2 ../tests/kms_setmode.c:834 __real_main883()
> #3 ../tests/kms_setmode.c:883 main()
> #4 ../csu/libc-start.c:342 __libc_start_main()
> #5 [_start+0x2e]
> Subtest basic failed.
> **** DEBUG ****
>
> So after in digging into the below comments it turns out that *confidence interval* with respect to standard deviation is calculated with *Cumulative distribution function* (formula)
>
> As per the Wikipedia table
> For various values of z, the percentage of values expected to lie in and outside the symmetric interval, CI = (−zσ, zσ), are as follows:
> [0] https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_distributed_data
>
> It turns out that *3 σ* gives around proportion percentage 99.7300204% which is according to the CI = (−zσ, zσ) table.
>
> I would like to submit a below patch as this changes fix the overall bug.
> Please share your thought and suggestion on any alternative approach I should try.
You would also have to update the comments as it no longer matches the
broaden 90% critical value.
Still debating what is a sensible error rate, but in the meantime I
would suggest a different approach: ignore the deviation if it is within
a single scanline.
Something like:
- igt_assert_f(fabs(mean - expected) < 1.718 * stddev,
- "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma)\n",
+ igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
+ "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f scanlines)\n",
expected, mean, stddev,
- fabs(mean - expected), fabs(mean - expected) / stddev);
+ fabs(mean - expected),
+ fabs(mean - expected) / stddev,
+ fabs(mean - expected) / line_time(kmode));
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
2020-11-20 15:05 ` Chris Wilson
@ 2020-11-23 13:57 ` Ram Moon, AnandX
0 siblings, 0 replies; 7+ messages in thread
From: Ram Moon, AnandX @ 2020-11-23 13:57 UTC (permalink / raw)
To: Chris Wilson, igt-dev
Hi Chris,
-----Original Message-----
From: Chris Wilson <chris@chris-wilson.co.uk>
Sent: Friday, November 20, 2020 8:36 PM
To: Ram Moon, AnandX <anandx.ram.moon@intel.com>; igt-dev@lists.freedesktop.org
Subject: Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
Quoting Ram Moon, AnandX (2020-11-20 09:12:40)
> Hi Chris,
>
> -----Original Message-----
> From: Ram Moon, AnandX
> Sent: Tuesday, November 3, 2020 11:28 PM
> To: Chris Wilson <chris@chris-wilson.co.uk>;
> igt-dev@lists.freedesktop.org
> Subject: RE: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve
> accuracy with using of confidence interval
>
> Hi Chris,
>
> Thanks for your review comment.
>
> -----Original Message-----
> From: Chris Wilson <chris@chris-wilson.co.uk>
> Sent: Tuesday, November 3, 2020 4:19 PM
> To: Ram Moon, AnandX <anandx.ram.moon@intel.com>;
> igt-dev@lists.freedesktop.org
> Cc: Ram Moon, AnandX <anandx.ram.moon@intel.com>
> Subject: Re: [igt-dev] [i-g-t] tests/kms_setmode: basic Improve
> accuracy with using of confidence interval
>
> Quoting Anand Moon (2020-11-02 09:08:36)
> > Using confidence interval formula accuracy is estimates
> > approximately at 95% when {X} is the average of a sample of size n.
> > Use this confidence interval formula as used in statistics to
> > estimate the accuracy.
> >
> > Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
> > ---
> > See below link.
> > https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumula
> > ti
> > ve_distribution_function
> > ---
> > tests/kms_setmode.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c index
> > 92f3ead2..58814a13 100644
> > --- a/tests/kms_setmode.c
> > +++ b/tests/kms_setmode.c
> > @@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
> > * which depend on timing, giving the lowest acceptable MTBF of 5.6s
> > * for 60Hz sampling rate.
> > */
> > - accuracy = 3. * stddev;
> > + accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
>
> Oops, yes, this should be standard error rather than deviation. Could you add igt_stats_get_std_error() so we don't hardcode N?
> -Chris
>
> Do you want me to rename accuracy to std_error and use stats.n_values instead of CALIBRATE_TS_STEPS I could not find the definition of igt_stats_get_std_error function.
>
> -Anand
>
> Thanks for fixing the code changes.
>
> Actually above changes fix the issue to improve the accuracy calculations.
> But now I am able to reproduce the original bug using 4K monitors on 5.8.x kernel, See below logs.
>
> CRTC[98] [Pipe A] Mode: 3840x2160@60Hz Connectors: DP-2[330]
> Expected frametime: 16668us; measured 16672.3us +- 1.805us accuracy
> 0.00% [0.07 scanlines]
> (kms_setmode:14315) CRITICAL: Test assertion failure function check_timings, file ../tests/kms_setmode.c:544:
> (kms_setmode:14315) CRITICAL: Failed assertion: fabs(mean - expected)
> < 1.718 * stddev
> (kms_setmode:14315) CRITICAL: vblank interval differs from modeline! expected 16667.6us, measured 16672us +- 1.805us, difference 4.7us (2.6 sigma) Stack trace:
> #0 ../lib/igt_core.c:1734 __igt_fail_assert()
> #1 ../tests/kms_setmode.c:544 test_one_combination()
> #2 ../tests/kms_setmode.c:834 __real_main883()
> #3 ../tests/kms_setmode.c:883 main()
> #4 ../csu/libc-start.c:342 __libc_start_main()
> #5 [_start+0x2e]
> Subtest basic failed.
> **** DEBUG ****
>
> So after in digging into the below comments it turns out that
> *confidence interval* with respect to standard deviation is calculated
> with *Cumulative distribution function* (formula)
>
> As per the Wikipedia table
> For various values of z, the percentage of values expected to lie in and outside the symmetric interval, CI = (−zσ, zσ), are as follows:
> [0]
> https://en.wikipedia.org/wiki/Standard_deviation#Rules_for_normally_di
> stributed_data
>
> It turns out that *3 σ* gives around proportion percentage 99.7300204% which is according to the CI = (−zσ, zσ) table.
>
> I would like to submit a below patch as this changes fix the overall bug.
> Please share your thought and suggestion on any alternative approach I should try.
You would also have to update the comments as it no longer matches the broaden 90% critical value.
Still debating what is a sensible error rate, but in the meantime I would suggest a different approach: ignore the deviation if it is within a single scanline.
Something like:
- igt_assert_f(fabs(mean - expected) < 1.718 * stddev,
- "vblank interval differs from modeline! expected %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma)\n",
+ igt_assert_f(fabs(mean - expected) < max(line_time(kmode), 1.718 * stddev),
+ "vblank interval differs from modeline! expected
+ %.1fus, measured %1.fus +- %.3fus, difference %.1fus (%.1f sigma, %.1f
+ scanlines)\n",
expected, mean, stddev,
- fabs(mean - expected), fabs(mean - expected) / stddev);
+ fabs(mean - expected),
+ fabs(mean - expected) / stddev,
+ fabs(mean - expected) / line_time(kmode));
Thanks for this new approach, I have tested this patch on different machines
with different modes as below.
Mode: 1920x1080@60Hz Connectors: HDMI-A-2
Mode: 1920x1080@60Hz Connectors: DP-1
Mode: 1920x1080@60Hz Connectors: DP-2
Mode: 2560x1440@60Hz Connectors: eDP-1
Mode: 3840x2160@60Hz Connectors: DP-2
And all the testcase regarding kms_setmode seems to passing on 5.8.x and 5.10-0-rcx kernel.
Thanks
-Anand
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply [flat|nested] 7+ messages in thread
* [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval
@ 2020-11-02 9:02 Anand Moon
0 siblings, 0 replies; 7+ messages in thread
From: Anand Moon @ 2020-11-02 9:02 UTC (permalink / raw)
To: igt-dev, Dorota Czaplejewicz; +Cc: Anand Moon
Using confidence interval formula accuracy is estimates approximately
at 95% when {X} is the average of a sample of size n.
Use this confidence interval formula as used in statistics
to estimate the accuracy.
Cc: Dorota Czaplejewicz <dorota.czaplejewicz@collabora.co.uk>
Signed-off-by: Anand Moon <anandx.ram.moon@intel.com>
---
See below link.
https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Cumulative_distribution_function
---
tests/kms_setmode.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/kms_setmode.c b/tests/kms_setmode.c
index 92f3ead2..58814a13 100644
--- a/tests/kms_setmode.c
+++ b/tests/kms_setmode.c
@@ -494,7 +494,7 @@ static void check_timings(int crtc_idx, const drmModeModeInfo *kmode)
* which depend on timing, giving the lowest acceptable MTBF of 5.6s
* for 60Hz sampling rate.
*/
- accuracy = 3. * stddev;
+ accuracy = 3. * stddev / sqrt(CALIBRATE_TS_STEPS);
igt_info("Expected frametime: %.0fus; measured %.1fus +- %.3fus accuracy %.2f%% [%.2f scanlines]\n",
expected, mean, stddev,
--
2.25.1
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-11-23 13:57 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-02 9:08 [igt-dev] [i-g-t] tests/kms_setmode: basic Improve accuracy with using of confidence interval Anand Moon
2020-11-03 10:49 ` Chris Wilson
2020-11-03 17:58 ` Ram Moon, AnandX
2020-11-20 9:12 ` Ram Moon, AnandX
2020-11-20 15:05 ` Chris Wilson
2020-11-23 13:57 ` Ram Moon, AnandX
-- strict thread matches above, loose matches on Subject: below --
2020-11-02 9:02 Anand Moon
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.