* [LTP] ee
@ 2021-06-29 20:22 Petr Vorel
2021-06-29 20:59 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
2021-06-30 7:11 ` [LTP] ee Richard Palethorpe
0 siblings, 2 replies; 8+ messages in thread
From: Petr Vorel @ 2021-06-29 20:22 UTC (permalink / raw)
To: ltp
Hi Richie,
sporadically tst_fuzzy_sync01 fails. I wonder if we can relax it somehow.
I found it while working on enabling API tests in CI, I'll have to disable this
one to avoid false positives.
Kind regards,
Petr
./tst_fuzzy_sync01
tst_test.c:1311: TINFO: Timeout per run is 0h 05m 00s
tst_fuzzy_sync01.c:224: TPASS: acs:0 act:0 art:0 | =:101 -:148 +:65
tst_fuzzy_sync01.c:224: TPASS: acs:0 act:1 art:0 | =:101 -:0 +:0
tst_fuzzy_sync01.c:224: TPASS: acs:1 act:1 art:1 | =:101 -:0 +:0
tst_fuzzy_sync01.c:224: TPASS: acs:3 act:1 art:1 | =:101 -:5 +:1
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -17ns, avg_dev = 4ns, dev_ratio = 0.25 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20773ns, avg_dev = 4714ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 20849ns, avg_dev = 4710ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -94ns, avg_dev = 24ns, dev_ratio = 0.26 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 54 , avg_dev = 17 , dev_ratio = 0.31 }
../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:3 act:1 art:1 | =:0 -:1 +:2999999
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -18ns, avg_dev = 8ns, dev_ratio = 0.46 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 21475ns, avg_dev = 4558ns, dev_ratio = 0.21 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 19960ns, avg_dev = 4702ns, dev_ratio = 0.24 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 1497ns, avg_dev = 1956ns, dev_ratio = 1.31 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 43 , avg_dev = 17 , dev_ratio = 0.39 }
../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = 1529ns, avg_dev = 1930ns, dev_ratio = 1.26 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 18630ns, avg_dev = 4565ns, dev_ratio = 0.25 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 8430ns, avg_dev = 4163ns, dev_ratio = 0.49 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 11729ns, avg_dev = 2725ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6115 , avg_dev = 33 , dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:3 act:1 art:1 | =:1 -:1 +:2999998
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -126ns, avg_dev = 256ns, dev_ratio = 2.03 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 19028ns, avg_dev = 4655ns, dev_ratio = 0.24 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 5548ns, avg_dev = 1172ns, dev_ratio = 0.21 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 13354ns, avg_dev = 4411ns, dev_ratio = 0.33 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6070 , avg_dev = 157 , dev_ratio = 0.03 }
tst_fuzzy_sync01.c:224: TPASS: acs:1 act:1 art:3 | =:101 -:69713 +:82
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -1ns, avg_dev = 35ns, dev_ratio = 41.69 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 5178ns, avg_dev = 496ns, dev_ratio = 0.10 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 20746ns, avg_dev = 4711ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -15569ns, avg_dev = 4639ns, dev_ratio = 0.30 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6127 , avg_dev = 48 , dev_ratio = 0.01 }
tst_fuzzy_sync01.c:224: TPASS: acs:2 act:1 art:1 | =:101 -:67 +:103496
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -174ns, avg_dev = 515ns, dev_ratio = 2.95 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 5382ns, avg_dev = 1036ns, dev_ratio = 0.19 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 19221ns, avg_dev = 4722ns, dev_ratio = 0.25 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -14014ns, avg_dev = 4570ns, dev_ratio = 0.33 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6141 , avg_dev = 57 , dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:678: TINFO: Exceeded execution time, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:2 | =:6 -:2311522 +:7
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -16ns, avg_dev = 3ns, dev_ratio = 0.20 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20317ns, avg_dev = 4728ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 20374ns, avg_dev = 4707ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -73ns, avg_dev = 31ns, dev_ratio = 0.42 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 58 , avg_dev = 18 , dev_ratio = 0.31 }
../../include/tst_fuzzy_sync.h:532: TINFO: Reached deviation ratios < 0.10, introducing randomness
../../include/tst_fuzzy_sync.h:535: TINFO: Delay range is [-14037, 8587]
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 663327, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -43ns, avg_dev = 3ns, dev_ratio = 0.08 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 17713ns, avg_dev = 59ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 28954ns, avg_dev = 31ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -11283ns, avg_dev = 43ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 5471 , avg_dev = 38 , dev_ratio = 0.01 }
tst_fuzzy_sync01.c:224: TPASS: acs:3 act:1 art:0 | =:101 -:186 +:666740
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -12ns, avg_dev = 6ns, dev_ratio = 0.47 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20287ns, avg_dev = 4717ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 20310ns, avg_dev = 4685ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -35ns, avg_dev = 41ns, dev_ratio = 1.18 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 18 , avg_dev = 14 , dev_ratio = 0.77 }
../../include/tst_fuzzy_sync.h:678: TINFO: Exceeded execution time, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:0 act:1 art:3 | =:3 -:837016 +:1
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -30ns, avg_dev = 8ns, dev_ratio = 0.28 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20287ns, avg_dev = 4703ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 4954ns, avg_dev = 1134ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 15303ns, avg_dev = 4468ns, dev_ratio = 0.29 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6042 , avg_dev = 34 , dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:532: TINFO: Reached deviation ratios < 0.10, introducing randomness
../../include/tst_fuzzy_sync.h:535: TINFO: Delay range is [-2861, 8786]
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 402095, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -18ns, avg_dev = 2ns, dev_ratio = 0.09 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 14168ns, avg_dev = 585ns, dev_ratio = 0.04 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 4615ns, avg_dev = 10ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 9535ns, avg_dev = 587ns, dev_ratio = 0.06 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 5913 , avg_dev = 157 , dev_ratio = 0.03 }
tst_fuzzy_sync01.c:224: TPASS: acs:3 act:1 art:0 | =:101 -:95 +:403404
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -8ns, avg_dev = 0ns, dev_ratio = 0.06 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20354ns, avg_dev = 4775ns, dev_ratio = 0.23 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 4607ns, avg_dev = 8ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 15739ns, avg_dev = 4783ns, dev_ratio = 0.30 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6109 , avg_dev = 66 , dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:678: TINFO: Exceeded execution time, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:0 act:1 art:3 | =:31 -:210089 +:83
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -1ns, avg_dev = 6ns, dev_ratio = 5.49 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 4594ns, avg_dev = 185ns, dev_ratio = 0.04 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 18613ns, avg_dev = 4594ns, dev_ratio = 0.25 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -14020ns, avg_dev = 4509ns, dev_ratio = 0.32 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6192 , avg_dev = 44 , dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:678: TINFO: Exceeded execution time, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:2 act:1 art:0 | =:7 -:0 +:142178
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = 314ns, avg_dev = 251ns, dev_ratio = 0.80 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 4606ns, avg_dev = 27ns, dev_ratio = 0.01 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 14040ns, avg_dev = 44ns, dev_ratio = 0.00 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -9119ns, avg_dev = 234ns, dev_ratio = 0.03 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 5928 , avg_dev = 146 , dev_ratio = 0.02 }
../../include/tst_fuzzy_sync.h:678: TINFO: Exceeded execution time, requesting exit
tst_fuzzy_sync01.c:224: TFAIL: acs:0 act:1 art:2 | =:0 -:47266 +:0
../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -7ns, avg_dev = 7ns, dev_ratio = 0.98 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 19278ns, avg_dev = 4713ns, dev_ratio = 0.24 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 646ns, avg_dev = 82ns, dev_ratio = 0.13 }
../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 18625ns, avg_dev = 4725ns, dev_ratio = 0.25 }
../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 8988 , avg_dev = 20 , dev_ratio = 0.00 }
Test timeouted, sending SIGKILL!
tst_test.c:1357: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1359: TBROK: Test killed! (timeout?)
Summary:
passed 8
failed 8
broken 1
skipped 0
warnings 0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] tst_fuzzy_sync01 sporadically fails
2021-06-29 20:22 [LTP] ee Petr Vorel
@ 2021-06-29 20:59 ` Petr Vorel
2021-06-30 7:11 ` [LTP] ee Richard Palethorpe
1 sibling, 0 replies; 8+ messages in thread
From: Petr Vorel @ 2021-06-29 20:59 UTC (permalink / raw)
To: ltp
I'm sorry, the subject was supposed to be tst_fuzzy_sync01 sporadically fails.
Kind regards,
Petr
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] ee
2021-06-29 20:22 [LTP] ee Petr Vorel
2021-06-29 20:59 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
@ 2021-06-30 7:11 ` Richard Palethorpe
2021-06-30 8:01 ` Petr Vorel
1 sibling, 1 reply; 8+ messages in thread
From: Richard Palethorpe @ 2021-06-30 7:11 UTC (permalink / raw)
To: ltp
Hello Petr,
Petr Vorel <pvorel@suse.cz> writes:
> Hi Richie,
>
> sporadically tst_fuzzy_sync01 fails. I wonder if we can relax it somehow.
> I found it while working on enabling API tests in CI, I'll have to disable this
> one to avoid false positives.
>
> Kind regards,
> Petr
>
> ./tst_fuzzy_sync01
> tst_test.c:1311: TINFO: Timeout per run is 0h 05m 00s
> tst_fuzzy_sync01.c:224: TPASS: acs:0 act:0 art:0 | =:101 -:148 +:65
> tst_fuzzy_sync01.c:224: TPASS: acs:0 act:1 art:0 | =:101 -:0 +:0
> tst_fuzzy_sync01.c:224: TPASS: acs:1 act:1 art:1 | =:101 -:0 +:0
> tst_fuzzy_sync01.c:224: TPASS: acs:3 act:1 art:1 | =:101 -:5 +:1
> ../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
> ../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
> ../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -17ns, avg_dev = 4ns, dev_ratio = 0.25 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 20773ns, avg_dev = 4714ns, dev_ratio = 0.23 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 20849ns, avg_dev = 4710ns, dev_ratio = 0.23 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = -94ns, avg_dev = 24ns, dev_ratio = 0.26 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 54 , avg_dev = 17 , dev_ratio = 0.31 }
> ../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
> tst_fuzzy_sync01.c:224: TFAIL: acs:3 act:1 art:1 | =:0 -:1 +:2999999
> ../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
> ../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
> ../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = -18ns, avg_dev = 8ns, dev_ratio = 0.46 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 21475ns, avg_dev = 4558ns, dev_ratio = 0.21 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 19960ns, avg_dev = 4702ns, dev_ratio = 0.24 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 1497ns, avg_dev = 1956ns, dev_ratio = 1.31 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 43 , avg_dev = 17 , dev_ratio = 0.39 }
> ../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
> tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
> ../../include/tst_fuzzy_sync.h:522: TINFO: Minimum sampling period ended
> ../../include/tst_fuzzy_sync.h:345: TINFO: loop = 10000, delay_bias = 0
> ../../include/tst_fuzzy_sync.h:333: TINFO: start_a - start_b: { avg = 1529ns, avg_dev = 1930ns, dev_ratio = 1.26 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - start_a : { avg = 18630ns, avg_dev = 4565ns, dev_ratio = 0.25 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_b - start_b : { avg = 8430ns, avg_dev = 4163ns, dev_ratio = 0.49 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: end_a - end_b : { avg = 11729ns, avg_dev = 2725ns, dev_ratio = 0.23 }
> ../../include/tst_fuzzy_sync.h:333: TINFO: spins : { avg = 6115 , avg_dev = 33 , dev_ratio = 0.01 }
> ../../include/tst_fuzzy_sync.h:684: TINFO: Exceeded execution loops, requesting exit
> ...
It looks like the CI machines are too noisy/contended. The avg_dev is
very high. Probably we could relax the dev_ratio threshold to 0.2 or
0.3. Although we would still get failures occassionally. As this is a
probabalistic test.
Could you change the script so that it passes so long as the test
returns TPASS or TFAIL?
We don't want TBROK, TCONF or no result.
--
Thank you,
Richard.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] ee
2021-06-30 7:11 ` [LTP] ee Richard Palethorpe
@ 2021-06-30 8:01 ` Petr Vorel
2021-06-30 9:12 ` Richard Palethorpe
0 siblings, 1 reply; 8+ messages in thread
From: Petr Vorel @ 2021-06-30 8:01 UTC (permalink / raw)
To: ltp
Hi Richie,
...
> > tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
> It looks like the CI machines are too noisy/contended. The avg_dev is
> very high. Probably we could relax the dev_ratio threshold to 0.2 or
> 0.3. Although we would still get failures occassionally. As this is a
> probabalistic test.
Test is failing on my laptop, thus haven't enabled it in CI.
But maybe it'll be working on it more reliably than my busy machine.
But I'd prefer to wasting time with false positives, thus I guess we should
enable only tests which are working reliably.
> Could you change the script so that it passes so long as the test
> returns TPASS or TFAIL?
Well, accepting TFAIL sounds a bit strange to me :).
Also next effort will be (at least for shell tests) to compare actual test
output. Obviously that will not be straightforward for some tests, which aren't
reproducible (avg = 11729ns could be matched by regex, but having more variants
of results is kind of special case).
> We don't want TBROK, TCONF or no result.
FYI in my CI patchset is TCONF accepted. Motivation was to not require root for
make test as some tests needed it. Thus TCONF will be a special case, then I
guess we could add tst_fuzzy_sync01 accepting TFAIL as a special case.
Kind regards,
Petr
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] ee
2021-06-30 8:01 ` Petr Vorel
@ 2021-06-30 9:12 ` Richard Palethorpe
2021-06-30 10:10 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
0 siblings, 1 reply; 8+ messages in thread
From: Richard Palethorpe @ 2021-06-30 9:12 UTC (permalink / raw)
To: ltp
Hello Petr,
Petr Vorel <pvorel@suse.cz> writes:
> Hi Richie,
>
> ...
>> > tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
>
>> It looks like the CI machines are too noisy/contended. The avg_dev is
>> very high. Probably we could relax the dev_ratio threshold to 0.2 or
>> 0.3. Although we would still get failures occassionally. As this is a
>> probabalistic test.
> Test is failing on my laptop, thus haven't enabled it in CI.
> But maybe it'll be working on it more reliably than my busy machine.
Is it really that busy? Perhaps we should increase the dev ratio
threshold. Clearly the deviations from contention are not enough to
reproduce the races, but are enough to prevent the radomization phase.
> But I'd prefer to wasting time with false positives, thus I guess we should
> enable only tests which are working reliably.
>
>> Could you change the script so that it passes so long as the test
>> returns TPASS or TFAIL?
> Well, accepting TFAIL sounds a bit strange to me :).
> Also next effort will be (at least for shell tests) to compare actual test
> output. Obviously that will not be straightforward for some tests, which aren't
> reproducible (avg = 11729ns could be matched by regex, but having more variants
> of results is kind of special case).
>
>> We don't want TBROK, TCONF or no result.
> FYI in my CI patchset is TCONF accepted. Motivation was to not require root for
> make test as some tests needed it. Thus TCONF will be a special case, then I
> guess we could add tst_fuzzy_sync01 accepting TFAIL as a special case.
At least if we run the tests and look for TPASS or TFAIL, we will catch
segfaults and similar.
Also, for fuzzy sync, returning TCONF would be a major error. It should
run on all systems.
>
> Kind regards,
> Petr
--
Thank you,
Richard.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] tst_fuzzy_sync01 sporadically fails
2021-06-30 9:12 ` Richard Palethorpe
@ 2021-06-30 10:10 ` Petr Vorel
2021-06-30 12:05 ` Richard Palethorpe
0 siblings, 1 reply; 8+ messages in thread
From: Petr Vorel @ 2021-06-30 10:10 UTC (permalink / raw)
To: ltp
Hi Richie,
> Hello Petr,
> Petr Vorel <pvorel@suse.cz> writes:
> > Hi Richie,
> > ...
> >> > tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
> >> It looks like the CI machines are too noisy/contended. The avg_dev is
> >> very high. Probably we could relax the dev_ratio threshold to 0.2 or
> >> 0.3. Although we would still get failures occassionally. As this is a
> >> probabalistic test.
> > Test is failing on my laptop, thus haven't enabled it in CI.
> > But maybe it'll be working on it more reliably than my busy machine.
> Is it really that busy? Perhaps we should increase the dev ratio
> threshold. Clearly the deviations from contention are not enough to
> reproduce the races, but are enough to prevent the radomization phase.
I probably did some VM testing or kernel compilation or something.
I'll try to enable for next patchset version it to see how it works on CI.
> > But I'd prefer to wasting time with false positives, thus I guess we should
> > enable only tests which are working reliably.
> >> Could you change the script so that it passes so long as the test
> >> returns TPASS or TFAIL?
> > Well, accepting TFAIL sounds a bit strange to me :).
> > Also next effort will be (at least for shell tests) to compare actual test
> > output. Obviously that will not be straightforward for some tests, which aren't
> > reproducible (avg = 11729ns could be matched by regex, but having more variants
> > of results is kind of special case).
> >> We don't want TBROK, TCONF or no result.
> > FYI in my CI patchset is TCONF accepted. Motivation was to not require root for
> > make test as some tests needed it. Thus TCONF will be a special case, then I
> > guess we could add tst_fuzzy_sync01 accepting TFAIL as a special case.
> At least if we run the tests and look for TPASS or TFAIL, we will catch
> segfaults and similar.
> Also, for fuzzy sync, returning TCONF would be a major error. It should
> run on all systems.
Well, TCONF should be used on places where it's really a configuration issue.
IMHO only TBROK and TFAIL should be a problem. Or is fuzzy sync part somehow
special in this?
Kind regards,
Petr
> > Kind regards,
> > Petr
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] tst_fuzzy_sync01 sporadically fails
2021-06-30 10:10 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
@ 2021-06-30 12:05 ` Richard Palethorpe
2021-06-30 12:19 ` Petr Vorel
0 siblings, 1 reply; 8+ messages in thread
From: Richard Palethorpe @ 2021-06-30 12:05 UTC (permalink / raw)
To: ltp
Hello Petr,
Petr Vorel <pvorel@suse.cz> writes:
> Hi Richie,
>
>> Hello Petr,
>
>> Petr Vorel <pvorel@suse.cz> writes:
>
>> > Hi Richie,
>
>> > ...
>> >> > tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
>
>> >> It looks like the CI machines are too noisy/contended. The avg_dev is
>> >> very high. Probably we could relax the dev_ratio threshold to 0.2 or
>> >> 0.3. Although we would still get failures occassionally. As this is a
>> >> probabalistic test.
>> > Test is failing on my laptop, thus haven't enabled it in CI.
>> > But maybe it'll be working on it more reliably than my busy machine.
>
>> Is it really that busy? Perhaps we should increase the dev ratio
>> threshold. Clearly the deviations from contention are not enough to
>> reproduce the races, but are enough to prevent the radomization phase.
> I probably did some VM testing or kernel compilation or something.
> I'll try to enable for next patchset version it to see how it works on CI.
>
>> > But I'd prefer to wasting time with false positives, thus I guess we should
>> > enable only tests which are working reliably.
>
>> >> Could you change the script so that it passes so long as the test
>> >> returns TPASS or TFAIL?
>> > Well, accepting TFAIL sounds a bit strange to me :).
>> > Also next effort will be (at least for shell tests) to compare actual test
>> > output. Obviously that will not be straightforward for some tests, which aren't
>> > reproducible (avg = 11729ns could be matched by regex, but having more variants
>> > of results is kind of special case).
>
>> >> We don't want TBROK, TCONF or no result.
>> > FYI in my CI patchset is TCONF accepted. Motivation was to not require root for
>> > make test as some tests needed it. Thus TCONF will be a special case, then I
>> > guess we could add tst_fuzzy_sync01 accepting TFAIL as a special case.
>
>> At least if we run the tests and look for TPASS or TFAIL, we will catch
>> segfaults and similar.
>
>> Also, for fuzzy sync, returning TCONF would be a major error. It should
>> run on all systems.
> Well, TCONF should be used on places where it's really a configuration issue.
> IMHO only TBROK and TFAIL should be a problem. Or is fuzzy sync part somehow
> special in this?
I can't imagine any Linux config where fuzzy sync won't work. Even if we
are compiling with some libc that doesn't have POSIX threads, we can
work around that. Probably if it returns TCONF it's because some other
library func has an error in it.
For example if tst_ncpus_available starts aborting with TCONF. Then that
is an error. Fuzzy Sync should be able to work around that.
--
Thank you,
Richard.
^ permalink raw reply [flat|nested] 8+ messages in thread
* [LTP] tst_fuzzy_sync01 sporadically fails
2021-06-30 12:05 ` Richard Palethorpe
@ 2021-06-30 12:19 ` Petr Vorel
0 siblings, 0 replies; 8+ messages in thread
From: Petr Vorel @ 2021-06-30 12:19 UTC (permalink / raw)
To: ltp
> Hello Petr,
> Petr Vorel <pvorel@suse.cz> writes:
> > Hi Richie,
> >> Hello Petr,
> >> Petr Vorel <pvorel@suse.cz> writes:
> >> > Hi Richie,
> >> > ...
> >> >> > tst_fuzzy_sync01.c:224: TFAIL: acs:1 act:1 art:3 | =:3 -:2999996 +:1
> >> >> It looks like the CI machines are too noisy/contended. The avg_dev is
> >> >> very high. Probably we could relax the dev_ratio threshold to 0.2 or
> >> >> 0.3. Although we would still get failures occassionally. As this is a
> >> >> probabalistic test.
> >> > Test is failing on my laptop, thus haven't enabled it in CI.
> >> > But maybe it'll be working on it more reliably than my busy machine.
> >> Is it really that busy? Perhaps we should increase the dev ratio
> >> threshold. Clearly the deviations from contention are not enough to
> >> reproduce the races, but are enough to prevent the radomization phase.
> > I probably did some VM testing or kernel compilation or something.
> > I'll try to enable for next patchset version it to see how it works on CI.
> >> > But I'd prefer to wasting time with false positives, thus I guess we should
> >> > enable only tests which are working reliably.
> >> >> Could you change the script so that it passes so long as the test
> >> >> returns TPASS or TFAIL?
> >> > Well, accepting TFAIL sounds a bit strange to me :).
> >> > Also next effort will be (at least for shell tests) to compare actual test
> >> > output. Obviously that will not be straightforward for some tests, which aren't
> >> > reproducible (avg = 11729ns could be matched by regex, but having more variants
> >> > of results is kind of special case).
> >> >> We don't want TBROK, TCONF or no result.
> >> > FYI in my CI patchset is TCONF accepted. Motivation was to not require root for
> >> > make test as some tests needed it. Thus TCONF will be a special case, then I
> >> > guess we could add tst_fuzzy_sync01 accepting TFAIL as a special case.
> >> At least if we run the tests and look for TPASS or TFAIL, we will catch
> >> segfaults and similar.
> >> Also, for fuzzy sync, returning TCONF would be a major error. It should
> >> run on all systems.
> > Well, TCONF should be used on places where it's really a configuration issue.
> > IMHO only TBROK and TFAIL should be a problem. Or is fuzzy sync part somehow
> > special in this?
> I can't imagine any Linux config where fuzzy sync won't work. Even if we
> are compiling with some libc that doesn't have POSIX threads, we can
> work around that. Probably if it returns TCONF it's because some other
> library func has an error in it.
> For example if tst_ncpus_available starts aborting with TCONF. Then that
> is an error. Fuzzy Sync should be able to work around that.
Thanks for info, good to know. I'll see if I manage to handle this in this first
attempt, even if not I see we'll need to have some metadata whether TCONF is
safe (i.e. missing root) or something else.
Kind regards,
Petr
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-06-30 12:19 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-29 20:22 [LTP] ee Petr Vorel
2021-06-29 20:59 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
2021-06-30 7:11 ` [LTP] ee Richard Palethorpe
2021-06-30 8:01 ` Petr Vorel
2021-06-30 9:12 ` Richard Palethorpe
2021-06-30 10:10 ` [LTP] tst_fuzzy_sync01 sporadically fails Petr Vorel
2021-06-30 12:05 ` Richard Palethorpe
2021-06-30 12:19 ` Petr Vorel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.