[U-Boot] dm: rtc: random Gitlab results for dm_test_rtc_dual()

[U-Boot] dm: rtc: random Gitlab results for dm_test_rtc_dual()

[Heinrich Schuchardt]

Hello Simon,

the gitlab 'sandbox test.py' job seems to be kind of a lottery:

https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/25678

Job succeeded

https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/25499

/u-boot-efi/test/dm/rtc.c:167, dm_test_rtc_dual():
-EINVAL == cmp_times(&now1, &cmp, false):
Expected -22, got 0\r\r\nFailures: 1'.endswith

dm_test_rtc_dual() fails or succeeds for the same code.

@Tom:
Did you also see such spurious results?

At first sight I could not anything wrong in the dm_test_rtc_dual() test
assuming that the Gitlab environment provides a monotonic clock for the
time() function.

The findings below do not explain the problem:

now2 is read but never used. We are testing that we can get the time
from emul2 later on. So I think this line is not needed.

The offset of emul1 is not set before reading now1. If it were -2 (but I
would not know why it should get to that value), than setting offset + 1
for emul2 would be a NOP. So shouldn't we set the offset of emul1 to a
well defined value before reading now1?

Shouldn't dm_test_rtc_dual() reset the offset of emul2 to 0 before
returning?

Best regards

Heinrich

dm: rtc: random Gitlab results for dm_test_rtc_dual()

[Simon Glass]

Hi Heinrich,

On Wed, 30 Oct 2019 at 22:10, Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Hello Simon,
>
> the gitlab 'sandbox test.py' job seems to be kind of a lottery:
>
> https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/25678
>
> Job succeeded
>
> https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/25499
>
> /u-boot-efi/test/dm/rtc.c:167, dm_test_rtc_dual():
> -EINVAL == cmp_times(&now1, &cmp, false):
> Expected -22, got 0\r\r\nFailures: 1'.endswith
>
> dm_test_rtc_dual() fails or succeeds for the same code.
>
> @Tom:
> Did you also see such spurious results?
>
> At first sight I could not anything wrong in the dm_test_rtc_dual() test
> assuming that the Gitlab environment provides a monotonic clock for the
> time() function.
>
> The findings below do not explain the problem:
>
> now2 is read but never used. We are testing that we can get the time
> from emul2 later on. So I think this line is not needed.
>
> The offset of emul1 is not set before reading now1. If it were -2 (but I
> would not know why it should get to that value), than setting offset + 1
> for emul2 would be a NOP. So shouldn't we set the offset of emul1 to a
> well defined value before reading now1?
>
> Shouldn't dm_test_rtc_dual() reset the offset of emul2 to 0 before
> returning?

I have never seen this problem.

But I notice that the timers are probed one after the other. Although
use_system_time is changed to false (for testing purposes) that still
retains the base_time value which comes from os_localtime(). So the
two timers can have a different time value if the next second ticks
over between the two uclass_get_device() calls and I wonder if the
difference is exactly 1 then you will see the error you mention.

I suspect instead that we need a way to set the actually base_time of
a timer. Then it can be set to the same value for each, at the start
of the test.

Regards,
Simon