RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Joakim Plate]

Hi, 

I'm currently trying to make use of OML_sync_control extension to schedule 
presentation of video frames in xbmc.

I've run into somewhat of a snag. It seem the spec doesn't specify what
time the UST clock really is, nor can i find any mention of it elsewhere
in docs.

Code wise it seem to be using do_gettimeofday(), which seems like a rather
poor choice given that it can jump forward and back in time due to
settimeofday calls. 

We normally make use of clock_gettime(CLOCK_MONOTONIC) to timestamp display
of video frames, so to avoid major changes I'd need a way to convert to 
gettimeofday (seem same as CLOCK_REALTIME).

Currently i'm trying:
  struct timespec mon, rel;
  clock_gettime(CLOCK_MONOTONIC, &mon);
  clock_gettime(CLOCK_REALTIME , &rel);

  ticks += (rel.tv_sec  - mon.tv_sec)  * 1000000000;
  ticks += (rel.tv_nsec - mon.tv_nsec);

To convert between the two, but that is quite a hack both in the
possibility of clock changes and scheduling errors.

Is there a better way, or perhaps the DRI code should use CLOCK_MONOTONIC
in the first place?

Regards
/Joakim

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Michel Dänzer]

On Son, 2012-06-10 at 11:56 +0000, Joakim Plate wrote: 
> Hi, 
> 
> I'm currently trying to make use of OML_sync_control extension to schedule 
> presentation of video frames in xbmc.
> 
> I've run into somewhat of a snag. It seem the spec doesn't specify what
> time the UST clock really is, nor can i find any mention of it elsewhere
> in docs.
> 
> Code wise it seem to be using do_gettimeofday(), which seems like a rather
> poor choice given that it can jump forward and back in time due to
> settimeofday calls. 
> 
> We normally make use of clock_gettime(CLOCK_MONOTONIC) to timestamp display
> of video frames, so to avoid major changes I'd need a way to convert to 
> gettimeofday (seem same as CLOCK_REALTIME).
> 
> Currently i'm trying:
>   struct timespec mon, rel;
>   clock_gettime(CLOCK_MONOTONIC, &mon);
>   clock_gettime(CLOCK_REALTIME , &rel);
> 
>   ticks += (rel.tv_sec  - mon.tv_sec)  * 1000000000;
>   ticks += (rel.tv_nsec - mon.tv_nsec);
> 
> To convert between the two, but that is quite a hack both in the
> possibility of clock changes and scheduling errors.
> 
> Is there a better way, or perhaps the DRI code should use CLOCK_MONOTONIC
> in the first place?

From the GLX_OML_sync_control spec:

        The Unadjusted System Time (or UST) is a 64-bit monotonically
        increasing counter [...]

So, without having thought a lot about potential ramifications, I'm
inclined to say that CLOCK_MONOTONIC should indeed be used.


-- 
Earthling Michel Dänzer           |                   http://www.amd.com
Libre software enthusiast         |          Debian, X and DRI developer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Ian Romanick]

On 06/11/2012 03:02 AM, Michel Dänzer wrote:
> On Son, 2012-06-10 at 11:56 +0000, Joakim Plate wrote:
>> Hi,
>>
>> I'm currently trying to make use of OML_sync_control extension to schedule
>> presentation of video frames in xbmc.
>>
>> I've run into somewhat of a snag. It seem the spec doesn't specify what
>> time the UST clock really is, nor can i find any mention of it elsewhere
>> in docs.
>>
>> Code wise it seem to be using do_gettimeofday(), which seems like a rather
>> poor choice given that it can jump forward and back in time due to
>> settimeofday calls.
>>
>> We normally make use of clock_gettime(CLOCK_MONOTONIC) to timestamp display
>> of video frames, so to avoid major changes I'd need a way to convert to
>> gettimeofday (seem same as CLOCK_REALTIME).
>>
>> Currently i'm trying:
>>    struct timespec mon, rel;
>>    clock_gettime(CLOCK_MONOTONIC,&mon);
>>    clock_gettime(CLOCK_REALTIME ,&rel);
>>
>>    ticks += (rel.tv_sec  - mon.tv_sec)  * 1000000000;
>>    ticks += (rel.tv_nsec - mon.tv_nsec);
>>
>> To convert between the two, but that is quite a hack both in the
>> possibility of clock changes and scheduling errors.
>>
>> Is there a better way, or perhaps the DRI code should use CLOCK_MONOTONIC
>> in the first place?
>
>  From the GLX_OML_sync_control spec:
>
>          The Unadjusted System Time (or UST) is a 64-bit monotonically
>          increasing counter [...]
>
> So, without having thought a lot about potential ramifications, I'm
> inclined to say that CLOCK_MONOTONIC should indeed be used.

Maybe.  Isn't CLOCK_MONOTONIC per-process?  UST is supposed to be 
relatively consistent across the system.  Some relevant was elided above:

     "The Unadjusted System Time (or UST) is a 64-bit monotonically
     increasing counter that is available throughout the system. UST
     is not a resource that is controlled by OpenGL, so it is not
     defined further as part of this extension."

do_gettimeofday is definitely the wrong choice, but it was the best 
thing that I knew to use back in 2002(ish) when I wrote that code.  I'm 
just not sure CLOCK_MONOTONIC is quite right either.

What's behind door #3? :)
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Michel Dänzer]

On Die, 2012-06-12 at 20:04 -0700, Ian Romanick wrote: 
> On 06/11/2012 03:02 AM, Michel Dänzer wrote:
> > On Son, 2012-06-10 at 11:56 +0000, Joakim Plate wrote:
> >> Hi,
> >>
> >> I'm currently trying to make use of OML_sync_control extension to schedule
> >> presentation of video frames in xbmc.
> >>
> >> I've run into somewhat of a snag. It seem the spec doesn't specify what
> >> time the UST clock really is, nor can i find any mention of it elsewhere
> >> in docs.
> >>
> >> Code wise it seem to be using do_gettimeofday(), which seems like a rather
> >> poor choice given that it can jump forward and back in time due to
> >> settimeofday calls.
> >>
> >> We normally make use of clock_gettime(CLOCK_MONOTONIC) to timestamp display
> >> of video frames, so to avoid major changes I'd need a way to convert to
> >> gettimeofday (seem same as CLOCK_REALTIME).
> >>
> >> Currently i'm trying:
> >>    struct timespec mon, rel;
> >>    clock_gettime(CLOCK_MONOTONIC,&mon);
> >>    clock_gettime(CLOCK_REALTIME ,&rel);
> >>
> >>    ticks += (rel.tv_sec  - mon.tv_sec)  * 1000000000;
> >>    ticks += (rel.tv_nsec - mon.tv_nsec);
> >>
> >> To convert between the two, but that is quite a hack both in the
> >> possibility of clock changes and scheduling errors.
> >>
> >> Is there a better way, or perhaps the DRI code should use CLOCK_MONOTONIC
> >> in the first place?
> >
> >  From the GLX_OML_sync_control spec:
> >
> >          The Unadjusted System Time (or UST) is a 64-bit monotonically
> >          increasing counter [...]
> >
> > So, without having thought a lot about potential ramifications, I'm
> > inclined to say that CLOCK_MONOTONIC should indeed be used.
> 
> Maybe.  Isn't CLOCK_MONOTONIC per-process?

According to the test code below, it's not here. While the clock_gettime
manpage doesn't explicitly state that it's system-wide, it does mention
it being affect by adjtime and NTP, which kind of implies it is?


#include <stdio.h>
#include <string.h>
#include <time.h>

int main(void)
{
  struct timespec tp;
  int ret;

  memset(&tp, 0, sizeof(tp));
  ret = clock_gettime(CLOCK_MONOTONIC, &tp);

  printf("clock_gettime(CLOCK_MONOTONIC) returned %d, timestamp %lu.%09lu\n",
	 ret, tp.tv_sec, tp.tv_nsec);

  return ret;
}


-- 
Earthling Michel Dänzer           |                   http://www.amd.com
Libre software enthusiast         |          Debian, X and DRI developer
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Julien Cristau]

On Tue, Jun 12, 2012 at 20:04:03 -0700, Ian Romanick wrote:

> Isn't CLOCK_MONOTONIC per-process?

"This clock represents the monotonic clock for the system."
(http://pubs.opengroup.org/onlinepubs/009604599/functions/clock_getres.html)
So, no.

Cheers,
Julien

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Joakim Plate]

Michel Dänzer <michel <at> daenzer.net> writes:

> > >
> > >  From the GLX_OML_sync_control spec:
> > >
> > >          The Unadjusted System Time (or UST) is a 64-bit monotonically
> > >          increasing counter [...]
> > >

From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000. Only 
issue is that changing it will break any app relying on it being REALTIME clock. 

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Jerome Glisse]

On Wed, Jun 13, 2012 at 4:11 PM, Joakim Plate <elupus@ecce.se> wrote:
> Michel Dänzer <michel <at> daenzer.net> writes:
>
>> > >
>> > >  From the GLX_OML_sync_control spec:
>> > >
>> > >          The Unadjusted System Time (or UST) is a 64-bit monotonically
>> > >          increasing counter [...]
>> > >
>
> From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000. Only
> issue is that changing it will break any app relying on it being REALTIME clock.
>

App that rely on it being anything special are badly broken and i
don't think there is any such app. The specification strongly stress
that app should make no assumption about it.

Cheers,
Jerome

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Joakim Plate]

> >
> > From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000. 
Only
> > issue is that changing it will break any app relying on it being REALTIME 
clock.
> >
> 
> App that rely on it being anything special are badly broken and i
> don't think there is any such app. The specification strongly stress
> that app should make no assumption about it.
> 

While that may be true... Since there is no other API for getting this UST 
clock, it's somewhat limited in use. Even if i know vsync happened at time X, if 
don't know what time it is "now" how can i make use of it?

Spec says: "The Unadjusted System Time (or UST)
    is a 64-bit monotonically increasing counter that is available
    throughout the system."

If across the system, the only API to get to this value is through GLX api, it's 
rather hard to make use of.

For example syncing audio to vsync. One need to sync audio output written to 
audio renderer now, with this clock.

Also regarding relying on current behavior... Even if this change is made now, 
there will be a lot of system with the old behaviour. So knowning if the change 
has been made in a system is crucial for supporting both / not enabling when 
feature is unreliable.

/Joakim

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Jerome Glisse]

On Thu, Jun 14, 2012 at 1:19 PM, Joakim Plate <elupus@ecce.se> wrote:
>
>> >
>> > From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000.
> Only
>> > issue is that changing it will break any app relying on it being REALTIME
> clock.
>> >
>>
>> App that rely on it being anything special are badly broken and i
>> don't think there is any such app. The specification strongly stress
>> that app should make no assumption about it.
>>
>
> While that may be true... Since there is no other API for getting this UST
> clock, it's somewhat limited in use. Even if i know vsync happened at time X, if
> don't know what time it is "now" how can i make use of it?
>
> Spec says: "The Unadjusted System Time (or UST)
>    is a 64-bit monotonically increasing counter that is available
>    throughout the system."
>
> If across the system, the only API to get to this value is through GLX api, it's
> rather hard to make use of.
>
> For example syncing audio to vsync. One need to sync audio output written to
> audio renderer now, with this clock.
>
> Also regarding relying on current behavior... Even if this change is made now,
> there will be a lot of system with the old behaviour. So knowning if the change
> has been made in a system is crucial for supporting both / not enabling when
> feature is unreliable.
>
> /Joakim
>

This extension is not for predicting when is the next vblank and it's
wrong to try to use it for that. My understanding is that you should
use this extension for instance to show N video frame per second,
based on the number of frame you play per second you can synchronize
sounds output.

Cheers,
Jerome

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Jerome Glisse]

On Thu, Jun 14, 2012 at 2:17 PM, Jerome Glisse <j.glisse@gmail.com> wrote:
> On Thu, Jun 14, 2012 at 1:19 PM, Joakim Plate <elupus@ecce.se> wrote:
>>
>>> >
>>> > From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000.
>> Only
>>> > issue is that changing it will break any app relying on it being REALTIME
>> clock.
>>> >
>>>
>>> App that rely on it being anything special are badly broken and i
>>> don't think there is any such app. The specification strongly stress
>>> that app should make no assumption about it.
>>>
>>
>> While that may be true... Since there is no other API for getting this UST
>> clock, it's somewhat limited in use. Even if i know vsync happened at time X, if
>> don't know what time it is "now" how can i make use of it?
>>
>> Spec says: "The Unadjusted System Time (or UST)
>>    is a 64-bit monotonically increasing counter that is available
>>    throughout the system."
>>
>> If across the system, the only API to get to this value is through GLX api, it's
>> rather hard to make use of.
>>
>> For example syncing audio to vsync. One need to sync audio output written to
>> audio renderer now, with this clock.
>>
>> Also regarding relying on current behavior... Even if this change is made now,
>> there will be a lot of system with the old behaviour. So knowning if the change
>> has been made in a system is crucial for supporting both / not enabling when
>> feature is unreliable.
>>
>> /Joakim
>>
>
> This extension is not for predicting when is the next vblank and it's
> wrong to try to use it for that. My understanding is that you should
> use this extension for instance to show N video frame per second,
> based on the number of frame you play per second you can synchronize
> sounds output.
>
> Cheers,
> Jerome

Note that if you really think that you need something like
GLX_OML_sync_control but with UST being some sensible value from the
system that you can query by other mean we could easily add a new mesa
extension GLX_MESA_sync_control that properly defines ust and keep
everything else the same. Then we can fix the kernel and expose this
extension only on fixed kernel. Of course this would probably only be
available on open source driver but with a bit of lobying nvidia & amd
might pick the extension in their closed source driver.

Cheers,
Jerome

Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC

[Mario Kleiner]

> 
> Message: 2
> Date: Thu, 14 Jun 2012 17:19:11 +0000 (UTC)
> From: Joakim Plate <elupus@ecce.se>
> Subject: Re: RFC: Change OML_sync_control UST to CLOCK_MONOTONIC
> To: dri-devel@lists.freedesktop.org
> Message-ID: <loom.20120614T191057-86@post.gmane.org>
> Content-Type: text/plain; charset=us-ascii
> 
> 
>>> 
>>> From what I can tell, it should be using: ktime_to_ns(ktime_get()) / 1000. 
> Only
>>> issue is that changing it will break any app relying on it being REALTIME 
> clock.
>>> 
>> 
>> App that rely on it being anything special are badly broken and i
>> don't think there is any such app. The specification strongly stress
>> that app should make no assumption about it.
>> 
> 
> While that may be true... Since there is no other API for getting this UST 
> clock, it's somewhat limited in use. Even if i know vsync happened at time X, if 
> don't know what time it is "now" how can i make use of it?
> 
> Spec says: "The Unadjusted System Time (or UST)
>    is a 64-bit monotonically increasing counter that is available
>    throughout the system."
> 
> If across the system, the only API to get to this value is through GLX api, it's 
> rather hard to make use of.
> 
> For example syncing audio to vsync. One need to sync audio output written to 
> audio renderer now, with this clock.
> 
> Also regarding relying on current behavior... Even if this change is made now, 
> there will be a lot of system with the old behaviour. So knowning if the change 
> has been made in a system is crucial for supporting both / not enabling when 
> feature is unreliable.
> 
> /Joakim
> 

According to the spec, CLOCK_MONOTONIC would have been the right choice.

In practice, as far as my experience with using it goes, it doesn't really matter much, as long as you don't manually set a new time for the system clock while such an application is running - something rather infrequent. Very small, slow clock changes, e.g., due to ntp adjustments, shouldn't matter much, because your app will probably not use a returned timestamp to schedule some action very far ahead in time, when error could accumulate due to multiple adjustments, but only to synchronize stuff on a short timescale.

In my toolkit, e.g., i use the OML_sync_control timestamps to correlate and/or synchronize onset of a visual stimulus with audio playback or audio capture, timestamps from keyboard/mouse/whatever... input, time stamped eye movement information from eye trackers, or reception or emission of digital trigger signals to control some research equipment. The whole thing is timing sensitive at the millisecond level, but usually we only use timestamps to schedule ahead a few video refresh cycles, or a few seconds at most, and typical ntp adjustments don't do harm. I'd think that a video player or similar app is a simpler special case when it comes to audio-video sync.

A change now would break apps that strictly relied on CLOCK_REALTIME semantic to synchronize with other modalities, which themselves may use CLOCK_REALTIME / gettimeofday(), although the OML_sync_control spec says that an app shouldn't rely on it. What i do in my software is this:

<https://github.com/kleinerm/Psychtoolbox-3/blob/master/PsychSourceGL/Source/Linux/Base/PsychTimeGlue.c#L263>

Because the offset between gettimeofday and CLOCK_MONOTONIC is something like 40 years, you can reliably find out by comparison if the returned timestamp is from CLOCK_REALTIME or CLOCK_MONOTONIC and just remap to whatever you find convenient. It's not beautiful, but it allows to do the right thing, should the timebase change at some Linux version. I also needed this for audio sync, because at some ALSA version, some (all?) sound drivers just switched from one timebase to the other.

Not sure if this adds anything new to the topic, i just thought i tell you my experience with using it so far.
-mario