[Xenomai-help] Question about getting system time

[Xenomai-help] Question about getting system time

[Abhijit Majumdar]

[-- Attachment #1: Type: text/plain, Size: 923 bytes --]

Hi folks

 

I am facing a strange problem in getting system time.

 

My requirement:

 

 In a hard real time task I need to get the current system time and
store it in a structure. Later at some point of time a java application
a windows system is supposed to convert it into user-readable date-time
format. I am using rt_timer_tsc2ns(rt_timer_read()) to get the current
time. 

 

Problem: 

 

However the java application prints a date in 1980 (I converted nanosec
to millisec before using java API). I kind of expected rt_timer_tsc2ns
to return nanoseconds from 1st Jan 1970 because it is common for
programming tools and languages, although I must admit that I did not
see that in the xenomai API documentation.

 

Am I doing something wrong? Should I have to compile the kernel with
some specific parameter?

 

Any help will be appreciated.

 

thanks

Abhijit

 


[-- Attachment #2: Type: text/html, Size: 4770 bytes --]

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Abhijit Majumdar wrote:
> Hi folks
> 
>  
> 
> I am facing a strange problem in getting system time.
> 
>  
> 
> My requirement:
> 
>  
> 
>  In a hard real time task I need to get the current system time and
> store it in a structure. Later at some point of time a java application
> a windows system is supposed to convert it into user-readable date-time
> format. I am using rt_timer_tsc2ns(rt_timer_read()) to get the current
> time. 

You should be using rt_timer_ticks2ns(rt_timer_read()), rt_timer_tsc2ns
should be used with rt_timer_tsc().


> 
>  
> 
> Problem: 
> 
>  
> 
> However the java application prints a date in 1980 (I converted nanosec
> to millisec before using java API). I kind of expected rt_timer_tsc2ns
> to return nanoseconds from 1st Jan 1970 because it is common for
> programming tools and languages, although I must admit that I did not
> see that in the xenomai API documentation.
> 
>  
> 
> Am I doing something wrong? Should I have to compile the kernel with
> some specific parameter?

The problem you may have is that Xenomai clock is synchronized with
Linux clock when Xenomai starts, and it may happen at a time when Linux
clock is not yet set to the current time. If that is the case, you
should set xenomai's time during the boot process after Linux' time has
been set. Currently, the only service to do this is clock_settime,
implemented by the posix skin.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Andreas Glatz]

Hi,

We had a similar problem. We solved it by compiling the native skin as a module and loading it _after_ the kernel has been syncronized with the real-time clock chip.

Andreas


________________________________________
From: xenomai-help-bounces@domain.hid [xenomai-help-bounces@domain.hid] On Behalf Of Abhijit Majumdar [abhijit.majumdar@domain.hid]
Sent: Thursday, May 13, 2010 6:14 PM
To: xenomai@xenomai.org
Subject: [Xenomai-help] Question about getting system time

Hi folks

I am facing a strange problem in getting system time.

My requirement:

 In a hard real time task I need to get the current system time and store it in a structure. Later at some point of time a java application a windows system is supposed to convert it into user-readable date-time format. I am using rt_timer_tsc2ns(rt_timer_read()) to get the current time.

Problem:

However the java application prints a date in 1980 (I converted nanosec to millisec before using java API). I kind of expected rt_timer_tsc2ns to return nanoseconds from 1st Jan 1970 because it is common for programming tools and languages, although I must admit that I did not see that in the xenomai API documentation.

Am I doing something wrong? Should I have to compile the kernel with some specific parameter?

Any help will be appreciated.

thanks
Abhijit

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Andreas Glatz wrote:
> Hi,
> 
> We had a similar problem. We solved it by compiling the native skin
> as a module and loading it _after_ the kernel has been syncronized
> with the real-time clock chip.

Yes, that works too. There is also an option in the kernel for the
kernel to synchronize with the RTC clock directly, without requiring
hwclock. I do not know if it is run before starting Xenomai though.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Steve Deiters]

> > Hi,
> > 
> > We had a similar problem. We solved it by compiling the 
> native skin as 
> > a module and loading it _after_ the kernel has been 
> syncronized with 
> > the real-time clock chip.
> 
> Yes, that works too. There is also an option in the kernel 
> for the kernel to synchronize with the RTC clock directly, 
> without requiring hwclock. I do not know if it is run before 
> starting Xenomai though.

I have a similar requirement, except I want to keep the Xenomai clock
synchronized with Linux services that use adjtime/adjtimex, e.g. NTP.
It is not sufficient in this case to delay the startup of Xenomai.  I
suppose as a workaround I could create a low priority periodic thread
that reads the Linux clock with gettimeofday and sets the Xenomai one
with clock_settime.

Any other suggestions for providing timestamps to real time tasks in
this case?

Thanks.

Re: [Xenomai-help] Question about getting system time

[Jan Kiszka]

Steve Deiters wrote:
>>> Hi,
>>>
>>> We had a similar problem. We solved it by compiling the 
>> native skin as 
>>> a module and loading it _after_ the kernel has been 
>> syncronized with 
>>> the real-time clock chip.
>> Yes, that works too. There is also an option in the kernel 
>> for the kernel to synchronize with the RTC clock directly, 
>> without requiring hwclock. I do not know if it is run before 
>> starting Xenomai though.
> 
> I have a similar requirement, except I want to keep the Xenomai clock
> synchronized with Linux services that use adjtime/adjtimex, e.g. NTP.
> It is not sufficient in this case to delay the startup of Xenomai.  I
> suppose as a workaround I could create a low priority periodic thread
> that reads the Linux clock with gettimeofday and sets the Xenomai one
> with clock_settime.

Periodically setting the time is risky if timed jobs depend on Xenomai's
real-time clock - it may jump in all directions...

> 
> Any other suggestions for providing timestamps to real time tasks in
> this case?

Do you just need precise timestamps from with real-time tasks, or do you
have to synchronize timer events of the Xenomai core on an external clock?

For the former case (precisely our scenario), we laid the ground to
extend Xenomai 2.5 with RT-safe syscalls to obtain Linux's view on
gettimeofday. It "just" needs some polishing to post this for upstream.
Wolfgang (CC'ed) is working on this.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-help] Question about getting system time

[Steve Deiters]

> Periodically setting the time is risky if timed jobs depend 
> on Xenomai's real-time clock - it may jump in all directions...
> 
> > 
> > Any other suggestions for providing timestamps to real time 
> tasks in 
> > this case?
> 
> Do you just need precise timestamps from with real-time 
> tasks, or do you have to synchronize timer events of the 
> Xenomai core on an external clock?
> 
> For the former case (precisely our scenario), we laid the 
> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain 
> Linux's view on gettimeofday. It "just" needs some polishing 
> to post this for upstream.
> Wolfgang (CC'ed) is working on this.

I'm just looking to get timestamps in the real time task.  At least in
my case being able to call gettimeofday from the real time thread would
be exactly what I need.

By the way, calling gettimeofday currently within a real time thread
seems to occasionally freeze up my whole system.  I was going to make
another post for this after I got a chance to verify some details.  Has
anyone noticed anything similar to this?

Re: [Xenomai-help] Question about getting system time

[Jan Kiszka]

Steve Deiters wrote:
>> Periodically setting the time is risky if timed jobs depend 
>> on Xenomai's real-time clock - it may jump in all directions...
>>
>>> Any other suggestions for providing timestamps to real time 
>> tasks in 
>>> this case?
>> Do you just need precise timestamps from with real-time 
>> tasks, or do you have to synchronize timer events of the 
>> Xenomai core on an external clock?
>>
>> For the former case (precisely our scenario), we laid the 
>> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain 
>> Linux's view on gettimeofday. It "just" needs some polishing 
>> to post this for upstream.
>> Wolfgang (CC'ed) is working on this.
> 
> I'm just looking to get timestamps in the real time task.  At least in
> my case being able to call gettimeofday from the real time thread would
> be exactly what I need.
> 
> By the way, calling gettimeofday currently within a real time thread
> seems to occasionally freeze up my whole system.  I was going to make
> another post for this after I got a chance to verify some details.  Has
> anyone noticed anything similar to this?
> 

Yes, that's expected: If gettimeofday runs syscall-less, it tries to
read the time offset from a page which the Linux kernel updates
regularly. When user-space detects an ongoing update, it spins until
that has completed. But as Xenomai runs the task with higher priority
than the Linux update handler, you just ran into a live-lock.

Switching on the Xenomai watchdog will confirm this: It will shoot that
task, and the system will recover.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-help] Question about getting system time

[Josh Karch]

Jan,  

If no ntp server updates are made to the Linux clock,  we have noticed over a week's period a drift of ten or so seconds difference between the Linux clock and the Xenomai get time function and often have to reboot our machine to resynchronize the Xenomai and Linux clocks back to less than a couple seconds difference.  I was wondering if you might explain why that might happen?

SIncerely,

Joshua Karch


________________________________________
From: xenomai-help-bounces@domain.hid [xenomai-help-bounces@domain.hid] On Behalf Of Jan Kiszka [jan.kiszka@domain.hid]
Sent: Monday, May 17, 2010 10:13 AM
To: Steve Deiters
Cc: xenomai@xenomai.org; Mauerer,      Wolfgang; Andreas Glatz
Subject: Re: [Xenomai-help] Question about getting system time

Steve Deiters wrote:
>> Periodically setting the time is risky if timed jobs depend
>> on Xenomai's real-time clock - it may jump in all directions...
>>
>>> Any other suggestions for providing timestamps to real time
>> tasks in
>>> this case?
>> Do you just need precise timestamps from with real-time
>> tasks, or do you have to synchronize timer events of the
>> Xenomai core on an external clock?
>>
>> For the former case (precisely our scenario), we laid the
>> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain
>> Linux's view on gettimeofday. It "just" needs some polishing
>> to post this for upstream.
>> Wolfgang (CC'ed) is working on this.
>
> I'm just looking to get timestamps in the real time task.  At least in
> my case being able to call gettimeofday from the real time thread would
> be exactly what I need.
>
> By the way, calling gettimeofday currently within a real time thread
> seems to occasionally freeze up my whole system.  I was going to make
> another post for this after I got a chance to verify some details.  Has
> anyone noticed anything similar to this?
>

Yes, that's expected: If gettimeofday runs syscall-less, it tries to
read the time offset from a page which the Linux kernel updates
regularly. When user-space detects an ongoing update, it spins until
that has completed. But as Xenomai runs the task with higher priority
than the Linux update handler, you just ran into a live-lock.

Switching on the Xenomai watchdog will confirm this: It will shoot that
task, and the system will recover.

Jan

--
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

_______________________________________________
Xenomai-help mailing list
Xenomai-help@domain.hid
https://mail.gna.org/listinfo/xenomai-help

Re: [Xenomai-help] Question about getting system time

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

[please avoid top posting]

Josh Karch wrote:
> Jan,  
> 
> If no ntp server updates are made to the Linux clock,  we have noticed over a week's period a drift of ten or so seconds difference between the Linux clock and the Xenomai get time function and often have to reboot our machine to resynchronize the Xenomai and Linux clocks back to less than a couple seconds difference.  I was wondering if you might explain why that might happen?

I assume you are talking about an x86 platform: Normally, Linux should
run on the TSC clocksource, just like Xenomai does. To ensure that the
hardware isn't broken, Linux continuously performs runtime checks. The
preemption by Xenomai tasks my fool this check an make Linux switch to a
different clocksource. Now you have unsynchronized sources that usually
drift.

I haven't tried this with Xenomai yet but I could imagine it works just
like it does in virtual machines: Pass tsc=reliable on the kernel
command line. This disables the runtime checks, thus should take away
any reason Linux may want to switch to anything else than TSC.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] Question about getting system time

[Thomas Lockhart]

>  If no ntp server updates are made to the Linux clock,  we have
>  noticed over a week's period a drift of ten or so seconds difference
>  between the Linux clock and the Xenomai get time function and often
>  have to reboot our machine to resynchronize the Xenomai and Linux
>  clocks back to less than a couple seconds difference.  I was
>  wondering if you might explain why that might happen?

Your clock oscillator on the x86 PC is drifting wrt NTP time (the 
oscillators are known to be bad time sources). We see the same effect 
here. With NTP running, try running a RT periodic thread and obtain both 
Linux system time and Xenomai RT time. You will see a sawtooth time 
difference reaching around 1msec in magnitude before NTP readjusts Linux 
system time back down to near zero. Then the clocks drift wrt one 
another until you have around a 1msec difference, and NTP adjusts the 
time again. I recall a periodicity of around 30-60sec per adjustment 
cycle (5-10 seconds per week) but it is dependent on the characteristics 
of your specific oscillator on the motherboard.

I have thought to cope with this by having a rate parameter on the code 
encapsulating my timing source (I have "drivers" for Linux system time, 
Xenomai time, and hardware timing sources). The NTP rate in the drift 
file is pretty constant, so one could presumably use that rate to adjust 
your timers.

hth

                                      - Tom

Re: [Xenomai-help] Question about getting system time

[Steve Deiters]

> Yes, that's expected: If gettimeofday runs syscall-less, it 
> tries to read the time offset from a page which the Linux 
> kernel updates regularly. When user-space detects an ongoing 
> update, it spins until that has completed. But as Xenomai 
> runs the task with higher priority than the Linux update 
> handler, you just ran into a live-lock.
> 
> Switching on the Xenomai watchdog will confirm this: It will 
> shoot that task, and the system will recover.
> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT T DE IT 1 Corporate 
> Competence Center Embedded Linux
> 

I'm not all too familiar with the time handling in the kernel.  If you
have any refernces that help me better follow your explanation that
would be appreciated.

If you linked against the POSIX skin it would then seem there is no safe
way of calling gettimeofday.  Even starting from main in this case is a
real time task.  In this case I would think that gettimeofday should be
either wrapped to force a mode change before calling
__real_gettimeofday, or should somehow be expressly forbidden.

Even more of a problem is how I am supposed to know which calls are
similar to this.  I would expect I could call any Linux service without
causing a deadlock/livelock situation, although in a nondeterministic
manner.  If I understood your explanation correctly, this is hanging
whenever the Xenomai task tries to spin on the same lock as the kernel.
However, I don't see how this is not a problem in general for acquiring
any spinlock from a Xenomai task.  I would expect once you introduce a
priority based scheduler any spinlock becomes unsafe.

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Steve Deiters wrote:
>> Yes, that's expected: If gettimeofday runs syscall-less, it 
>> tries to read the time offset from a page which the Linux 
>> kernel updates regularly. When user-space detects an ongoing 
>> update, it spins until that has completed. But as Xenomai 
>> runs the task with higher priority than the Linux update 
>> handler, you just ran into a live-lock.
>>
>> Switching on the Xenomai watchdog will confirm this: It will 
>> shoot that task, and the system will recover.
>>
>> Jan
>>
>> --
>> Siemens AG, Corporate Technology, CT T DE IT 1 Corporate 
>> Competence Center Embedded Linux
>>
> 
> I'm not all too familiar with the time handling in the kernel.  If you
> have any refernces that help me better follow your explanation that
> would be appreciated.
> 
> If you linked against the POSIX skin it would then seem there is no safe
> way of calling gettimeofday.  Even starting from main in this case is a
> real time task.  In this case I would think that gettimeofday should be
> either wrapped to force a mode change before calling
> __real_gettimeofday, or should somehow be expressly forbidden.


The reason why I did not wrap gettimeofday is that we already have a
wrapped clock_gettime, which is the newer service supposed to be used by
real-time applications. If you are not porting a non real-time
application, there is no reason to use gettimeofday, and if you are,
this guide:
http://www.xenomai.org/index.php/Porting_POSIX_applications_to_Xenomai#Timing_services.

Should explain you what you need to do.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Steve Deiters wrote:
> Even more of a problem is how I am supposed to know which calls are
> similar to this.  I would expect I could call any Linux service without
> causing a deadlock/livelock situation, although in a nondeterministic
> manner.  If I understood your explanation correctly, this is hanging
> whenever the Xenomai task tries to spin on the same lock as the kernel.
> However, I don't see how this is not a problem in general for acquiring
> any spinlock from a Xenomai task.  I would expect once you introduce a
> priority based scheduler any spinlock becomes unsafe.

The set of services which you can call from a real-time thread without
loosing determinism are those found in the Xenomai posix skin documentation:
http://www.xenomai.org/documentation/xenomai-2.5/html/api/group__posix.html

Other than that, any service emitting a syscall causes a switch to
secondary mode. What remains are services not emitting syscalls, and for
these ones, all bets are off.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Steve Deiters]

> The set of services which you can call from a real-time 
> thread without loosing determinism are those found in the 
> Xenomai posix skin documentation:
> http://www.xenomai.org/documentation/xenomai-2.5/html/api/grou
> p__posix.html
> 
> Other than that, any service emitting a syscall causes a 
> switch to secondary mode. What remains are services not 
> emitting syscalls, and for these ones, all bets are off.
> 
> -- 
> 					    Gilles.
> 

I guess that is my point though.  Not being all too familiar with the
inner workings of Linux or glibc, how am I supposed to know which
services are not emitting a syscall?  I accept that I will lose
determinism in calling these services, but I find it odd that it may
allow the system to deadlock.

If nothing else the syscall-less services should be listed and noted as
unsafe.  I am new to most of this, but I have went through most of the
documentation and, unless I have overlooked it, found nothing mentioning
this.

Re: [Xenomai-help] Question about getting system time

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]

Steve Deiters wrote:
>> The set of services which you can call from a real-time 
>> thread without loosing determinism are those found in the 
>> Xenomai posix skin documentation:
>> http://www.xenomai.org/documentation/xenomai-2.5/html/api/grou
>> p__posix.html
>>
>> Other than that, any service emitting a syscall causes a 
>> switch to secondary mode. What remains are services not 
>> emitting syscalls, and for these ones, all bets are off.
>>
>> -- 
>> 					    Gilles.
>>
> 
> I guess that is my point though.  Not being all too familiar with the
> inner workings of Linux or glibc, how am I supposed to know which
> services are not emitting a syscall?  I accept that I will lose
> determinism in calling these services, but I find it odd that it may
> allow the system to deadlock.
> 
> If nothing else the syscall-less services should be listed and noted as
> unsafe.  I am new to most of this, but I have went through most of the
> documentation and, unless I have overlooked it, found nothing mentioning
> this.

Key services that do not reliably trigger syscalls:

 - gettimeofday
 - clock_gettime (unless the POSIX skin is in use)
 - malloc
 - free

There are no guarantees either, but this list turned out to be fairly
complete while porting a large (x86) application from plain Linux to
Xenomai. Only gettimeofday (and, conditionally, clock_gettime) suffer
from the live-lock problem.

For runtime checks, enable T_WARNSW / PTHREAD_WARNSW for your RT
threads, additionally link against librtdk, --wrap the above services,
and assert_nrt() will trigger a SIGDEBUG when they are used from a RT
thread (just like normal syscalls will do).

There should be a few more services in the glibc that trigger malloc
internally without issuing a syscall. When you come across one (check
the man pages), additions to xenomai/src/rtdk/assert_context.c are welcome.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Steve Deiters wrote:
>> I guess that is my point though.  Not being all too familiar with the
>> inner workings of Linux or glibc, how am I supposed to know which
>> services are not emitting a syscall?  I accept that I will lose
>> determinism in calling these services, but I find it odd that it may
>> allow the system to deadlock.
>>
>> If nothing else the syscall-less services should be listed and noted as
>> unsafe.  I am new to most of this, but I have went through most of the
>> documentation and, unless I have overlooked it, found nothing mentioning
>> this.
> 
> Key services that do not reliably trigger syscalls:
> 
>  - gettimeofday
>  - clock_gettime (unless the POSIX skin is in use)
>  - malloc
>  - free
> 
> There are no guarantees either, but this list turned out to be fairly
> complete while porting a large (x86) application from plain Linux to
> Xenomai. Only gettimeofday (and, conditionally, clock_gettime) suffer
> from the live-lock problem.
> 
> For runtime checks, enable T_WARNSW / PTHREAD_WARNSW for your RT
> threads, additionally link against librtdk, --wrap the above services,
> and assert_nrt() will trigger a SIGDEBUG when they are used from a RT
> thread (just like normal syscalls will do).
> 
> There should be a few more services in the glibc that trigger malloc
> internally without issuing a syscall. When you come across one (check
> the man pages), additions to xenomai/src/rtdk/assert_context.c are welcome.

Should not we make things more automatic? Integrate rtdk into libxenomai
and forcibly do the wrapping? I am also thinking about modifying
rt_print to make it possible to use it as a drop-in replacement of
printf, and wrap it when using the posix skin.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Wolfgang Mauerer]

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

Steve Deiters wrote:
>> Periodically setting the time is risky if timed jobs depend 
>> on Xenomai's real-time clock - it may jump in all directions...
>>
>>> Any other suggestions for providing timestamps to real time 
>> tasks in 
>>> this case?
>> Do you just need precise timestamps from with real-time 
>> tasks, or do you have to synchronize timer events of the 
>> Xenomai core on an external clock?
>>
>> For the former case (precisely our scenario), we laid the 
>> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain 
>> Linux's view on gettimeofday. It "just" needs some polishing 
>> to post this for upstream.
>> Wolfgang (CC'ed) is working on this.
> 
> I'm just looking to get timestamps in the real time task.  At least in
> my case being able to call gettimeofday from the real time thread would
> be exactly what I need.

then you might want to try the attached patches for Ipipe and
Xenomai. For upstream submission, they still need a bit of
cleaning up as Jan mentioned, and I'll also prepare a proper
series then. Testing is very welcome, however.

Best, Wolfgang

[-- Attachment #2: 01-generic-gtod-base.diff --]
[-- Type: text/x-patch, Size: 5206 bytes --]

Pass generic time-of-day information from Linux to higher domains

From: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>

Introduce a mechanism to pass all information
required to implement a gettimeofday() call with
NTP corrections as delivered by the Linux domain
to other domains. Essentially, this is an equivalent
of the Linux vsyscall to perform this very action.

We need to ensure that updates of the timing information are
atomic wrt. non-Linux domains. While a simple solution would be to place
an appropriate lock around the updates, this would eliminate
the speed advantages of the kernel's seqlock. Therefore,
we implement a transactional mechanism using two copies of the
timing information that does not interfere with the updates
of the kernel itself.

The timing information is only updated from a single CPU
in the Linux domain, and updates preempted by Xenomai are
protected by the transactional mechanism. However, when a
reader on another CPU runs in parallel with an update on
CPU 0, an inconsistent state can arise. This is mended by
using a sequence counter to ensure that the data structure
was not modified during the read.

Note that since updating the structure takes a constant amount
of time that is much smaller than the interval between updates,
the number of iterations in the retry loop (in case of
modifications during the read) is bounded:

TODO: Explain why this is the case
---
 include/linux/clocksource.h   |    8 ++++++++
 include/linux/ipipe_tickdev.h |   20 ++++++++++++++++++++
 kernel/ipipe/core.c           |   36 ++++++++++++++++++++++++++++++++++++
 3 files changed, 64 insertions(+), 0 deletions(-)

diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 64b1a4c..5ac07a7 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -281,12 +281,20 @@ extern void clocksource_resume(void);
 extern struct clocksource * __init __weak clocksource_default_clock(void);
 extern void clocksource_mark_unstable(struct clocksource *cs);
 
+/* Including ipipe_tickdev.h that declares update_ipipe_gtod()
+   would lead to a cycle, so we declare update_ipipe_gtod() here */
+void update_ipipe_gtod(struct timespec *ts, struct clocksource *c);
+
 #ifdef CONFIG_GENERIC_TIME_VSYSCALL
 extern void update_vsyscall(struct timespec *ts, struct clocksource *c);
 extern void update_vsyscall_tz(void);
 #else
+
 static inline void update_vsyscall(struct timespec *ts, struct clocksource *c)
 {
+#ifdef CONFIG_IPIPE
+	update_ipipe_gtod(ts, c);
+#endif
 }
 
 static inline void update_vsyscall_tz(void)
diff --git a/include/linux/ipipe_tickdev.h b/include/linux/ipipe_tickdev.h
index 4a1cb1b..68d9100 100644
--- a/include/linux/ipipe_tickdev.h
+++ b/include/linux/ipipe_tickdev.h
@@ -25,6 +25,7 @@
 #if defined(CONFIG_IPIPE) && defined(CONFIG_GENERIC_CLOCKEVENTS)
 
 #include <linux/clockchips.h>
+#include <linux/clocksource.h>
 
 struct tick_device;
 
@@ -44,6 +45,25 @@ struct ipipe_tick_device {
 	int real_shift;
 };
 
+struct ipipe_gtod_data {
+	seqcount_t lock;
+	cycle_t (*vread)(void);
+	time_t wall_time_sec;
+	u32 wall_time_nsec;
+	struct timespec wall_to_monotonic;
+	cycle_t cycle_last;
+	cycle_t mask;
+	u32 mult;
+	u32 shift;
+};
+
+struct gtod_data_exchange {
+	int gtod_active;
+	struct ipipe_gtod_data gtod_data[2];
+};
+
+void ipipe_set_gtod_data_exchange(struct gtod_data_exchange *exchg);
+
 int ipipe_request_tickdev(const char *devname,
 			  void (*emumode)(enum clock_event_mode mode,
 					  struct clock_event_device *cdev),
diff --git a/kernel/ipipe/core.c b/kernel/ipipe/core.c
index 63deaf9..53d9654 100644
--- a/kernel/ipipe/core.c
+++ b/kernel/ipipe/core.c
@@ -48,6 +48,12 @@ static unsigned long __ipipe_domain_slot_map;
 
 struct ipipe_domain ipipe_root;
 
+static struct gtod_data_exchange *gtod_data_exchange = NULL;
+
+void ipipe_set_gtod_data_exchange(struct gtod_data_exchange *exchg) {
+	gtod_data_exchange = exchg;
+}
+
 #ifndef CONFIG_SMP
 /*
  * Create an alias to the unique root status, so that arch-dep code
@@ -203,6 +209,36 @@ void ipipe_release_tickdev(int cpu)
 	ipipe_critical_exit(flags);
 }
 
+#define UPDATE_GTOD_DATA(base)				\
+	base.vread = clock->vread;			\
+	base.cycle_last = clock->cycle_last;		\
+	base.mask = clock->mask;			\
+	base.mult = clock->mult;			\
+	base.shift = clock->shift;			\
+	base.wall_time_sec = wall_time->tv_sec;		\
+	base.wall_time_nsec = wall_time->tv_nsec;	\
+	base.wall_to_monotonic = wall_to_monotonic;
+
+void update_ipipe_gtod(struct timespec *wall_time, struct clocksource *clock)
+{
+	if (!gtod_data_exchange)
+		return;
+
+	/* Activate second instance, update first */
+	gtod_data_exchange->gtod_active = 1;
+
+	write_seqcount_begin(&gtod_data_exchange->gtod_data[0].lock);
+	UPDATE_GTOD_DATA(gtod_data_exchange->gtod_data[0]);
+	write_seqcount_end(&gtod_data_exchange->gtod_data[0].lock);
+
+	/* Activate first instance, update second */
+	gtod_data_exchange->gtod_active = 0;
+
+	write_seqcount_begin(&gtod_data_exchange->gtod_data[1].lock);
+	UPDATE_GTOD_DATA(gtod_data_exchange->gtod_data[1]);
+	write_seqcount_end(&gtod_data_exchange->gtod_data[1].lock);
+};
+
 #endif /* CONFIG_GENERIC_CLOCKEVENTS */
 
 void __init ipipe_init_early(void)

[-- Attachment #3: 02-x86-gtod-part.diff --]
[-- Type: text/x-patch, Size: 1121 bytes --]

ipipe-gtod: x86-specific part

From: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>

x86 specific mechanism to make gtod information available
to non-Linux domains.

Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>
---
 arch/x86/kernel/vsyscall_64.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/x86/kernel/vsyscall_64.c b/arch/x86/kernel/vsyscall_64.c
index 8cb4974..565a5ba 100644
--- a/arch/x86/kernel/vsyscall_64.c
+++ b/arch/x86/kernel/vsyscall_64.c
@@ -32,6 +32,7 @@
 #include <linux/cpu.h>
 #include <linux/smp.h>
 #include <linux/notifier.h>
+#include <linux/ipipe_tickdev.h>
 
 #include <asm/vsyscall.h>
 #include <asm/pgtable.h>
@@ -89,6 +90,10 @@ void update_vsyscall(struct timespec *wall_time, struct clocksource *clock)
 	vsyscall_gtod_data.wall_to_monotonic = wall_to_monotonic;
 	vsyscall_gtod_data.wall_time_coarse = __current_kernel_time();
 	write_sequnlock_irqrestore(&vsyscall_gtod_data.lock, flags);
+
+#ifdef CONFIG_IPIPE
+	update_ipipe_gtod(wall_time, clock);
+#endif
 }
 
 /* RED-PEN may want to readd seq locking, but then the variable should be

[-- Attachment #4: series-ipipe --]
[-- Type: text/plain, Size: 125 bytes --]

# This series applies on GIT commit 2b9142396e28b8c3d0e6e7d758924f5f5ce81045
01-generic-gtod-base.diff
02-x86-gtod-part.diff

[-- Attachment #5: 01-add-gtod-bits-to-nkvdso.diff --]
[-- Type: text/x-patch, Size: 3317 bytes --]

Add gtod bits to nkvdso

From: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>


---
 include/nucleus/vdso.h |   53 +++++++++++++++++++++++++++++++++++++++++++++---
 ksrc/nucleus/shadow.c  |    8 +++++++
 2 files changed, 58 insertions(+), 3 deletions(-)

diff --git a/include/nucleus/vdso.h b/include/nucleus/vdso.h
index c431f88..ab67a3f 100644
--- a/include/nucleus/vdso.h
+++ b/include/nucleus/vdso.h
@@ -23,7 +23,42 @@
  * 02111-1307, USA.
  */
 
+#ifdef __KERNEL__
+#include <linux/time.h>
+#include <linux/seqlock.h>
+#include <linux/ipipe_tickdev.h>
+#define testbits(flags,mask) ((flags) & (mask))
+#else /* !__KERNEL__ */
+#include <time.h>
+#include <sys/types.h>
+#include <nucleus/seqlock_user.h>
 #include <nucleus/types.h>
+typedef u_int32_t u32;
+typedef u_int64_t cycle_t;
+
+/*
+ * ipipe_gtod_data and gtod_data_exchange must be kept in sync with
+ * the corresponding definitions in the ipipe: ipipe_tickdev.h
+ * We cannot include the file here because the definitios are also
+ * required in xenomai userland.
+ */
+struct ipipe_gtod_data {
+	seqcount_t lock;
+	cycle_t (*vread)(void);
+	time_t wall_time_sec;
+	u32 wall_time_nsec;
+	struct timespec wall_to_monotonic;
+	cycle_t cycle_last;
+	cycle_t mask;
+	u32 mult;
+	u32 shift;
+};
+
+struct gtod_data_exchange {
+	int gtod_active;
+	struct ipipe_gtod_data gtod_data[2];
+};
+#endif /* __KERNEL__ */
 
 /*
  * Data shared between Xenomai kernel/userland and the Linux kernel/userland
@@ -33,8 +68,11 @@
 struct xnvdso {
 	unsigned long long features;
 
-	/* Embed domain specific structures that describe the
-	 * shared data here */
+	struct gtod_data_exchange gtod_data_exchange;
+	/*
+	 * Embed further domain specific structures that
+	 * describe the shared data here
+	 */
 };
 
 /*
@@ -48,7 +86,8 @@ struct xnvdso {
 #define XNVDSO_FEATURES	(XNVDSO_FEAT_A | XNVDSO_FEAT_B | XVDSO_FEAT_C)
 */
 #define XNVDSO_FEAT_DROP_U_MODE 0x0000000000000001ULL
-#define XNVDSO_FEATURES (XNVDSO_FEAT_DROP_U_MODE)
+#define XNVDSO_FEAT_GTOD	0x0000000000000002ULL
+#define XNVDSO_FEATURES (XNVDSO_FEAT_DROP_U_MODE | XNVDSO_FEAT_GTOD)
 
 extern struct xnvdso *nkvdso;
 
@@ -57,5 +96,13 @@ static inline int xnvdso_test_feature(unsigned long long feature)
 	return testbits(nkvdso->features, feature);
 }
 
+static inline struct ipipe_gtod_data *get_gtod_data(void)
+{
+	// TODO: Is this safe, especially because of the array index operation?
+	struct gtod_data_exchange *exchg = &nkvdso->gtod_data_exchange;
+
+	return &exchg->gtod_data[exchg->gtod_active];
+}
+
 extern void xnheap_init_vdso(void);
 #endif /* _XENO_NUCLEUS_VDSO_H */
diff --git a/ksrc/nucleus/shadow.c b/ksrc/nucleus/shadow.c
index 872c37f..7db740d 100644
--- a/ksrc/nucleus/shadow.c
+++ b/ksrc/nucleus/shadow.c
@@ -713,6 +713,14 @@ void __init xnheap_init_vdso(void)
 		xnpod_fatal("Xenomai: cannot allocate memory for xnvdso!\n");
 
 	nkvdso->features = XNVDSO_FEATURES;
+
+	nkvdso->gtod_data_exchange.gtod_active = 0;
+	seqcount_init(&nkvdso->gtod_data_exchange.gtod_data[0].lock);
+	nkvdso->gtod_data_exchange.gtod_data[0].vread = NULL;
+	seqcount_init(&nkvdso->gtod_data_exchange.gtod_data[1].lock);
+	nkvdso->gtod_data_exchange.gtod_data[1].vread = NULL;
+
+	ipipe_set_gtod_data_exchange(&nkvdso->gtod_data_exchange);
 }
 
 static inline void handle_rt_signals(xnthread_t *thread,

[-- Attachment #6: 02-support-gettimeofday-in-realti.diff --]
[-- Type: text/x-patch, Size: 7740 bytes --]

Support gettimeofday in realtime context

From: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>

Wall time management is typically assisted by the NTP
protocol in the Linux context, but this information is
not propagated to Xenomai. This patch adds support
for gettimeofday(2) in primary mode by transactionally
copying the required information from the Kernel into
Xenomai. On the reader-side part, a seqlock is used
to ensure correctness of the mechanism when readers
on CPU n compete with a writer on CPU m != n

Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>
Signed-off-by: Jan Kiszka <jan.kiszka@domain.hid>
---
 include/posix/syscall.h        |    1 
 include/posix/time.h           |    4 ++
 ksrc/skins/posix/clock.c       |   84 ++++++++++++++++++++++++++++++++++++++++
 ksrc/skins/posix/syscall.c     |   18 +++++++++
 src/skins/posix/clock.c        |   15 +++++++
 src/skins/posix/posix.wrappers |    1 
 6 files changed, 123 insertions(+), 0 deletions(-)

diff --git a/include/posix/syscall.h b/include/posix/syscall.h
index 04a528d..b8d4f38 100644
--- a/include/posix/syscall.h
+++ b/include/posix/syscall.h
@@ -104,6 +104,7 @@
 #define __pse51_select                77
 #define __pse51_thread_setschedparam_ex	78
 #define __pse51_thread_getschedparam_ex	79
+#define __pse51_gettimeofday          80
 
 #ifdef __KERNEL__
 
diff --git a/include/posix/time.h b/include/posix/time.h
index 4f2d760..d1fe28d 100644
--- a/include/posix/time.h
+++ b/include/posix/time.h
@@ -74,6 +74,8 @@ int clock_nanosleep(clockid_t clock_id,
                     const struct timespec *rqtp,
 		    struct timespec *rmtp);
 
+int gettimeofday(struct timeval *tp, void *tzp);
+
 int nanosleep(const struct timespec *rqtp,
               struct timespec *rmtp);
 
@@ -111,6 +113,8 @@ int __real_clock_gettime(clockid_t clock_id,
 int __real_clock_settime(clockid_t clock_id,
 			 const struct timespec *tp);
 
+int __real_gettimeofday(struct timeval *tv, void *tzp);
+
 int __real_clock_nanosleep(clockid_t clock_id,
 			   int flags,
 			   const struct timespec *rqtp,
diff --git a/ksrc/skins/posix/clock.c b/ksrc/skins/posix/clock.c
index 553e123..fcf3e4d 100644
--- a/ksrc/skins/posix/clock.c
+++ b/ksrc/skins/posix/clock.c
@@ -50,6 +50,9 @@
  *@{*/
 
 #include <posix/thread.h>
+#include <linux/ipipe_tickdev.h>
+#include <linux/math64.h>
+#include <nucleus/vdso.h>
 
 /**
  * Get the resolution of the specified clock.
@@ -88,6 +91,86 @@ int clock_getres(clockid_t clock_id, struct timespec *res)
 }
 
 /**
+ * Read the clock.
+ *
+ * Obtain the current time with NTP corrections from the Linux domain
+ *
+ * @param tv pointer to a struct timeval
+ * @param tzp Timezone (ignored)
+ *
+ * @retval 0 on success; note however that if no suitable NTP-corrected
+ *         clocksource is available, the call still succeeds, but returns
+ *         the time obtained by clock_gettime() _without_ NTP corrections.
+ *
+ * @see
+ * <a href="http://www.opengroup.org/onlinepubs/000095399/functions/gettimeofday.html">
+ * Specification.</a>
+ *
+ */
+
+int gettimeofday(struct timeval *tv, void *tzp)
+{
+	cycle_t now, base, mask, cycle_delta;
+	unsigned long mult, shift, nsec, rem;
+	struct timespec tp;
+	struct ipipe_gtod_data *gtod_data;
+	unsigned int seq;
+
+retry:
+	gtod_data = get_gtod_data();
+
+	/* Note: gtod_data must not be protected by a lock because Linux
+	   cannot preempt us and gtod_data is guaranteed to point to a
+	   consistent data structure from the ipipe layer when the reader has
+	   interrupted the writer on the same CPU.
+
+	   The sequence counter is required when a reader is interleaved by a
+	   writer on a different CPU. It allows us to detect that the data
+	   have changed while reading them, and we switch to the alternative
+	   data element in this case (get_gtod_data() will automatically
+	   return it). When the time required to read the data is much smaller
+	   than the intervals between the updates (and no real-time
+	   process with higher priority preempts us), we loop at most thrice
+	   before we obtain a consistent data set. */
+	if (unlikely(!gtod_data) || unlikely(!gtod_data->vread)) {
+		printk(KERN_INFO "No gtod data provided, "
+		       "using non-rt gettimeofday");
+		clock_gettime(CLOCK_MONOTONIC, &tp);
+		tv->tv_sec = tp.tv_sec;
+		tv->tv_usec = tp.tv_nsec/1000;
+		return 0;
+	}
+
+	seq = read_seqcount_begin(&gtod_data->lock);
+
+	now = gtod_data->vread();
+	base = gtod_data->cycle_last;
+	mask = gtod_data->mask;
+	mult = gtod_data->mult;
+	shift = gtod_data->shift;
+	tv->tv_sec = gtod_data->wall_time_sec;
+	nsec = gtod_data->wall_time_nsec;
+
+	/* If the data changed during the read, try the
+	   alternative data element */
+	if (read_seqcount_retry(&gtod_data->lock, seq))
+		goto retry;
+
+	/* At this point, we have a consistent copy of the fundamental
+	 * data structure - calculate the interval between the current
+	 * and base time stamp cycles, and convert the difference
+	 * to nanoseconds. */
+	cycle_delta = (now - base) & mask;
+	nsec += (cycle_delta * mult) >> shift;
+
+	/* Convert to the desired sec, usec representation */
+	tv->tv_sec +=  xnarch_divrem_billion(nsec, &rem);
+	tv->tv_usec = rem / NSEC_PER_USEC;
+
+	return 0;
+}
+
+/**
  * Read the specified clock. 
  *
  * This service returns, at the address @a tp the current value of the clock @a
@@ -322,4 +405,5 @@ EXPORT_SYMBOL(clock_getres);
 EXPORT_SYMBOL(clock_gettime);
 EXPORT_SYMBOL(clock_settime);
 EXPORT_SYMBOL(clock_nanosleep);
+EXPORT_SYMBOL(gettimeofday);
 EXPORT_SYMBOL(nanosleep);
diff --git a/ksrc/skins/posix/syscall.c b/ksrc/skins/posix/syscall.c
index 2a97a2d..e403d12 100644
--- a/ksrc/skins/posix/syscall.c
+++ b/ksrc/skins/posix/syscall.c
@@ -765,6 +765,23 @@ static int __clock_gettime(struct pt_regs *regs)
 	return err ? -thread_get_errno() : 0;
 }
 
+static int __gettimeofday(struct pt_regs *regs)
+{
+	struct timeval tv;
+	int err;
+
+	/* POSIX leaves the behaviour of gettimeofday unspecified
+	   when tzp != NULL, so we adjust it to this value for
+	   simplicity's sake */
+	err = gettimeofday(&tv, NULL);
+
+	if (!err && __xn_safe_copy_to_user((void __user *)__xn_reg_arg1(regs),
+					   &tv, sizeof(tv)))
+		return -EFAULT;
+
+	return err ? -thread_get_errno() : 0;
+}
+
 static int __clock_settime(struct pt_regs *regs)
 {
 	struct timespec ts;
@@ -2775,6 +2792,7 @@ static xnsysent_t __systab[] = {
 	[__pse51_sem_open] = {&__sem_open, __xn_exec_any},
 	[__pse51_sem_close] = {&__sem_close, __xn_exec_any},
 	[__pse51_sem_unlink] = {&__sem_unlink, __xn_exec_any},
+	[__pse51_gettimeofday] = {&__gettimeofday, __xn_exec_any},
 	[__pse51_clock_getres] = {&__clock_getres, __xn_exec_any},
 	[__pse51_clock_gettime] = {&__clock_gettime, __xn_exec_any},
 	[__pse51_clock_settime] = {&__clock_settime, __xn_exec_any},
diff --git a/src/skins/posix/clock.c b/src/skins/posix/clock.c
index e27a2ec..4c31114 100644
--- a/src/skins/posix/clock.c
+++ b/src/skins/posix/clock.c
@@ -56,6 +56,21 @@ int __wrap_clock_getres(clockid_t clock_id, struct timespec *tp)
 	return -1;
 }
 
+int __wrap_gettimeofday(struct timeval *tv, void *tzp)
+{
+	int err;
+
+	err = -XENOMAI_SKINCALL2(__pse51_muxid,
+				 __pse51_gettimeofday,
+				 tv, tzp);
+
+	if (!err)
+		return 0;
+
+	errno = err;
+	return -1;
+}
+
 int __wrap_clock_gettime(clockid_t clock_id, struct timespec *tp)
 {
 	int err;
diff --git a/src/skins/posix/posix.wrappers b/src/skins/posix/posix.wrappers
index 776c5d5..b8927b5 100644
--- a/src/skins/posix/posix.wrappers
+++ b/src/skins/posix/posix.wrappers
@@ -18,6 +18,7 @@
 --wrap clock_gettime
 --wrap clock_settime
 --wrap clock_nanosleep
+--wrap gettimeofday
 --wrap nanosleep
 --wrap pthread_mutexattr_init
 --wrap pthread_mutexattr_destroy

[-- Attachment #7: 03-userspace-gtod-reading-without.diff --]
[-- Type: text/x-patch, Size: 6514 bytes --]

Userspace gtod reading without switching to kernel mode

From: Wolfgang Mauerer (none) <wolfgang@domain.hid>

We can do this since the data are on the shared semaphore heap.
The basis data structure is placed so that it is accessible from
both the Linux kernel and Xenomai kernel/userland. This also
requires to make the structure work with both kernel and userland
definitions for elementary data types.

Although we could read the TSC directly using __xn_rdtsc(), we
use the vread function also employed by the kernel to make the
code in both worlds aligned as good as possible.

Signed-off-by: Wolfgang Mauerer <wolfgang.mauerer@domain.hid>
Signed-off-by: Jan Kiszka <jan.kiszka@domain.hid>
---
 include/nucleus/seqlock_user.h |   77 ++++++++++++++++++++++++++++++++++++++++
 src/skins/posix/clock.c        |   47 ++++++++++++++++++++++++
 src/skins/posix/seqlock.h      |   77 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 201 insertions(+), 0 deletions(-)
 create mode 100644 include/nucleus/seqlock_user.h
 create mode 100644 src/skins/posix/seqlock.h

diff --git a/include/nucleus/seqlock_user.h b/include/nucleus/seqlock_user.h
new file mode 100644
index 0000000..35ed045
--- /dev/null
+++ b/include/nucleus/seqlock_user.h
@@ -0,0 +1,77 @@
+#ifndef __SEQLOCK_USER_H
+#define __SEQLOCK_USER_H
+
+/* Stolen from the linux kernel and slightly adapted for userland */
+
+// TODO: This is just the version for x86, other CPUs require
+// different definitions. Though it would make sense to remove
+// the cpu_relax() in the waiting path at all.
+static inline void rep_nop(void)
+{
+	asm volatile("rep; nop" ::: "memory");
+}
+
+static inline void cpu_relax(void)
+{
+	rep_nop();
+}
+
+
+// This definitely needs some work. It only works for the SMP and x86 case.
+#define barrier() __asm__ __volatile__("": : :"memory")
+
+#define smp_rmb()	barrier()
+#define smp_wmb()	barrier()
+
+//#define unlikely(x)    __builtin_expect(!!(x), 0)
+
+typedef struct seqcount {
+	unsigned sequence;
+} seqcount_t;
+
+#define SEQCNT_ZERO { 0 }
+#define seqcount_init(x)	do { *(x) = (seqcount_t) SEQCNT_ZERO; } while (0)
+
+/* Start of read using pointer to a sequence counter only.  */
+static inline unsigned read_seqcount_begin(const seqcount_t *s)
+{
+	unsigned ret;
+
+repeat:
+	ret = s->sequence;
+	smp_rmb();
+	if (unlikely(ret & 1)) {
+		cpu_relax();
+		goto repeat;
+	}
+	return ret;
+}
+
+/*
+ * Test if reader processed invalid data because sequence number has changed.
+ */
+static inline int read_seqcount_retry(const seqcount_t *s, unsigned start)
+{
+	smp_rmb();
+
+	return s->sequence != start;
+}
+
+
+/*
+ * Sequence counter only version assumes that callers are using their
+ * own mutexing.
+ */
+static inline void write_seqcount_begin(seqcount_t *s)
+{
+	s->sequence++;
+	smp_wmb();
+}
+
+static inline void write_seqcount_end(seqcount_t *s)
+{
+	smp_wmb();
+	s->sequence++;
+}
+
+#endif
diff --git a/src/skins/posix/clock.c b/src/skins/posix/clock.c
index 4c31114..ba1bc20 100644
--- a/src/skins/posix/clock.c
+++ b/src/skins/posix/clock.c
@@ -25,6 +25,9 @@
 #include <time.h>
 #include <asm/xenomai/arith.h>
 #include <asm-generic/xenomai/timeconv.h>
+#include <nucleus/seqlock_user.h>
+#include <sys/types.h>
+#include <nucleus/vdso.h>
 
 extern int __pse51_muxid;
 
@@ -59,6 +62,50 @@ int __wrap_clock_getres(clockid_t clock_id, struct timespec *tp)
 int __wrap_gettimeofday(struct timeval *tv, void *tzp)
 {
 	int err;
+#ifdef XNARCH_HAVE_NONPRIV_TSC
+	unsigned int seq;
+	cycle_t now, base, mask, cycle_delta;
+	unsigned long mult, shift, nsec, rem;
+
+        struct gtod_data_exchange *exchg = &nkvdso->gtod_data_exchange;
+	struct ipipe_gtod_data *gtod_data;
+
+retry:
+        gtod_data = &exchg->gtod_data[exchg->gtod_active];
+	if(unlikely(!gtod_data->vread))
+		goto fallback;
+
+	/*
+	 * The following is essentially a verbatim copy of the
+	 * mechanism in the kernel
+	 */
+	seq = read_seqcount_begin(&gtod_data->lock);
+
+	now = gtod_data->vread();
+	base = gtod_data->cycle_last;
+	mask = gtod_data->mask;
+	mult = gtod_data->mult;
+	shift = gtod_data->shift;
+	tv->tv_sec = gtod_data->wall_time_sec;
+	nsec = gtod_data->wall_time_nsec;
+
+	/* If the data changed during the read, try the
+	   alternative data element */
+	if (read_seqcount_retry(&gtod_data->lock, seq))
+		goto retry;
+
+	cycle_delta = (now - base) & mask;
+	nsec += (cycle_delta * mult) >> shift;
+
+#define NSEC_PER_USEC	1000L
+#define NSEC_PER_SEC	1000000000L
+	tv->tv_sec += xnarch_divrem_billion(nsec, &rem);
+	tv->tv_usec = rem / NSEC_PER_USEC;
+
+	return 0;
+
+fallback:
+#endif /* XNARCH_HAVE_NONPRIV_TSC */
 
 	err = -XENOMAI_SKINCALL2(__pse51_muxid,
 				 __pse51_gettimeofday,
diff --git a/src/skins/posix/seqlock.h b/src/skins/posix/seqlock.h
new file mode 100644
index 0000000..9929247
--- /dev/null
+++ b/src/skins/posix/seqlock.h
@@ -0,0 +1,77 @@
+#ifndef __SEQLOCK_H
+#define __SEQLOCK_H
+
+/* Stolen from the linux kernel and slightly adapted for userland */
+
+// TODO: This is just the version for x86, other CPUs require
+// different definitions. Though it would make sense to remove
+// the cpu_relax() in the waiting path at all.
+static inline void rep_nop(void)
+{
+	asm volatile("rep; nop" ::: "memory");
+}
+
+static inline void cpu_relax(void)
+{
+	rep_nop();
+}
+
+
+// This definitely needs some work. It only works for the SMP and x86 case.
+#define barrier() __asm__ __volatile__("": : :"memory")
+
+#define smp_rmb()	barrier()
+#define smp_wmb()	barrier()
+
+//#define unlikely(x)    __builtin_expect(!!(x), 0)
+
+typedef struct seqcount {
+	unsigned sequence;
+} seqcount_t;
+
+#define SEQCNT_ZERO { 0 }
+#define seqcount_init(x)	do { *(x) = (seqcount_t) SEQCNT_ZERO; } while (0)
+
+/* Start of read using pointer to a sequence counter only.  */
+static inline unsigned read_seqcount_begin(const seqcount_t *s)
+{
+	unsigned ret;
+
+repeat:
+	ret = s->sequence;
+	smp_rmb();
+	if (unlikely(ret & 1)) {
+		cpu_relax();
+		goto repeat;
+	}
+	return ret;
+}
+
+/*
+ * Test if reader processed invalid data because sequence number has changed.
+ */
+static inline int read_seqcount_retry(const seqcount_t *s, unsigned start)
+{
+	smp_rmb();
+
+	return s->sequence != start;
+}
+
+
+/*
+ * Sequence counter only version assumes that callers are using their
+ * own mutexing.
+ */
+static inline void write_seqcount_begin(seqcount_t *s)
+{
+	s->sequence++;
+	smp_wmb();
+}
+
+static inline void write_seqcount_end(seqcount_t *s)
+{
+	smp_wmb();
+	s->sequence++;
+}
+
+#endif

[-- Attachment #8: series-xenomai --]
[-- Type: text/plain, Size: 187 bytes --]

# This series applies on GIT commit 811b7af4ddb08f39ea12f4c98e56eee690e77769
01-add-gtod-bits-to-nkvdso.diff
02-support-gettimeofday-in-realti.diff
03-userspace-gtod-reading-without.diff

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Steve Deiters wrote:
>>> Periodically setting the time is risky if timed jobs depend 
>>> on Xenomai's real-time clock - it may jump in all directions...
>>>
>>>> Any other suggestions for providing timestamps to real time 
>>> tasks in 
>>>> this case?
>>> Do you just need precise timestamps from with real-time 
>>> tasks, or do you have to synchronize timer events of the 
>>> Xenomai core on an external clock?
>>>
>>> For the former case (precisely our scenario), we laid the 
>>> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain 
>>> Linux's view on gettimeofday. It "just" needs some polishing 
>>> to post this for upstream.
>>> Wolfgang (CC'ed) is working on this.
>> I'm just looking to get timestamps in the real time task.  At least in
>> my case being able to call gettimeofday from the real time thread would
>> be exactly what I need.
> 
> then you might want to try the attached patches for Ipipe and
> Xenomai. For upstream submission, they still need a bit of
> cleaning up as Jan mentioned, and I'll also prepare a proper
> series then. Testing is very welcome, however.

Ok. The following points should be fixed before submission:
- gettimeofday should not have another timebase than
clock_gettime(CLOCK_REALTIME): in other word, the whole clock system
should be based on the ntp clock.
- you should not use vread, you should use __xn_rdtsc directly in
user-space, otherwise, your code would only work on x86. Of course this
means that the I-pipe should create a clocksource with whatever hardware
counter the architecture is providing to Xenomai. This also means that
the I-pipe should declare a clocksource using whatever hardware counter
is provided by the architecture with highest priority than other
clocksources, to ensure that Linux is using the same clock source as
Xenomai, and that NTP is correcting that clock source. This would also
remove the issue with Linux declaring the tsc unstable.
- why the transactionnal mechanism at all? Why not simply using seqlocks
with an ipipe_spinlock, and do the update with irqs off? the locking
section is much shorter than, say, xnpod_schedule, so it will not have
any influence on the worst case latency, and the reader side will still
be lockless.
- the NTP event should trigger an ipipe event with ipipe_dispatch_event.

-- 
					    Gilles.

Re: [Xenomai-help] Question about getting system time

[Wolfgang Mauerer]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>> Steve Deiters wrote:
>>>> Periodically setting the time is risky if timed jobs depend 
>>>> on Xenomai's real-time clock - it may jump in all directions...
>>>>
>>>>> Any other suggestions for providing timestamps to real time 
>>>> tasks in 
>>>>> this case?
>>>> Do you just need precise timestamps from with real-time 
>>>> tasks, or do you have to synchronize timer events of the 
>>>> Xenomai core on an external clock?
>>>>
>>>> For the former case (precisely our scenario), we laid the 
>>>> ground to extend Xenomai 2.5 with RT-safe syscalls to obtain 
>>>> Linux's view on gettimeofday. It "just" needs some polishing 
>>>> to post this for upstream.
>>>> Wolfgang (CC'ed) is working on this.
>>> I'm just looking to get timestamps in the real time task.  At least in
>>> my case being able to call gettimeofday from the real time thread would
>>> be exactly what I need.
>> then you might want to try the attached patches for Ipipe and
>> Xenomai. For upstream submission, they still need a bit of
>> cleaning up as Jan mentioned, and I'll also prepare a proper
>> series then. Testing is very welcome, however.
> 
> Ok. The following points should be fixed before submission:
thanks for the comments!

> - gettimeofday should not have another timebase than
> clock_gettime(CLOCK_REALTIME): in other word, the whole clock system
> should be based on the ntp clock.
Sorry, I'm not quite sure what you are talking about here. The NTP
corrections are provided for the clock offered in the vsyscall page
by Linux, so the clock is based on the NTP clock, isn't it? Or
am I misparsing your statement?

> - you should not use vread, you should use __xn_rdtsc directly in
> user-space, otherwise, your code would only work on x86. Of course this
> means that the I-pipe should create a clocksource with whatever hardware
> counter the architecture is providing to Xenomai. This also means that
> the I-pipe should declare a clocksource using whatever hardware counter
> is provided by the architecture with highest priority than other
> clocksources, to ensure that Linux is using the same clock source as
> Xenomai, and that NTP is correcting that clock source. This would also
> remove the issue with Linux declaring the tsc unstable.
the reason why it's based on vread() is because the Linux kernel
automatically makes sure that it points to a function that can be called
from userland, so why would it only run on x86? Currently, the
userland patch is limited to x86 because I've only adapted the
sequence counter for this arch.

Besides, vread() could also work with a different source than the TSC as
long as it's accessible from userland.

> - why the transactionnal mechanism at all? Why not simply using seqlocks
> with an ipipe_spinlock, and do the update with irqs off? the locking
> section is much shorter than, say, xnpod_schedule, so it will not have
> any influence on the worst case latency, and the reader side will still
> be lockless.
Because even if we don't increase the peak latency, we'll still
increase the average latency. Additionally, it would also be possible
to extend the base mechanism to an RCU-style communication channel
between the kernel and Xenomai in the long run, so I'd argue that
the lock-less solution is nicer.

> - the NTP event should trigger an ipipe event with ipipe_dispatch_event.
could do, but I was following Gilles' suggestion  ;-) to use an
arch-specific hook since it's easier to maintain in the long run than
writing a generic function and replacing every call to
vsyscall_update(), also the future.
(http://www.opensubscriber.com/message/xenomai@xenomai.org,
although I can change that, certainly no religious issues here)

Thanks, Wolfgang

Re: [Xenomai-help] Question about getting system time

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
>> - gettimeofday should not have another timebase than
>> clock_gettime(CLOCK_REALTIME): in other word, the whole clock system
>> should be based on the ntp clock.
> Sorry, I'm not quite sure what you are talking about here. The NTP
> corrections are provided for the clock offered in the vsyscall page
> by Linux, so the clock is based on the NTP clock, isn't it? Or
> am I misparsing your statement?

Unless I misunderstood your patch, what you provide is:
gettimeofday which uses linux timebase
clock_gettime which uses Xenomai timebase unrelated to linux' timebase
This is unacceptable. What we want is a unique timebase.

And since this will have a cost not everyone is willing to pay, I have
another requirement: this code should be compiled conditionally.

> 
>> - you should not use vread, you should use __xn_rdtsc directly in
>> user-space, otherwise, your code would only work on x86. Of course this
>> means that the I-pipe should create a clocksource with whatever hardware
>> counter the architecture is providing to Xenomai. This also means that
>> the I-pipe should declare a clocksource using whatever hardware counter
>> is provided by the architecture with highest priority than other
>> clocksources, to ensure that Linux is using the same clock source as
>> Xenomai, and that NTP is correcting that clock source. This would also
>> remove the issue with Linux declaring the tsc unstable.
> the reason why it's based on vread() is because the Linux kernel
> automatically makes sure that it points to a function that can be called
> from userland, so why would it only run on x86? Currently, the
> userland patch is limited to x86 because I've only adapted the
> sequence counter for this arch.
> 
> Besides, vread() could also work with a different source than the TSC as
> long as it's accessible from userland.

vread is a function pointer call, which:
- requires vsyscalls, currently only implemented on x86 (and maybe ppc)
- is function pointer call, so damn slow on low end hardware.

Xenomai has __xn_rdtsc on all architectures, so, we should be based on that.

Since what we want is Xenomai to use the same clock source as Linux, and
anything else than tsc is not implemented for Xenomai, we should
implement tsc first and keep other clock sources for later. And when we
use another clock, __xn_rdtsc will use that clock anyway.

> 
>> - why the transactionnal mechanism at all? Why not simply using seqlocks
>> with an ipipe_spinlock, and do the update with irqs off? the locking
>> section is much shorter than, say, xnpod_schedule, so it will not have
>> any influence on the worst case latency, and the reader side will still
>> be lockless.
> Because even if we don't increase the peak latency, we'll still
> increase the average latency. Additionally, it would also be possible
> to extend the base mechanism to an RCU-style communication channel
> between the kernel and Xenomai in the long run, so I'd argue that
> the lock-less solution is nicer.

It is a useless optimization, it is a lot of code to avoid shutting
interrupts of for a handful of assignments (since you can avoid copying
vread, and mult and shift which never change if we enfore the clock
source). There are many more, longer masking sections everywhere in the
I-pipe patch and in Xenomai code.

On the one hand you make complicated code (which will be costly on low
end hardware) to avoid shutting interrupts around a few assignments, but
on the other hand you leave an architecture specific function pointer
call where we want a fast behaviour on average (remember, we do all this
to avoid a system call, which is only a few hundreds nanoseconds on your
big iron x86), and where we have a generic fast replacement. Sometimes,
I do not understand your logic.

> 
>> - the NTP event should trigger an ipipe event with ipipe_dispatch_event.
> could do, but I was following Gilles' suggestion  ;-) to use an
> arch-specific hook since it's easier to maintain in the long run than
> writing a generic function and replacing every call to
> vsyscall_update(), also the future.
> (http://www.opensubscriber.com/message/xenomai@xenomai.org,
> although I can change that, certainly no religious issues here)

What I mean is that vsyscall_update should trigger ipipe_dispatch_event.
This is what I meant from the beginning.

Ok. I will not be able to answer much more during the rest of the
afternoon, so do not be surprised if I remain silent until tonight.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Wolfgang Mauerer]

[moved to xenomai-core]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>>> - gettimeofday should not have another timebase than
>>> clock_gettime(CLOCK_REALTIME): in other word, the whole clock system
>>> should be based on the ntp clock.
>> Sorry, I'm not quite sure what you are talking about here. The NTP
>> corrections are provided for the clock offered in the vsyscall page
>> by Linux, so the clock is based on the NTP clock, isn't it? Or
>> am I misparsing your statement?
> 
> Unless I misunderstood your patch, what you provide is:
> gettimeofday which uses linux timebase
> clock_gettime which uses Xenomai timebase unrelated to linux' timebase
> This is unacceptable. What we want is a unique timebase.

okay, I see your point now. But this sound more like 3.0 stuff, doesn't
it? The goal right now is just to have some code that generates
synchronised, NTP corrected timestamps on Linux and Xenomai, not
to rework the base timer handling.
> 
> And since this will have a cost not everyone is willing to pay, I have
> another requirement: this code should be compiled conditionally.
> 
>>> - you should not use vread, you should use __xn_rdtsc directly in
>>> user-space, otherwise, your code would only work on x86. Of course this
>>> means that the I-pipe should create a clocksource with whatever hardware
>>> counter the architecture is providing to Xenomai. This also means that
>>> the I-pipe should declare a clocksource using whatever hardware counter
>>> is provided by the architecture with highest priority than other
>>> clocksources, to ensure that Linux is using the same clock source as
>>> Xenomai, and that NTP is correcting that clock source. This would also
>>> remove the issue with Linux declaring the tsc unstable.
>> the reason why it's based on vread() is because the Linux kernel
>> automatically makes sure that it points to a function that can be called
>> from userland, so why would it only run on x86? Currently, the
>> userland patch is limited to x86 because I've only adapted the
>> sequence counter for this arch.
>>
>> Besides, vread() could also work with a different source than the TSC as
>> long as it's accessible from userland.
> 
> vread is a function pointer call, which:
> - requires vsyscalls, currently only implemented on x86 (and maybe ppc)
> - is function pointer call, so damn slow on low end hardware.

... and fast as lightning on x86 when the call is made often, which
is the important case when speed matters. So it makes sense on this
platform.
> 
> Xenomai has __xn_rdtsc on all architectures, so, we should be based on that.
> 
> Since what we want is Xenomai to use the same clock source as Linux, and
> anything else than tsc is not implemented for Xenomai, we should
> implement tsc first and keep other clock sources for later. And when we
> use another clock, __xn_rdtsc will use that clock anyway.
> 
>>> - why the transactionnal mechanism at all? Why not simply using seqlocks
>>> with an ipipe_spinlock, and do the update with irqs off? the locking
>>> section is much shorter than, say, xnpod_schedule, so it will not have
>>> any influence on the worst case latency, and the reader side will still
>>> be lockless.
>> Because even if we don't increase the peak latency, we'll still
>> increase the average latency. Additionally, it would also be possible
>> to extend the base mechanism to an RCU-style communication channel
>> between the kernel and Xenomai in the long run, so I'd argue that
>> the lock-less solution is nicer.
> 
> It is a useless optimization, it is a lot of code to avoid shutting
> interrupts of for a handful of assignments (since you can avoid copying
> vread, and mult and shift which never change if we enfore the clock
> source). There are many more, longer masking sections everywhere in the
> I-pipe patch and in Xenomai code.
> 
> On the one hand you make complicated code (which will be costly on low
> end hardware) to avoid shutting interrupts around a few assignments, but
> on the other hand you leave an architecture specific function pointer
> call where we want a fast behaviour on average (remember, we do all this
> to avoid a system call, which is only a few hundreds nanoseconds on your
> big iron x86), and where we have a generic fast replacement. Sometimes,
> I do not understand your logic.

But using the same argument, you could get rid of Linux vsyscall based
gettimeofday()...

Would having the synchronised time stamp feature (maybe by another name
than gettimeofday()) conditionally compiled on x86 be acceptable?

Best, Wolfgang

> 
>>> - the NTP event should trigger an ipipe event with ipipe_dispatch_event.
>> could do, but I was following Gilles' suggestion  ;-) to use an
>> arch-specific hook since it's easier to maintain in the long run than
>> writing a generic function and replacing every call to
>> vsyscall_update(), also the future.
>> (http://www.opensubscriber.com/message/xenomai@xenomai.org,
>> although I can change that, certainly no religious issues here)
> 
> What I mean is that vsyscall_update should trigger ipipe_dispatch_event.
> This is what I meant from the beginning.
> 
> Ok. I will not be able to answer much more during the rest of the
> afternoon, so do not be surprised if I remain silent until tonight.
> 

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> [moved to xenomai-core]
> 
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>>> - gettimeofday should not have another timebase than
>>>> clock_gettime(CLOCK_REALTIME): in other word, the whole clock system
>>>> should be based on the ntp clock.
>>> Sorry, I'm not quite sure what you are talking about here. The NTP
>>> corrections are provided for the clock offered in the vsyscall page
>>> by Linux, so the clock is based on the NTP clock, isn't it? Or
>>> am I misparsing your statement?
>> Unless I misunderstood your patch, what you provide is:
>> gettimeofday which uses linux timebase
>> clock_gettime which uses Xenomai timebase unrelated to linux' timebase
>> This is unacceptable. What we want is a unique timebase.
> 
> okay, I see your point now. But this sound more like 3.0 stuff, doesn't
> it? The goal right now is just to have some code that generates
> synchronised, NTP corrected timestamps on Linux and Xenomai, not
> to rework the base timer handling.

We are going to open a 2.6 branch soon (we are migrating Xenomai to a
new hoster, and this job is almost finished). This will go to 2.6. And I
am not going to merge anything that would provide two different
timebases for CLOCK_REALTIME. That is too unnatural. There are a bunch
of natural things which will not work with this approach. It create many
ways to shoot oneself in the foot. So the answer is definitely no.

>> vread is a function pointer call, which:
>> - requires vsyscalls, currently only implemented on x86 (and maybe ppc)
>> - is function pointer call, so damn slow on low end hardware.
> 
> ... and fast as lightning on x86 when the call is made often, which
> is the important case when speed matters. So it makes sense on this
> platform.

rdtsc is even faster. It is just one instruction.
The generic solution is even faster than the architecture specific one.
So, please implement the generic one.

>> On the one hand you make complicated code (which will be costly on low
>> end hardware) to avoid shutting interrupts around a few assignments, but
>> on the other hand you leave an architecture specific function pointer
>> call where we want a fast behaviour on average (remember, we do all this
>> to avoid a system call, which is only a few hundreds nanoseconds on your
>> big iron x86), and where we have a generic fast replacement. Sometimes,
>> I do not understand your logic.
> 
> But using the same argument, you could get rid of Linux vsyscall based
> gettimeofday()...
> 
> Would having the synchronised time stamp feature (maybe by another name
> than gettimeofday()) conditionally compiled on x86 be acceptable?

No. We have a generic implementation at hand for a limited amount of
additional effort. Let us do things properly.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
>> On the one hand you make complicated code (which will be costly on low
>> end hardware) to avoid shutting interrupts around a few assignments, but
>> on the other hand you leave an architecture specific function pointer
>> call where we want a fast behaviour on average (remember, we do all this
>> to avoid a system call, which is only a few hundreds nanoseconds on your
>> big iron x86), and where we have a generic fast replacement. Sometimes,
>> I do not understand your logic.
> 
> But using the same argument, you could get rid of Linux vsyscall based
> gettimeofday()...

I do not see your point, the Linux code does not go a long way to make
lockless code, it simply turns off interrupts around the gtod data
update, which is really reasonable given the size of the masking
section. The reading is lockless, the writing is not.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Wolfgang Mauerer]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>>> On the one hand you make complicated code (which will be costly on low
>>> end hardware) to avoid shutting interrupts around a few assignments, but
>>> on the other hand you leave an architecture specific function pointer
>>> call where we want a fast behaviour on average (remember, we do all this
>>> to avoid a system call, which is only a few hundreds nanoseconds on your
>>> big iron x86), and where we have a generic fast replacement. Sometimes,
>>> I do not understand your logic.
>> But using the same argument, you could get rid of Linux vsyscall based
>> gettimeofday()...
> 
> I do not see your point, the Linux code does not go a long way to make
> lockless code, it simply turns off interrupts around the gtod data
> update, which is really reasonable given the size of the masking
> section. The reading is lockless, the writing is not.
> 
I was referring to the argument that system calls are so fast that
replacing gtod with a syscall-less version that uses function
pointer dereferencing instead does not make much of a difference.

Be it as it may, I need to check how far our budget can cover
the (much more comprehensive) modifications required for the
solution suggested by you. Let's see.

Regards, Wolfgang

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>>> On the one hand you make complicated code (which will be costly on low
>>>> end hardware) to avoid shutting interrupts around a few assignments, but
>>>> on the other hand you leave an architecture specific function pointer
>>>> call where we want a fast behaviour on average (remember, we do all this
>>>> to avoid a system call, which is only a few hundreds nanoseconds on your
>>>> big iron x86), and where we have a generic fast replacement. Sometimes,
>>>> I do not understand your logic.
>>> But using the same argument, you could get rid of Linux vsyscall based
>>> gettimeofday()...
>> I do not see your point, the Linux code does not go a long way to make
>> lockless code, it simply turns off interrupts around the gtod data
>> update, which is really reasonable given the size of the masking
>> section. The reading is lockless, the writing is not.
>>
> I was referring to the argument that system calls are so fast that
> replacing gtod with a syscall-less version that uses function
> pointer dereferencing instead does not make much of a difference.

That is not what I said. I compared the weight of a function pointer
call with the one of four asignments with irqs off. And yes syscalls are
fast on x86, do the measurements yourself, you may be surprised.

> 
> Be it as it may, I need to check how far our budget can cover
> the (much more comprehensive) modifications required for the
> solution suggested by you. Let's see.

I do not think there is that much work involved. The way I see it, we
would need to replace our tsc reading function with one returning
"ntp-corrected" tsc (that is, essentially a subset of the gettimeofday
function you implemented, without conversion to ns and to CLOCK_REALTIME).

Changes in this monotonic clock would trigger a recomputation of the
next timer event date.
Changes in monotonic to real-time conversion would trigger a call to
xnpod_set_time.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2834 bytes --]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>> Gilles Chanteperdrix wrote:
>>> Wolfgang Mauerer wrote:
>>>>> On the one hand you make complicated code (which will be costly on low
>>>>> end hardware) to avoid shutting interrupts around a few assignments, but
>>>>> on the other hand you leave an architecture specific function pointer
>>>>> call where we want a fast behaviour on average (remember, we do all this
>>>>> to avoid a system call, which is only a few hundreds nanoseconds on your
>>>>> big iron x86), and where we have a generic fast replacement. Sometimes,
>>>>> I do not understand your logic.
>>>> But using the same argument, you could get rid of Linux vsyscall based
>>>> gettimeofday()...
>>> I do not see your point, the Linux code does not go a long way to make
>>> lockless code, it simply turns off interrupts around the gtod data
>>> update, which is really reasonable given the size of the masking
>>> section. The reading is lockless, the writing is not.
>>>
>> I was referring to the argument that system calls are so fast that
>> replacing gtod with a syscall-less version that uses function
>> pointer dereferencing instead does not make much of a difference.
> 
> That is not what I said. I compared the weight of a function pointer
> call with the one of four asignments with irqs off. And yes syscalls are
> fast on x86, do the measurements yourself, you may be surprised.
> 
>> Be it as it may, I need to check how far our budget can cover
>> the (much more comprehensive) modifications required for the
>> solution suggested by you. Let's see.
> 
> I do not think there is that much work involved. The way I see it, we
> would need to replace our tsc reading function with one returning
> "ntp-corrected" tsc (that is, essentially a subset of the gettimeofday
> function you implemented, without conversion to ns and to CLOCK_REALTIME).
> 
> Changes in this monotonic clock would trigger a recomputation of the
> next timer event date.
> Changes in monotonic to real-time conversion would trigger a call to
> xnpod_set_time.

If all works out well, it might be that simple. But this is timer/clock
stuff, the heart of the system, and easy to get subtly wrong.

For that reason the plan was to gain more confidence in the externally
corrected clock source, collect experience in other use cases, and then
work on the core for its optional use.

So far our customer is using this clock for important but not yet
critical jobs. Making it the RT clock source is of a different quality,
and for now without a use case. I personally do want this feature
long-term, definitely, but I do not see ATM it's going to be something
we can deliver for 2.6. Maybe we can split out the lower bits and
prepare I-pipe as well as the hal already.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Wolfgang Mauerer wrote:
>>>>>> On the one hand you make complicated code (which will be costly on low
>>>>>> end hardware) to avoid shutting interrupts around a few assignments, but
>>>>>> on the other hand you leave an architecture specific function pointer
>>>>>> call where we want a fast behaviour on average (remember, we do all this
>>>>>> to avoid a system call, which is only a few hundreds nanoseconds on your
>>>>>> big iron x86), and where we have a generic fast replacement. Sometimes,
>>>>>> I do not understand your logic.
>>>>> But using the same argument, you could get rid of Linux vsyscall based
>>>>> gettimeofday()...
>>>> I do not see your point, the Linux code does not go a long way to make
>>>> lockless code, it simply turns off interrupts around the gtod data
>>>> update, which is really reasonable given the size of the masking
>>>> section. The reading is lockless, the writing is not.
>>>>
>>> I was referring to the argument that system calls are so fast that
>>> replacing gtod with a syscall-less version that uses function
>>> pointer dereferencing instead does not make much of a difference.
>> That is not what I said. I compared the weight of a function pointer
>> call with the one of four asignments with irqs off. And yes syscalls are
>> fast on x86, do the measurements yourself, you may be surprised.
>>
>>> Be it as it may, I need to check how far our budget can cover
>>> the (much more comprehensive) modifications required for the
>>> solution suggested by you. Let's see.
>> I do not think there is that much work involved. The way I see it, we
>> would need to replace our tsc reading function with one returning
>> "ntp-corrected" tsc (that is, essentially a subset of the gettimeofday
>> function you implemented, without conversion to ns and to CLOCK_REALTIME).
>>
>> Changes in this monotonic clock would trigger a recomputation of the
>> next timer event date.
>> Changes in monotonic to real-time conversion would trigger a call to
>> xnpod_set_time.
> 
> If all works out well, it might be that simple. But this is timer/clock
> stuff, the heart of the system, and easy to get subtly wrong.
> 
> For that reason the plan was to gain more confidence in the externally
> corrected clock source, collect experience in other use cases, and then
> work on the core for its optional use.
> 
> So far our customer is using this clock for important but not yet
> critical jobs. Making it the RT clock source is of a different quality,
> and for now without a use case.

To be quite frank about use cases, I do not really understand in what
use case the patch Wolfgang sent is useful. An application can not use
the timestamps returned by this gettimeofday syscall for anything
useful. And if we talk about things breaking subtly, an application that
would use these timestamps with Xenomai services would be subtly broken
too.

Avoiding the drift between Xenomai clock and Linux clock by making them
synchronized by design, on the other hand, albeit probably solving a
corner case, looks more useful, and some people asked for it (it looks
to me like what Steve asked for is that, not to have a third timebase
accessible in real-time context, and Steve asked for a solution for
powerpc, not for x86).

So, my feeling about all this is that Wolfgang's patch is not useful for
anyone else than your customer.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 4854 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Wolfgang Mauerer wrote:
>>>> Gilles Chanteperdrix wrote:
>>>>> Wolfgang Mauerer wrote:
>>>>>>> On the one hand you make complicated code (which will be costly on low
>>>>>>> end hardware) to avoid shutting interrupts around a few assignments, but
>>>>>>> on the other hand you leave an architecture specific function pointer
>>>>>>> call where we want a fast behaviour on average (remember, we do all this
>>>>>>> to avoid a system call, which is only a few hundreds nanoseconds on your
>>>>>>> big iron x86), and where we have a generic fast replacement. Sometimes,
>>>>>>> I do not understand your logic.
>>>>>> But using the same argument, you could get rid of Linux vsyscall based
>>>>>> gettimeofday()...
>>>>> I do not see your point, the Linux code does not go a long way to make
>>>>> lockless code, it simply turns off interrupts around the gtod data
>>>>> update, which is really reasonable given the size of the masking
>>>>> section. The reading is lockless, the writing is not.
>>>>>
>>>> I was referring to the argument that system calls are so fast that
>>>> replacing gtod with a syscall-less version that uses function
>>>> pointer dereferencing instead does not make much of a difference.
>>> That is not what I said. I compared the weight of a function pointer
>>> call with the one of four asignments with irqs off. And yes syscalls are
>>> fast on x86, do the measurements yourself, you may be surprised.
>>>
>>>> Be it as it may, I need to check how far our budget can cover
>>>> the (much more comprehensive) modifications required for the
>>>> solution suggested by you. Let's see.
>>> I do not think there is that much work involved. The way I see it, we
>>> would need to replace our tsc reading function with one returning
>>> "ntp-corrected" tsc (that is, essentially a subset of the gettimeofday
>>> function you implemented, without conversion to ns and to CLOCK_REALTIME).
>>>
>>> Changes in this monotonic clock would trigger a recomputation of the
>>> next timer event date.
>>> Changes in monotonic to real-time conversion would trigger a call to
>>> xnpod_set_time.
>> If all works out well, it might be that simple. But this is timer/clock
>> stuff, the heart of the system, and easy to get subtly wrong.
>>
>> For that reason the plan was to gain more confidence in the externally
>> corrected clock source, collect experience in other use cases, and then
>> work on the core for its optional use.
>>
>> So far our customer is using this clock for important but not yet
>> critical jobs. Making it the RT clock source is of a different quality,
>> and for now without a use case.
> 
> To be quite frank about use cases, I do not really understand in what
> use case the patch Wolfgang sent is useful. An application can not use
> the timestamps returned by this gettimeofday syscall for anything
> useful. And if we talk about things breaking subtly, an application that
> would use these timestamps with Xenomai services would be subtly broken
> too.

Just like it seems to be the case for Steve (unless I misunderstood his
reply), it is very useful for us being able to time-stamp events in RT
context that need to be correlated with events stamped in non-RT
(including non-Xenomai) parts or even on other systems: (offline) data
fusion, logging, tracing. I even bet that this is currently the major
use case for synchronized clocks, only a smaller part already has the
need to synchronize timed activities on a common clock source. But there
is huge potential in the second part once we can provide a stable
infrastructure.

> 
> Avoiding the drift between Xenomai clock and Linux clock by making them
> synchronized by design, on the other hand, albeit probably solving a
> corner case, looks more useful, and some people asked for it (it looks
> to me like what Steve asked for is that, not to have a third timebase

It is surely more useful, but also more complex. Nothing unsolvable, but
asking for more care.

> accessible in real-time context, and Steve asked for a solution for
> powerpc, not for x86).

Even a "third clock" would have to be delivered for more archs than x86,
no question. We would basically need a generic but slow syscall variant
and per-arch syscall-less optimizations (where feasible).

> 
> So, my feeling about all this is that Wolfgang's patch is not useful for
> anyone else than your customer.
> 

I think your feeling is a bit too pessimistic on this general approach
and a bit too optimistic regarding the complexity of a complete
solution. But I wouldn't mind being proven wrong, specifically regarding
the latter. However, let's see if we can do some uncontroversial steps
in this direction.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Just like it seems to be the case for Steve (unless I misunderstood his
> reply), it is very useful for us being able to time-stamp events in RT
> context that need to be correlated with events stamped in non-RT
> (including non-Xenomai) parts or even on other systems: (offline) data
> fusion, logging, tracing. I even bet that this is currently the major
> use case for synchronized clocks, only a smaller part already has the
> need to synchronize timed activities on a common clock source. But there
> is huge potential in the second part once we can provide a stable
> infrastructure.

I already had such issues, and I did not solve them by modifying Xenomai
core.

> Even a "third clock" would have to be delivered for more archs than x86,
> no question. We would basically need a generic but slow syscall variant
> and per-arch syscall-less optimizations (where feasible).

So, you would add a syscall which would becomre useless when you have
implemented synchronized clocks properly? Yet another reason for
avoiding this solution.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Just like it seems to be the case for Steve (unless I misunderstood his
>> reply), it is very useful for us being able to time-stamp events in RT
>> context that need to be correlated with events stamped in non-RT
>> (including non-Xenomai) parts or even on other systems: (offline) data
>> fusion, logging, tracing. I even bet that this is currently the major
>> use case for synchronized clocks, only a smaller part already has the
>> need to synchronize timed activities on a common clock source. But there
>> is huge potential in the second part once we can provide a stable
>> infrastructure.
> 
> I already had such issues, and I did not solve them by modifying Xenomai
> core.
> 
>> Even a "third clock" would have to be delivered for more archs than x86,
>> no question. We would basically need a generic but slow syscall variant
>> and per-arch syscall-less optimizations (where feasible).
> 
> So, you would add a syscall which would becomre useless when you have
> implemented synchronized clocks properly? Yet another reason for
> avoiding this solution.
> 

Could be "CLOCK_LINUX" - ie. no need for a new syscall.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-core] Question about getting system time

[Daniele Nicolodi]

On 19/05/10 07:49, Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Just like it seems to be the case for Steve (unless I misunderstood his
>> reply), it is very useful for us being able to time-stamp events in RT
>> context that need to be correlated with events stamped in non-RT
>> (including non-Xenomai) parts or even on other systems: (offline) data
>> fusion, logging, tracing. I even bet that this is currently the major
>> use case for synchronized clocks, only a smaller part already has the
>> need to synchronize timed activities on a common clock source. But there
>> is huge potential in the second part once we can provide a stable
>> infrastructure.
> 
> I already had such issues, and I did not solve them by modifying Xenomai
> core.

I'm using Xenomai for scientific data acquisition, so I'm quite
interested to the issue. Can I ask you to detail how you solved the issue?

Maybe it's not much related, but there is any possibility to synchronize
either linux or xenomay clocks to an external clock source, let say the
GPS timing signal from a GPS receiver?

Thanks. Cheers,
-- 
Daniele

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Daniele Nicolodi wrote:
> On 19/05/10 07:49, Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Just like it seems to be the case for Steve (unless I misunderstood his
>>> reply), it is very useful for us being able to time-stamp events in RT
>>> context that need to be correlated with events stamped in non-RT
>>> (including non-Xenomai) parts or even on other systems: (offline) data
>>> fusion, logging, tracing. I even bet that this is currently the major
>>> use case for synchronized clocks, only a smaller part already has the
>>> need to synchronize timed activities on a common clock source. But there
>>> is huge potential in the second part once we can provide a stable
>>> infrastructure.
>> I already had such issues, and I did not solve them by modifying Xenomai
>> core.
> 
> I'm using Xenomai for scientific data acquisition, so I'm quite
> interested to the issue. Can I ask you to detail how you solved the issue?

In my case, I was working on network equipments, and we needed
timestamping of rtnet packets to be synchronized with linux clock, to
compare/merge them with timestamps of network packets captured in the
Linux domain.

An rtnet packet is passed to the Linux domain not longer after it has
been captured (and timestamped) in Xenomai domain. So, I just corrected
the Xenomai timestamp using the difference between xenomai timebase and
Linux timebase at the time when the packet was passed to Linux domain.
It turned out to be quite precise, and if it is not precise enough, some
statistical method could probably be used to evaluate the drift and
compensate for it.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Just like it seems to be the case for Steve (unless I misunderstood his
>>> reply), it is very useful for us being able to time-stamp events in RT
>>> context that need to be correlated with events stamped in non-RT
>>> (including non-Xenomai) parts or even on other systems: (offline) data
>>> fusion, logging, tracing. I even bet that this is currently the major
>>> use case for synchronized clocks, only a smaller part already has the
>>> need to synchronize timed activities on a common clock source. But there
>>> is huge potential in the second part once we can provide a stable
>>> infrastructure.
>> I already had such issues, and I did not solve them by modifying Xenomai
>> core.
>>
>>> Even a "third clock" would have to be delivered for more archs than x86,
>>> no question. We would basically need a generic but slow syscall variant
>>> and per-arch syscall-less optimizations (where feasible).
>> So, you would add a syscall which would becomre useless when you have
>> implemented synchronized clocks properly? Yet another reason for
>> avoiding this solution.
>>
> 
> Could be "CLOCK_LINUX" - ie. no need for a new syscall.

I am Ok for this solution (and now that I think about it, I wonder if we
did not already have this discussion). Anyway, I would go for
CLOCK_HOST_REALTIME, in case someone wants to implement
CLOCK_HOST_MONOTONIC. The advantage is that we can return EINVAL in the
timer_create or clock_settime calls, to indicate clearly that using this
clock for timing services is verboten. And when/if the full
synchronization is implemented, the ID simply becomes a #define.

-- 
					    Gilles.

Re: [Xenomai-core] Question about getting system time

[Wolfgang Mauerer]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Just like it seems to be the case for Steve (unless I misunderstood his
>>>> reply), it is very useful for us being able to time-stamp events in RT
>>>> context that need to be correlated with events stamped in non-RT
>>>> (including non-Xenomai) parts or even on other systems: (offline) data
>>>> fusion, logging, tracing. I even bet that this is currently the major
>>>> use case for synchronized clocks, only a smaller part already has the
>>>> need to synchronize timed activities on a common clock source. But there
>>>> is huge potential in the second part once we can provide a stable
>>>> infrastructure.
>>> I already had such issues, and I did not solve them by modifying Xenomai
>>> core.
>>>
>>>> Even a "third clock" would have to be delivered for more archs than x86,
>>>> no question. We would basically need a generic but slow syscall variant
>>>> and per-arch syscall-less optimizations (where feasible).
>>> So, you would add a syscall which would becomre useless when you have
>>> implemented synchronized clocks properly? Yet another reason for
>>> avoiding this solution.
>>>
>> Could be "CLOCK_LINUX" - ie. no need for a new syscall.
> 
> I am Ok for this solution (and now that I think about it, I wonder if we
> did not already have this discussion). Anyway, I would go for
> CLOCK_HOST_REALTIME, in case someone wants to implement
> CLOCK_HOST_MONOTONIC. The advantage is that we can return EINVAL in the
> timer_create or clock_settime calls, to indicate clearly that using this
> clock for timing services is verboten. And when/if the full
> synchronization is implemented, the ID simply becomes a #define.

okay, so let's pursue this path. Patches will follow.

Best, Wolfgang