* [Xenomai] usage of rtdm_task_sleep_abs
@ 2015-07-09 12:15 Johann Obermayr
2015-07-09 14:19 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-09 12:15 UTC (permalink / raw)
To: Xenomai
Hello,
we have follow situation
u64 lrtdrv_time_of_irq_ns ;
void worker_task()
{
while(1)
{
rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
RTDM_TIMERMODE_ABSOLUTE);
do_something();
}
}
_kernel_rtdm_irq_handler()
{
lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
}
the _kernel_irq_callback() is called every 1ms.
we will , that the worker_task begin 50us before next irq
But sometime, the worker task start ~50us after irq. Why ?
Sometime we call xntbase_adjust_time to sync external RTC with internal
clocks.
can this be a problem ?
Hardware & Software
X86 SMP Dual celeron
Linux Kernel 3.10.0
Xenomai 2.6.3
Regards
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-09 12:15 [Xenomai] usage of rtdm_task_sleep_abs Johann Obermayr
@ 2015-07-09 14:19 ` Gilles Chanteperdrix
2015-07-09 15:43 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-09 14:19 UTC (permalink / raw)
To: Johann Obermayr; +Cc: Xenomai
On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
> Hello,
>
> we have follow situation
>
> u64 lrtdrv_time_of_irq_ns ;
> void worker_task()
> {
> while(1)
> {
> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
> RTDM_TIMERMODE_ABSOLUTE);
> do_something();
> }
> }
>
> _kernel_rtdm_irq_handler()
> {
> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
> }
>
> the _kernel_irq_callback() is called every 1ms.
> we will , that the worker_task begin 50us before next irq
>
> But sometime, the worker task start ~50us after irq. Why ?
Best way to know, enable the I-pipe tracer, setup sufficient back
trace points, and when the wrong wake up happens, trigger an I-pipe
trace.
>
> Sometime we call xntbase_adjust_time to sync external RTC with internal
> clocks.
> can this be a problem ?
Normally, xntbase_adjust_time has no influence whatsoever on the
monotonic time. However, I suppose if you have an awful lot of
pending timers with absolute wall clock expiration dates, the
adjustment made by xntbase_adjust_time could create a large masking
section.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-09 14:19 ` Gilles Chanteperdrix
@ 2015-07-09 15:43 ` Johann Obermayr
2015-07-09 15:51 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-09 15:43 UTC (permalink / raw)
Cc: Xenomai
Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
> On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
>> Hello,
>>
>> we have follow situation
>>
>> u64 lrtdrv_time_of_irq_ns ;
>> void worker_task()
>> {
>> while(1)
>> {
>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
>> RTDM_TIMERMODE_ABSOLUTE);
>> do_something();
>> }
>> }
>>
>> _kernel_rtdm_irq_handler()
>> {
>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
>> }
>>
>> the _kernel_irq_callback() is called every 1ms.
>> we will , that the worker_task begin 50us before next irq
>>
>> But sometime, the worker task start ~50us after irq. Why ?
> Best way to know, enable the I-pipe tracer, setup sufficient back
> trace points, and when the wrong wake up happens, trigger an I-pipe
> trace.
>
i will enable i-pipe tracer.
found out, that
rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
u64 calc = rtdm_clock_read_monotonic();
sometime the diff between xxx & calc is biger than 61000ns.
forget to say.
worker_task run on Core1
irq_handler run on Core0
Regards
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-09 15:43 ` Johann Obermayr
@ 2015-07-09 15:51 ` Gilles Chanteperdrix
2015-07-10 9:34 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-09 15:51 UTC (permalink / raw)
To: Johann Obermayr; +Cc: Xenomai
On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
> Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
> >On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
> >>Hello,
> >>
> >>we have follow situation
> >>
> >>u64 lrtdrv_time_of_irq_ns ;
> >>void worker_task()
> >>{
> >>while(1)
> >> {
> >> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
> >>RTDM_TIMERMODE_ABSOLUTE);
> >> do_something();
> >> }
> >>}
> >>
> >>_kernel_rtdm_irq_handler()
> >>{
> >> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
> >>}
> >>
> >>the _kernel_irq_callback() is called every 1ms.
> >>we will , that the worker_task begin 50us before next irq
> >>
> >>But sometime, the worker task start ~50us after irq. Why ?
> >Best way to know, enable the I-pipe tracer, setup sufficient back
> >trace points, and when the wrong wake up happens, trigger an I-pipe
> >trace.
> >
> i will enable i-pipe tracer.
> found out, that
>
> rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
> u64 calc = rtdm_clock_read_monotonic();
>
> sometime the diff between xxx & calc is biger than 61000ns.
This is called "kernel task scheduling latency". It can be measured by
the latency -t 1 test.
>
> forget to say.
> worker_task run on Core1
> irq_handler run on Core0
Are you sure that the tsc is synchronized on both cores (you did not
give us much information, in particular we do not know on what
hardware you have this issue).
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-09 15:51 ` Gilles Chanteperdrix
@ 2015-07-10 9:34 ` Johann Obermayr
2015-07-10 9:46 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-10 9:34 UTC (permalink / raw)
To: Xenomai
Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
> On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
>> Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
>>> On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
>>>> Hello,
>>>>
>>>> we have follow situation
>>>>
>>>> u64 lrtdrv_time_of_irq_ns ;
>>>> void worker_task()
>>>> {
>>>> while(1)
>>>> {
>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
>>>> RTDM_TIMERMODE_ABSOLUTE);
>>>> do_something();
>>>> }
>>>> }
>>>>
>>>> _kernel_rtdm_irq_handler()
>>>> {
>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
>>>> }
>>>>
>>>> the _kernel_irq_callback() is called every 1ms.
>>>> we will , that the worker_task begin 50us before next irq
>>>>
>>>> But sometime, the worker task start ~50us after irq. Why ?
>>> Best way to know, enable the I-pipe tracer, setup sufficient back
>>> trace points, and when the wrong wake up happens, trigger an I-pipe
>>> trace.
>>>
>> i will enable i-pipe tracer.
>> found out, that
>>
>> rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
>> u64 calc = rtdm_clock_read_monotonic();
>>
>> sometime the diff between xxx & calc is biger than 61000ns.
> This is called "kernel task scheduling latency". It can be measured by
> the latency -t 1 test.
Ok, i will test it with this test.
Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
rtdm_clock_read_monotonic
Need = xxx - calc;
Need: -1189
Need: -1433
Need: -1703
Need: -2244
Need: -2330
Need: -2295
Need: -2321
Need: -2252
Need: -2313
Need: -2264
Need: -2306
Need: -2267
Need: -2280
Need: -2245
Need: 27177
Need: 26705
Need: 27497
Need: -2250
Need: -2230
Need: -481
Need: -1723
Need: -1412
Need: -716
Need: -1662
Need: -2369
Need: -2384
Need: -585
Need: -1407
Need: -769
Need: -2253
Need: 26571
Need: 87642
Need: 28411
Need: 41489
Need: -2264
Need: -1638
Need: -435
Need: -492
Need: -2424
Need: -1658
Need: -686
Need: -552
Need: -746
Need: -780
Need: -1655
Need: -1435
Need: -2234
>> forget to say.
>> worker_task run on Core1
>> irq_handler run on Core0
> Are you sure that the tsc is synchronized on both cores (you did not
> give us much information, in particular we do not know on what
> hardware you have this issue).
>
tsc must be sync, because most time it is correct.
It's a Fujitsu Mainboard with Interl 8 Series C220 with dual core
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 60
model name : Intel(R) Celeron(R) CPU G1820 @ 2.70GHz
stepping : 3
microcode : 0x1c
cpu MHz : 2693.915
cache size : 2048 KB
Regards
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-10 9:34 ` Johann Obermayr
@ 2015-07-10 9:46 ` Gilles Chanteperdrix
2015-07-10 10:43 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-10 9:46 UTC (permalink / raw)
To: Johann Obermayr; +Cc: Xenomai
On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
> Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
> >On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
> >>Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
> >>>On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
> >>>>Hello,
> >>>>
> >>>>we have follow situation
> >>>>
> >>>>u64 lrtdrv_time_of_irq_ns ;
> >>>>void worker_task()
> >>>>{
> >>>>while(1)
> >>>> {
> >>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
> >>>>RTDM_TIMERMODE_ABSOLUTE);
> >>>> do_something();
> >>>> }
> >>>>}
> >>>>
> >>>>_kernel_rtdm_irq_handler()
> >>>>{
> >>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
> >>>>}
> >>>>
> >>>>the _kernel_irq_callback() is called every 1ms.
> >>>>we will , that the worker_task begin 50us before next irq
> >>>>
> >>>>But sometime, the worker task start ~50us after irq. Why ?
> >>>Best way to know, enable the I-pipe tracer, setup sufficient back
> >>>trace points, and when the wrong wake up happens, trigger an I-pipe
> >>>trace.
> >>>
> >>i will enable i-pipe tracer.
> >>found out, that
> >>
> >>rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
> >>u64 calc = rtdm_clock_read_monotonic();
> >>
> >>sometime the diff between xxx & calc is biger than 61000ns.
> >This is called "kernel task scheduling latency". It can be measured by
> >the latency -t 1 test.
>
> Ok, i will test it with this test.
> Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
> rtdm_clock_read_monotonic
Yeah, latency -t 1 does that too. Only much better.
> >Are you sure that the tsc is synchronized on both cores (you did not
> >give us much information, in particular we do not know on what
> >hardware you have this issue).
> >
> tsc must be sync, because most time it is correct.
Bad reasoning. The time may be good when the interrupt happens on
the same core as the one where the waiting task is running, and bad
when both do not happen on the same cpu.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-10 9:46 ` Gilles Chanteperdrix
@ 2015-07-10 10:43 ` Johann Obermayr
2015-07-10 10:49 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-10 10:43 UTC (permalink / raw)
To: Xenomai
Am 10.07.2015 um 11:46 schrieb Gilles Chanteperdrix:
> On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
>> Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
>>> On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
>>>> Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
>>>>> On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
>>>>>> Hello,
>>>>>>
>>>>>> we have follow situation
>>>>>>
>>>>>> u64 lrtdrv_time_of_irq_ns ;
>>>>>> void worker_task()
>>>>>> {
>>>>>> while(1)
>>>>>> {
>>>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
>>>>>> RTDM_TIMERMODE_ABSOLUTE);
>>>>>> do_something();
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> _kernel_rtdm_irq_handler()
>>>>>> {
>>>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
>>>>>> }
>>>>>>
>>>>>> the _kernel_irq_callback() is called every 1ms.
>>>>>> we will , that the worker_task begin 50us before next irq
>>>>>>
>>>>>> But sometime, the worker task start ~50us after irq. Why ?
>>>>> Best way to know, enable the I-pipe tracer, setup sufficient back
>>>>> trace points, and when the wrong wake up happens, trigger an I-pipe
>>>>> trace.
>>>>>
>>>> i will enable i-pipe tracer.
>>>> found out, that
>>>>
>>>> rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
>>>> u64 calc = rtdm_clock_read_monotonic();
>>>>
>>>> sometime the diff between xxx & calc is biger than 61000ns.
>>> This is called "kernel task scheduling latency". It can be measured by
>>> the latency -t 1 test.
>> Ok, i will test it with this test.
>> Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
>> rtdm_clock_read_monotonic
> Yeah, latency -t 1 does that too. Only much better.
>
Result a) only latency -t 1
RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat
best|--lat worst
RTD| -2.618| -2.481| -2.384| 0| 0| -2.618| -2.384
RTD| -2.506| -2.477| -2.301| 0| 0| -2.618| -2.301
RTD| -2.590| -2.479| -2.192| 0| 0| -2.618| -2.192
RTD| -2.517| -2.489| -2.324| 0| 0| -2.618| -2.192
RTD| -2.612| -2.490| -2.335| 0| 0| -2.618| -2.192
RTD| -2.723| -2.490| -2.300| 0| 0| -2.723| -2.192
RTD| -2.521| -2.491| -2.332| 0| 0| -2.723| -2.192
RTD| -2.624| -2.492| -2.346| 0| 0| -2.723| -2.192
RTD| -2.520| -2.490| -2.341| 0| 0| -2.723| -2.192
RTD| -2.584| -2.490| -2.353| 0| 0| -2.723| -2.192
RTD| -2.521| -2.487| -2.333| 0| 0| -2.723| -2.192
RTD| -2.521| -2.490| -2.329| 0| 0| -2.723| -2.192
RTD| -2.519| -2.488| -1.946| 0| 0| -2.723| -1.946
RTD| -2.521| -2.490| -2.353| 0| 0| -2.723| -1.946
RTD| -2.605| -2.490| -2.343| 0| 0| -2.723| -1.946
RTD| -2.521| -2.490| -2.328| 0| 0| -2.723| -1.946
RTD| -2.568| -2.489| -2.337| 0| 0| -2.723| -1.946
RTD| -2.520| -2.490| -2.329| 0| 0| -2.723| -1.946
RTD| -2.553| -2.491| -2.337| 0| 0| -2.723| -1.946
RTD| -2.521| -2.491| -2.325| 0| 0| -2.723| -1.946
RTD| -2.521| -2.490| -2.341| 0| 0| -2.723| -1.946
Result b) with our application, that copy with a releaxed task from/to a
SRAM on a PCI Card.
RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
RTH|---lat min|---lat avg|----lat max|-overrun|---msw|---lat best|--lat
worst
RTD| -2.680| -1.537| 716.496| 38| 0| -2.680| 716.496
RTD| -2.680| -1.414| 691.232| 81| 0| -2.680| 716.496
RTD| -3.051| -0.877| 739.939| 120| 0| -3.051| 739.939
RTD| -2.662| -1.422| 738.747| 160| 0| -3.051| 739.939
RTD| -2.713| -1.637| 726.880| 191| 0| -3.051| 739.939
RTD| -2.656| -1.044| 675.880| 218| 0| -3.051| 739.939
RTD| -2.655| -1.269| 753.354| 251| 0| -3.051| 753.354
RTD| -2.661| -1.344| 633.684| 301| 0| -3.051| 753.354
RTD| -2.663| -1.626| 698.008| 333| 0| -3.051| 753.354
RTD| -2.662| -0.579| 751.352| 379| 0| -3.051| 753.354
RTD| -2.669| -1.455| 733.275| 410| 0| -3.051| 753.354
RTD| -2.758| -1.670| 629.569| 434| 0| -3.051| 753.354
RTD| -2.760| -1.686| 690.451| 445| 0| -3.051| 753.354
RTD| -2.668| -1.032| 753.099| 483| 0| -3.051| 753.354
RTD| -2.661| -1.476| 735.519| 518| 0| -3.051| 753.354
RTD| -2.660| -1.647| 725.828| 552| 0| -3.051| 753.354
RTD| -2.661| -1.236| 250.879| 556| 0| -3.051| 753.354
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-10 10:43 ` Johann Obermayr
@ 2015-07-10 10:49 ` Gilles Chanteperdrix
2015-07-13 13:41 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-10 10:49 UTC (permalink / raw)
To: Johann Obermayr; +Cc: Xenomai
On Fri, Jul 10, 2015 at 12:43:29PM +0200, Johann Obermayr wrote:
> Am 10.07.2015 um 11:46 schrieb Gilles Chanteperdrix:
> >On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
> >>Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
> >>>On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
> >>>>Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
> >>>>>On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
> >>>>>>Hello,
> >>>>>>
> >>>>>>we have follow situation
> >>>>>>
> >>>>>>u64 lrtdrv_time_of_irq_ns ;
> >>>>>>void worker_task()
> >>>>>>{
> >>>>>>while(1)
> >>>>>> {
> >>>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
> >>>>>>RTDM_TIMERMODE_ABSOLUTE);
> >>>>>> do_something();
> >>>>>> }
> >>>>>>}
> >>>>>>
> >>>>>>_kernel_rtdm_irq_handler()
> >>>>>>{
> >>>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
> >>>>>>}
> >>>>>>
> >>>>>>the _kernel_irq_callback() is called every 1ms.
> >>>>>>we will , that the worker_task begin 50us before next irq
> >>>>>>
> >>>>>>But sometime, the worker task start ~50us after irq. Why ?
> >>>>>Best way to know, enable the I-pipe tracer, setup sufficient back
> >>>>>trace points, and when the wrong wake up happens, trigger an I-pipe
> >>>>>trace.
> >>>>>
> >>>>i will enable i-pipe tracer.
> >>>>found out, that
> >>>>
> >>>>rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
> >>>>u64 calc = rtdm_clock_read_monotonic();
> >>>>
> >>>>sometime the diff between xxx & calc is biger than 61000ns.
> >>>This is called "kernel task scheduling latency". It can be measured by
> >>>the latency -t 1 test.
> >>Ok, i will test it with this test.
> >>Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
> >>rtdm_clock_read_monotonic
> >Yeah, latency -t 1 does that too. Only much better.
> >
> Result a) only latency -t 1
> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
> RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat
> worst
> RTD| -2.618| -2.481| -2.384| 0| 0| -2.618| -2.384
> RTD| -2.506| -2.477| -2.301| 0| 0| -2.618| -2.301
> RTD| -2.590| -2.479| -2.192| 0| 0| -2.618| -2.192
> RTD| -2.517| -2.489| -2.324| 0| 0| -2.618| -2.192
> RTD| -2.612| -2.490| -2.335| 0| 0| -2.618| -2.192
> RTD| -2.723| -2.490| -2.300| 0| 0| -2.723| -2.192
> RTD| -2.521| -2.491| -2.332| 0| 0| -2.723| -2.192
> RTD| -2.624| -2.492| -2.346| 0| 0| -2.723| -2.192
> RTD| -2.520| -2.490| -2.341| 0| 0| -2.723| -2.192
> RTD| -2.584| -2.490| -2.353| 0| 0| -2.723| -2.192
> RTD| -2.521| -2.487| -2.333| 0| 0| -2.723| -2.192
> RTD| -2.521| -2.490| -2.329| 0| 0| -2.723| -2.192
> RTD| -2.519| -2.488| -1.946| 0| 0| -2.723| -1.946
> RTD| -2.521| -2.490| -2.353| 0| 0| -2.723| -1.946
> RTD| -2.605| -2.490| -2.343| 0| 0| -2.723| -1.946
> RTD| -2.521| -2.490| -2.328| 0| 0| -2.723| -1.946
> RTD| -2.568| -2.489| -2.337| 0| 0| -2.723| -1.946
> RTD| -2.520| -2.490| -2.329| 0| 0| -2.723| -1.946
> RTD| -2.553| -2.491| -2.337| 0| 0| -2.723| -1.946
> RTD| -2.521| -2.491| -2.325| 0| 0| -2.723| -1.946
> RTD| -2.521| -2.490| -2.341| 0| 0| -2.723| -1.946
>
> Result b) with our application, that copy with a releaxed task from/to a
> SRAM on a PCI Card.
> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
> RTH|---lat min|---lat avg|----lat max|-overrun|---msw|---lat best|--lat
> worst
> RTD| -2.680| -1.537| 716.496| 38| 0| -2.680| 716.496
> RTD| -2.680| -1.414| 691.232| 81| 0| -2.680| 716.496
> RTD| -3.051| -0.877| 739.939| 120| 0| -3.051| 739.939
> RTD| -2.662| -1.422| 738.747| 160| 0| -3.051| 739.939
> RTD| -2.713| -1.637| 726.880| 191| 0| -3.051| 739.939
> RTD| -2.656| -1.044| 675.880| 218| 0| -3.051| 739.939
> RTD| -2.655| -1.269| 753.354| 251| 0| -3.051| 753.354
> RTD| -2.661| -1.344| 633.684| 301| 0| -3.051| 753.354
> RTD| -2.663| -1.626| 698.008| 333| 0| -3.051| 753.354
> RTD| -2.662| -0.579| 751.352| 379| 0| -3.051| 753.354
> RTD| -2.669| -1.455| 733.275| 410| 0| -3.051| 753.354
> RTD| -2.758| -1.670| 629.569| 434| 0| -3.051| 753.354
> RTD| -2.760| -1.686| 690.451| 445| 0| -3.051| 753.354
> RTD| -2.668| -1.032| 753.099| 483| 0| -3.051| 753.354
> RTD| -2.661| -1.476| 735.519| 518| 0| -3.051| 753.354
> RTD| -2.660| -1.647| 725.828| 552| 0| -3.051| 753.354
> RTD| -2.661| -1.236| 250.879| 556| 0| -3.051| 753.354
Your application (and more probably driver) has a problem, it
probably creates large interrupts masking section. Once again, to
understand what the problem is, I suggest using the I-pipe tracer.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-10 10:49 ` Gilles Chanteperdrix
@ 2015-07-13 13:41 ` Johann Obermayr
2015-07-13 14:02 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 13:41 UTC (permalink / raw)
To: Xenomai
Am 10.07.2015 um 12:49 schrieb Gilles Chanteperdrix:
> On Fri, Jul 10, 2015 at 12:43:29PM +0200, Johann Obermayr wrote:
>> Am 10.07.2015 um 11:46 schrieb Gilles Chanteperdrix:
>>> On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
>>>> Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
>>>>> On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
>>>>>> Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
>>>>>>> On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> we have follow situation
>>>>>>>>
>>>>>>>> u64 lrtdrv_time_of_irq_ns ;
>>>>>>>> void worker_task()
>>>>>>>> {
>>>>>>>> while(1)
>>>>>>>> {
>>>>>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
>>>>>>>> RTDM_TIMERMODE_ABSOLUTE);
>>>>>>>> do_something();
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> _kernel_rtdm_irq_handler()
>>>>>>>> {
>>>>>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
>>>>>>>> }
>>>>>>>>
>>>>>>>> the _kernel_irq_callback() is called every 1ms.
>>>>>>>> we will , that the worker_task begin 50us before next irq
>>>>>>>>
>>>>>>>> But sometime, the worker task start ~50us after irq. Why ?
>>>>>>> Best way to know, enable the I-pipe tracer, setup sufficient back
>>>>>>> trace points, and when the wrong wake up happens, trigger an I-pipe
>>>>>>> trace.
>>>>>>>
>>>>>> i will enable i-pipe tracer.
>>>>>> found out, that
>>>>>>
>>>>>> rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
>>>>>> u64 calc = rtdm_clock_read_monotonic();
>>>>>>
>>>>>> sometime the diff between xxx & calc is biger than 61000ns.
>>>>> This is called "kernel task scheduling latency". It can be measured by
>>>>> the latency -t 1 test.
>>>> Ok, i will test it with this test.
>>>> Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
>>>> rtdm_clock_read_monotonic
>>> Yeah, latency -t 1 does that too. Only much better.
>>>
>> Result a) only latency -t 1
>> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
>> RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat
>> worst
>> RTD| -2.618| -2.481| -2.384| 0| 0| -2.618| -2.384
>> RTD| -2.506| -2.477| -2.301| 0| 0| -2.618| -2.301
>> RTD| -2.590| -2.479| -2.192| 0| 0| -2.618| -2.192
>> RTD| -2.517| -2.489| -2.324| 0| 0| -2.618| -2.192
>> RTD| -2.612| -2.490| -2.335| 0| 0| -2.618| -2.192
>> RTD| -2.723| -2.490| -2.300| 0| 0| -2.723| -2.192
>> RTD| -2.521| -2.491| -2.332| 0| 0| -2.723| -2.192
>> RTD| -2.624| -2.492| -2.346| 0| 0| -2.723| -2.192
>> RTD| -2.520| -2.490| -2.341| 0| 0| -2.723| -2.192
>> RTD| -2.584| -2.490| -2.353| 0| 0| -2.723| -2.192
>> RTD| -2.521| -2.487| -2.333| 0| 0| -2.723| -2.192
>> RTD| -2.521| -2.490| -2.329| 0| 0| -2.723| -2.192
>> RTD| -2.519| -2.488| -1.946| 0| 0| -2.723| -1.946
>> RTD| -2.521| -2.490| -2.353| 0| 0| -2.723| -1.946
>> RTD| -2.605| -2.490| -2.343| 0| 0| -2.723| -1.946
>> RTD| -2.521| -2.490| -2.328| 0| 0| -2.723| -1.946
>> RTD| -2.568| -2.489| -2.337| 0| 0| -2.723| -1.946
>> RTD| -2.520| -2.490| -2.329| 0| 0| -2.723| -1.946
>> RTD| -2.553| -2.491| -2.337| 0| 0| -2.723| -1.946
>> RTD| -2.521| -2.491| -2.325| 0| 0| -2.723| -1.946
>> RTD| -2.521| -2.490| -2.341| 0| 0| -2.723| -1.946
>>
>> Result b) with our application, that copy with a releaxed task from/to a
>> SRAM on a PCI Card.
>> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
>> RTH|---lat min|---lat avg|----lat max|-overrun|---msw|---lat best|--lat
>> worst
>> RTD| -2.680| -1.537| 716.496| 38| 0| -2.680| 716.496
>> RTD| -2.680| -1.414| 691.232| 81| 0| -2.680| 716.496
>> RTD| -3.051| -0.877| 739.939| 120| 0| -3.051| 739.939
>> RTD| -2.662| -1.422| 738.747| 160| 0| -3.051| 739.939
>> RTD| -2.713| -1.637| 726.880| 191| 0| -3.051| 739.939
>> RTD| -2.656| -1.044| 675.880| 218| 0| -3.051| 739.939
>> RTD| -2.655| -1.269| 753.354| 251| 0| -3.051| 753.354
>> RTD| -2.661| -1.344| 633.684| 301| 0| -3.051| 753.354
>> RTD| -2.663| -1.626| 698.008| 333| 0| -3.051| 753.354
>> RTD| -2.662| -0.579| 751.352| 379| 0| -3.051| 753.354
>> RTD| -2.669| -1.455| 733.275| 410| 0| -3.051| 753.354
>> RTD| -2.758| -1.670| 629.569| 434| 0| -3.051| 753.354
>> RTD| -2.760| -1.686| 690.451| 445| 0| -3.051| 753.354
>> RTD| -2.668| -1.032| 753.099| 483| 0| -3.051| 753.354
>> RTD| -2.661| -1.476| 735.519| 518| 0| -3.051| 753.354
>> RTD| -2.660| -1.647| 725.828| 552| 0| -3.051| 753.354
>> RTD| -2.661| -1.236| 250.879| 556| 0| -3.051| 753.354
> Your application (and more probably driver) has a problem, it
> probably creates large interrupts masking section. Once again, to
> understand what the problem is, I suggest using the I-pipe tracer.
>
Hello,
i have add a ipipe trace. But i don't understand it.
the function lrtdrv_monitoring_irq has no mutex, only a atomic funtion
and write result into a ring buffer.
(see your trace screen. red lines)
http://www.pic-upload.de/view-27670244/Trace001.png.html
-------------- next part --------------
I-pipe frozen back-tracing service on 3.10.0/ipipe release #1
------------------------------------------------------------
CPU: 0, Freeze: 1294584251633 cycles, Trace Points: 2000 (+10)
Calibrated minimum trace-point overhead: 0.078 us
+----- Hard IRQs ('|': locked)
|+-- Xenomai
||+- Linux ('*': domain stalled, '+': current, '#': current+stalled)
||| +---------- Delay flag ('+': > 1 us, '!': > 10 us)
||| | +- NMI noise ('N')
||| | |
Type User Val. Time Delay Function (Parent)
:| #func -1927 0.178 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #end 0x80000001 -1927 0.133 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
:| #begin 0x80000001 -1927 0.127 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -1926 0.138 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -1926 0.252 __schedule+0x3a3 (schedule+0x18)
:| #func -1926 0.374 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #func -1926 0.135 ipipe_unstall_head+0x4 (xnshadow_relax+0x91)
:| #end 0x80000000 -1926 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_head+0x2e)
: #func -1925 0.124 __ipipe_reenter_root+0x4 (xnshadow_relax+0xa2)
:| #begin 0x80000001 -1925 0.141 debug_smp_processor_id+0x36 (__ipipe_reenter_root+0x13)
:| #end 0x80000001 -1925 0.129 debug_smp_processor_id+0x82 (__ipipe_reenter_root+0x13)
: #func -1925 0.114 finish_task_switch+0x7 (__ipipe_reenter_root+0x28)
: #func -1925 0.126 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1925 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1925 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1925 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1924 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1924 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1924 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1924 0.256 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1924 0.175 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1924 0.115 sub_preempt_count+0x4 (__ipipe_reenter_root+0x42)
: +func -1923 0.126 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1923 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1923 0.181 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1923 0.210 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -1923 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -1923 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -1923 0.155 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -1922 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -1922 0.118 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| +begin 0x80000001 -1922 0.181 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -1922 0.147 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +begin 0x80000000 -1922 0.179 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +end 0x80000000 -1922 0.139 __ipipe_syscall_root+0x9e (system_call+0x30)
: +func -1922 0.112 SyS_select+0x9 (syscall_call+0x7)
: +func -1921 0.115 _copy_from_user+0x7 (SyS_select+0x39)
: +func -1921 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -1921 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1921 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1921 0.163 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -1921 0.121 poll_select_set_timeout+0x6 (SyS_select+0x60)
: +func -1921 0.118 core_sys_select+0xc (SyS_select+0x80)
: +func -1920 0.130 __rcu_read_lock+0x3 (core_sys_select+0x31)
: +func -1920 0.126 __rcu_read_unlock+0x4 (core_sys_select+0x47)
: +func -1920 0.112 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -1920 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -1920 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1920 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1920 0.132 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -1920 0.117 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -1919 0.117 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -1919 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1919 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1919 0.184 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -1919 0.115 do_select+0xc (core_sys_select+0x12b)
: +func -1919 0.144 __rcu_read_lock+0x3 (do_select+0x23)
: +func -1919 0.118 __rcu_read_unlock+0x4 (do_select+0xe5)
: +func -1918 0.163 poll_initwait+0x3 (do_select+0xfd)
: +func -1918 0.118 fget_light+0x6 (do_select+0x2aa)
: +func -1918 0.136 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1918 0.135 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1918 0.112 eventfd_poll+0x6 (do_select+0x30f)
: +func -1918 0.154 _raw_spin_lock_irqsave+0x5 (eventfd_poll+0x2f)
: #func -1918 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1918 0.126 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1917 0.154 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1917 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1917 0.123 __ipipe_spin_unlock_debug+0x3 (eventfd_poll+0x68)
: #func -1917 0.132 _raw_spin_unlock_irqrestore+0x4 (eventfd_poll+0x71)
: #func -1917 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1917 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1917 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1916 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1916 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1916 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1916 0.190 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1916 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1916 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1916 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1916 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1915 0.136 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1915 0.164 fput+0x5 (do_select+0x327)
: +func -1915 0.114 fget_light+0x6 (do_select+0x2aa)
: +func -1915 0.142 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1915 0.144 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1915 0.138 sock_poll+0x5 (do_select+0x30f)
: +func -1915 0.132 tcp_poll+0x6 (sock_poll+0x17)
: +func -1914 0.127 fput+0x5 (do_select+0x327)
: +func -1914 0.142 _cond_resched+0x3 (do_select+0x3f1)
: +func -1914 0.130 poll_freewait+0x9 (do_select+0x4ab)
: +func -1914 0.130 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -1914 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1914 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1914 0.139 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -1913 0.118 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -1913 0.178 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1913 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1913 0.142 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -1913! 43.981 poll_select_copy_remaining+0x9 (SyS_select+0x90)
: +func -1869 0.114 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +func -1869 0.118 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +begin 0x80000001 -1869 0.198 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1868 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1868 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1868 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1868 0.136 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1868 0.136 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| + begin 0x80000001 -1868 0.221 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -1868 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -1867 0.117 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -1867 0.155 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -1867 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -1867 0.151 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +func -1867 0.121 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: +func -1867 0.112 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: +func -1867 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| +begin 0x80000000 -1867 0.212 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1866 0.141 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1866 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000001 -1866 0.142 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| +end 0x80000001 -1866 0.142 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: +func -1866 0.115 lrt_non_rt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x51)
: +func -1866 0.187 lrt_non_rt_ioctl_work+0xc [sigmatek_lrt] (lrt_non_rt_ioctl+0x10 [sigmatek_lrt])
: +func -1865 0.120 xnshadow_harden+0x9 (losyscall_event+0x88)
: +func -1865 0.114 down_interruptible+0x6 (xnshadow_harden+0x4f)
: +func -1865 0.144 _raw_spin_lock_irqsave+0x5 (down_interruptible+0x12)
: #func -1865 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1865 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1865 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1865 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1865 0.130 __ipipe_spin_unlock_debug+0x3 (down_interruptible+0x33)
: #func -1864 0.124 _raw_spin_unlock_irqrestore+0x4 (down_interruptible+0x3c)
: #func -1864 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1864 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1864 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1864 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1864 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1864 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1863 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1863 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1863 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1863 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1863 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1863 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1863 0.112 add_preempt_count+0x4 (xnshadow_harden+0x76)
: +func -1863 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1862 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1862 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1862 0.123 wake_up_process+0x4 (xnshadow_harden+0xe1)
: +func -1862 0.112 try_to_wake_up+0x9 (wake_up_process+0x2e)
: +func -1862 0.142 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -1862 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1862 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1861 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1861 0.164 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1861 0.219 select_task_rq_rt+0x6 (select_task_rq+0x15)
:| #begin 0x80000001 -1861 0.142 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -1861 0.129 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -1861 0.118 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -1861 0.111 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -1860 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1860 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1860 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1860 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1860 0.115 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -1860 0.117 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -1860 0.115 enqueue_task+0x8 (activate_task+0x21)
: #func -1860 0.139 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -1859 0.154 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1859 0.147 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1859 0.112 enqueue_task_rt+0x7 (enqueue_task+0x28)
: #func -1859 0.130 dequeue_rt_stack+0x9 (enqueue_task_rt+0x2c)
: #func -1859 0.158 cpupri_set+0x6 (enqueue_task_rt+0xca)
: #func -1859 0.124 update_rt_migration+0x4 (enqueue_task_rt+0xea)
: #func -1859 0.115 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -1858 0.187 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -1858 0.120 check_preempt_curr_rt+0x5 (check_preempt_curr+0x2c)
: #func -1858 0.139 resched_task+0x5 (check_preempt_curr_rt+0x82)
:| #begin 0x80000001 -1858 0.142 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -1858 0.135 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -1858 0.117 task_woken_rt+0x4 (ttwu_do_wakeup+0x5b)
: #func -1858 0.123 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -1858 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1857 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1857 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1857 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1857 0.130 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -1857 0.121 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -1857 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1857 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1856 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1856 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1856 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1856 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1856 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1856 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1856 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1856 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1855 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1855 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1855 0.123 schedule+0x3 (xnshadow_harden+0xe6)
: +func -1855 0.112 __schedule+0x9 (schedule+0x18)
: +func -1855 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -1855 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1855 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1854 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1854 0.152 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1854 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1854 0.115 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1854 0.144 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1854 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -1854 0.111 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1854 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1853 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1853 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1853 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1853 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1853 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1853 0.130 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1853 0.118 sub_preempt_count+0x4 (__schedule+0x54)
: +func -1852 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1852 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1852 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1852 0.117 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1852 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1852 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1852 0.127 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1851 0.117 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1851 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1851 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1851 0.147 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1851 0.112 deactivate_task+0x4 (__schedule+0x10e)
: #func -1851 0.115 dequeue_task+0x8 (deactivate_task+0x21)
: #func -1851 0.190 update_rq_clock+0x6 (dequeue_task+0x1a)
: #func -1850 0.120 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -1850 0.124 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -1850 0.120 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -1850 0.144 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -1850 0.115 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -1850 0.127 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -1850 0.136 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -1850 0.123 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -1849 0.126 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -1849 0.123 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -1849 0.118 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -1849 0.132 dequeue_pushable_task+0x5 (pick_next_task_rt+0x5b)
: #func -1849 0.118 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1849 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1849 0.120 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1849 0.213 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1848 0.155 rt_task_set_mode+0x7 (taskSwitchLinux+0xbf [sigmatek_lrt])
: #func -1848 0.115 __ipipe_notify_kevent+0x9 (__schedule+0x391)
: #func -1848 0.115 ipipe_root_only+0x5 (__ipipe_notify_kevent+0x1b)
:| #begin 0x80000001 -1848 0.154 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1848 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1848 0.144 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x20)
:| #end 0x80000001 -1848 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x49)
: #func -1847 0.115 ipipe_kevent_hook+0x4 (__ipipe_notify_kevent+0x54)
: #func -1847 0.139 schedule_event+0x9 (ipipe_kevent_hook+0x1c)
:| #begin 0x80000000 -1847 0.161 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *#func -1847 0.161 xnsched_peek_rpi+0x3 (schedule_event+0x14c)
:| *#func -1847 0.145 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1847 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| #begin 0x80000001 -1847 0.127 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -1846 0.138 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -1846 0.155 __schedule+0x3a3 (schedule+0x18)
:| #func -1846 0.233 __switch_to+0x9 (__schedule+0x50a)
:| #func -1846 0.139 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1846 0.133 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1846 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1846 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1845 0.114 finish_task_switch+0x7 (__schedule+0x533)
: #func -1845 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1845 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1845 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1845 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1845 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1845 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1844 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1844 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1844 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1844 0.145 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1844 0.126 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1844 0.115 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1844 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1844 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1843 0.203 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1843 0.132 kthread_should_stop+0x3 (gatekeeper_thread+0xc3)
: +func -1843 0.117 rpi_pop+0x9 (gatekeeper_thread+0xf0)
:| +begin 0x80000000 -1843 0.170 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1843 0.176 xnsched_peek_rpi+0x3 (rpi_pop+0x6e)
:| *+func -1843 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1842 0.117 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -1842 0.120 xnsched_renice_root+0x7 (rpi_pop+0xb4)
:| +begin 0x80000000 -1842 0.210 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *+func -1842 0.126 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *+[ 0] -<?>- -1 -1842 0.145 xnsched_renice_root+0x82 (rpi_pop+0xb4)
:| *+func -1842 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| +end 0x80000000 -1842 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000000 -1841 0.157 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1841 0.145 xnpod_resume_thread+0x9 (gatekeeper_thread+0x11a)
:| *+[ 582] cyclic: 30 -1841 0.185 xnpod_resume_thread+0x39 (gatekeeper_thread+0x11a)
:| *+func -1841 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1841 0.121 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -1841 0.120 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| +begin 0x80000001 -1841 0.142 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| +end 0x80000001 -1840 0.132 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: +func -1840 0.120 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| +begin 0x80000001 -1840 0.141 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| +func -1840 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| +func -1840 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1840 0.129 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| +func -1840 0.120 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1840 0.163 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1839 0.147 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1839 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1839 0.203 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1839 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1839 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1839 0.135 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| # func -1839 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1838 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1838 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1838 0.301 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1838 0.253 __switch_to+0x9 (__schedule+0x50a)
:| # func -1838 0.190 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| # func -1837 0.234 xnpod_switch_fpu+0x3 (xnshadow_harden+0x133)
:| # func -1837 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1837 0.133 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1837 0.120 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: + func -1837 0.124 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1837 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1836 0.210 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1836 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1836 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1836 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1836 0.147 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1836 0.120 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1836 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1835 0.151 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -1835 0.126 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -1835 0.120 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1835 0.118 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1835 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -1835 0.147 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -1835 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -1835 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1834 0.123 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1834 0.154 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1834 0.172 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1834 0.144 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1834 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 582] cyclic: 30 -1834 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1834 0.262 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1833 0.317 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 23] gatekee -1 -1833 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1833 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1833 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1833 0.120 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1832 0.310 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1832 0.188 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1832 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1832 0.198 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x80000001 -1832 0.170 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
: +func -1831 0.118 up+0x5 (gatekeeper_thread+0xb9)
: +func -1831 0.141 _raw_spin_lock_irqsave+0x5 (up+0x11)
: #func -1831 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1831 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1831 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1831 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1831 0.123 __ipipe_spin_unlock_debug+0x3 (up+0x2e)
: #func -1831 0.121 _raw_spin_unlock_irqrestore+0x4 (up+0x37)
: #func -1830 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1830 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1830 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1830 0.126 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1830 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1830 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1830 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1829 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1829 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1829 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1829 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1829 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1829 0.115 schedule+0x3 (gatekeeper_thread+0xbe)
: +func -1829 0.115 __schedule+0x9 (schedule+0x18)
: +func -1829 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -1828 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1828 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1828 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1828 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1828 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1828 0.115 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1828 0.144 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1828 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -1827 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1827 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1827 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1827 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1827 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1827 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1827 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1826 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1826 0.117 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1826 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1826 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1826 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1826 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1826 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1826 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1825 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1825 0.114 deactivate_task+0x4 (__schedule+0x10e)
: #func -1825 0.114 dequeue_task+0x8 (deactivate_task+0x21)
: #func -1825 0.149 update_rq_clock+0x6 (dequeue_task+0x1a)
:| #begin 0x80000001 -1825 0.141 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1825 0.144 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1825 0.112 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -1824 0.127 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -1824 0.118 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -1824 0.138 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -1824 0.158 cpupri_set+0x6 (dequeue_rt_stack+0x104)
: #func -1824 0.120 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -1824 0.124 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -1824 0.111 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -1824 0.121 pull_rt_task+0x9 (pre_schedule_rt+0x18)
: #func -1823 0.129 idle_balance+0x9 (__schedule+0x252)
: #func -1823 0.124 _raw_spin_unlock+0x3 (idle_balance+0x5f)
: #func -1823 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1823 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1823 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1823 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1823 0.118 __rcu_read_lock+0x3 (idle_balance+0x64)
: #func -1823 0.112 __rcu_read_unlock+0x4 (idle_balance+0xee)
: #func -1822 0.112 _raw_spin_lock+0x4 (idle_balance+0xf5)
: #func -1822 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1822 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1822 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1822 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1822 0.118 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -1822 0.121 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -1822 0.114 pick_next_task_fair+0x7 (__schedule+0x28c)
: #func -1821 0.117 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -1821 0.115 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -1821 0.135 pick_next_task_fair+0x7 (__schedule+0x299)
: #func -1821 0.133 pick_next_task_idle+0x3 (__schedule+0x299)
: #func -1821 0.147 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1821 0.167 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1821 0.129 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1820 0.201 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1820 0.141 rt_task_set_mode+0x7 (taskSwitchLinux+0xbf [sigmatek_lrt])
:| #begin 0x80000000 -1820 0.161 __schedule+0x3a3 (schedule+0x18)
:| #func -1820 0.166 __switch_to+0x9 (__schedule+0x50a)
:| #func -1820 0.139 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1820 0.126 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1820 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1819 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1819 0.114 finish_task_switch+0x7 (__schedule+0x533)
: #func -1819 0.126 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1819 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1819 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1819 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1819 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1818 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1818 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1818 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1818 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1818 0.142 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1818 0.130 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1818 0.139 _raw_spin_lock_irqsave+0x5 (post_schedule.part.48+0xc)
: #func -1817 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1817 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1817 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1817 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1817 0.114 post_schedule_idle+0x3 (post_schedule.part.48+0x22)
: #func -1817 0.130 __ipipe_spin_unlock_debug+0x3 (post_schedule.part.48+0x29)
: #func -1817 0.123 _raw_spin_unlock_irqrestore+0x4 (post_schedule.part.48+0x32)
: #func -1817 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1816 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1816 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1816 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1816 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1816 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1816 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1816 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1815 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1815 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1815 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1815 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1815 0.114 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1815 0.126 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1815 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1815 0.161 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1814 0.118 add_preempt_count+0x4 (schedule_preempt_disabled+0x21)
: +func -1814 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1814 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1814 0.175 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1814 0.127 tick_nohz_idle_enter+0x4 (cpu_startup_entry+0x24)
: +func -1814 0.124 set_cpu_sd_state_idle+0x4 (tick_nohz_idle_enter+0x37)
:| +begin 0x80000001 -1814 0.142 debug_smp_processor_id+0x36 (set_cpu_sd_state_idle+0xe)
:| +end 0x80000001 -1813 0.126 debug_smp_processor_id+0x82 (set_cpu_sd_state_idle+0xe)
: +func -1813 0.117 __rcu_read_lock+0x3 (set_cpu_sd_state_idle+0x15)
: +func -1813 0.138 __rcu_read_unlock+0x4 (set_cpu_sd_state_idle+0x4d)
:| #begin 0x80000001 -1813 0.142 debug_smp_processor_id+0x36 (tick_nohz_idle_enter+0x46)
:| #end 0x80000001 -1813 0.127 debug_smp_processor_id+0x82 (tick_nohz_idle_enter+0x46)
: #func -1813 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_idle_enter+0x5b)
:| #begin 0x80000001 -1813 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -1812 0.129 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -1812 0.161 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -1812 0.151 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -1812 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -1812 0.149 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1812 0.149 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -1812 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -1811 0.141 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -1811 0.136 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -1811 0.142 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -1811 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -1811 0.133 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -1811 0.118 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -1811 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -1811 0.130 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -1810 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -1810 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1810 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1810 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1810 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1810 0.127 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -1810 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1809 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1809 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1809 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1809 0.123 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -1809 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -1809 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -1809 0.142 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -1809 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1808 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1808 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1808 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1808 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -1808 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -1808 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1808 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1807 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1807 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1807 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1807 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1807 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1807 0.170 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1807 0.130 nohz_balance_enter_idle+0x3 (__tick_nohz_idle_enter+0x24a)
: #func -1806 0.123 calc_load_enter_idle+0x4 (__tick_nohz_idle_enter+0x24f)
:| #begin 0x80000001 -1806 0.142 debug_smp_processor_id+0x36 (calc_load_enter_idle+0x13)
:| #end 0x80000001 -1806 0.132 debug_smp_processor_id+0x82 (calc_load_enter_idle+0x13)
: #func -1806 0.115 hrtimer_start+0x3 (__tick_nohz_idle_enter+0x2e1)
: #func -1806 0.117 __hrtimer_start_range_ns+0x9 (hrtimer_start+0x14)
: #func -1806 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -1806 0.141 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1806 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1805 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1805 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1805 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1805 0.148 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -1805 0.127 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -1805 0.124 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -1805 0.132 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
:| #begin 0x80000001 -1805 0.141 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1804 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1804 0.115 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1804 0.135 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -1804 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -1804 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -1804 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -1804 0.133 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -1803 0.115 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -1803 0.121 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -1803 0.149 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -1803 0.120 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -1803 0.164 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -1803 0.127 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -1803 0.141 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -1802 0.127 xntimer_next_local_shot+0x9 (xntimer_start_aperiodic+0x11a)
:| *#event tick@732804-1802 0.117 xntimer_next_local_shot+0xca (xntimer_start_aperiodic+0x11a)
:| *#func -1802 0.136 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| *#func -1802 0.242 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| *#func -1802 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1802 0.147 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -1802 0.117 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1801 0.127 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1801 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1801 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1801 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1801 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1801 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1801 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1801 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1800 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1800 0.120 ipipe_unstall_root+0x3 (tick_nohz_idle_enter+0x60)
:| #begin 0x80000000 -1800 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1800 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1800 0.130 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x80000001 -1800 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1800 0.151 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1799 0.114 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1799 0.121 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1799 0.147 rcu_idle_enter+0x5 (cpu_startup_entry+0x70)
:| #begin 0x80000001 -1799 0.151 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1799 0.130 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1799 0.121 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1799 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1798 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1798 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1798 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1798 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1798 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1798 0.151 debug_smp_processor_id+0x36 (cpu_startup_entry+0x75)
:| #end 0x80000001 -1798 0.129 debug_smp_processor_id+0x82 (cpu_startup_entry+0x75)
: #func -1797 0.121 ipipe_unstall_root+0x3 (cpu_startup_entry+0x8d)
:| #begin 0x80000000 -1797 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1797 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1797! 728.901 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x00000051 -1068 0.173 common_interrupt+0x39 (need_resched+0x0)
:| +func -1068 0.151 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -1068 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1068 0.246 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1067 0.164 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1067 0.135 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -1067 0.155 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -1067 0.118 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -1067 0.121 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -1067 0.120 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -1067 0.115 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -1066+ 1.348 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -1065 0.117 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -1065 0.132 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -1065 0.130 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -1065+ 1.161 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -1064 0.167 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -1063 0.130 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1063 0.210 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1063 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1063 0.170 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1063 0.142 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1063 0.120 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -1063 0.161 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x5a [sigmatek_lrt])
:| # func -1062+ 1.642 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -1061 0.155 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -1061+ 4.591 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x13d [sigmatek_lrt])
:| # func -1056 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1a4 [sigmatek_lrt])
:| # func -1056 0.184 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1d2 [sigmatek_lrt])
:| # func -1056 0.148 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x1e8 [sigmatek_lrt])
:| # func -1056 0.115 xnarch_get_cpu_time+0x4 (irqhandler_fpga+0x225 [sigmatek_lrt])
:| # func -1055 0.120 xnarch_tsc_to_ns+0x6 (xnarch_get_cpu_time+0x13)
:| # func -1055 0.148 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x245 [sigmatek_lrt])
:| # func -1055 0.120 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x25b [sigmatek_lrt])
:| # func -1055 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x280 [sigmatek_lrt])
:| # func -1055 0.120 period_update+0x4 [sigmatek_lrt] (irqhandler_fpga+0x287 [sigmatek_lrt])
:| # func -1055 0.163 lrtdrv_monitoring_start_timed+0x4 [sigmatek_lrt] (period_update+0x1f [sigmatek_lrt])
:| # func -1055 0.216 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -1054 0.215 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -1054 0.149 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -1054 0.218 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -1054 0.155 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -1054 0.169 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -1053 0.141 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 584] realtim 95 -1053 0.227 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1053 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -1053 0.158 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -1053 0.160 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 467] VARAN_T 30 -1053 0.236 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1052 0.133 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 582] cyclic: 30 -1052 0.155 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1052 0.133 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 580] Loader: 29 -1052 0.239 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1052 0.115 ipipe_raise_irq+0x8 (period_update+0x10f [sigmatek_lrt])
:| # func -1052 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| # func -1051 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1051 0.130 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| # func -1051 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| # func -1051 0.155 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1051 0.136 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| # func -1051 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| # func -1051 0.151 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1051 0.184 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| # func -1050 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| # func -1050 0.191 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1050+ 1.657 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -1048+ 1.634 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -1047 0.152 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -1047 0.123 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -1046 0.194 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x10d [sigmatek_lrt])
:| # func -1046 0.115 rthal_irq_end+0x5 (xnintr_irq_handler+0x14d)
:| # func -1046 0.123 irq_to_desc+0x3 (rthal_irq_end+0x1d)
:| # func -1046 0.120 irq_to_desc+0x3 (rthal_irq_end+0x28)
:| # func -1046 0.120 irq_to_desc+0x3 (rthal_irq_end+0x32)
:| # func -1046 0.115 __ipipe_end_fasteoi_irq+0x3 (rthal_irq_end+0x38)
:| # func -1046 0.115 release_ioapic_irq+0x5 (__ipipe_end_fasteoi_irq+0x16)
:| # func -1046 0.152 __ipipe_spin_lock_irqsave+0x6 (release_ioapic_irq+0x17)
:| # func -1045 0.114 io_apic_modify_irq.isra.9+0x7 (release_ioapic_irq+0x29)
:| # func -1045 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| # func -1045+ 1.346 io_apic_base+0x3 (native_io_apic_read+0x10)
:| # func -1044 0.111 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| # func -1044 0.123 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| # func -1044 0.201 __ipipe_spin_unlock_irqrestore+0x4 (release_ioapic_irq+0x35)
:| # func -1043 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 23] gatekee -1 -1043 0.147 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1043 0.142 ipipe_send_ipi+0x6 (__xnpod_schedule+0x7d)
:| # func -1043 0.178 default_send_IPI_mask_logical+0x6 (ipipe_send_ipi+0x40)
:| # func -1043 0.149 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1043 0.295 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1042 0.325 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 584] realtim 95 -1042 0.193 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1042 0.133 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1042 0.151 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1042 0.405 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1041 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1041 0.178 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1041 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -1041 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1041 0.129 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1040 0.270 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -1040 0.172 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1040 0.151 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1040 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1040! 18.755 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1021 0.120 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1021 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1021 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1021 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1020 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1020 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1020 0.201 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1020 0.144 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1020 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1020 0.114 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1020 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1019 0.262 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1019 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1019 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1019 0.142 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1019 0.129 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1019 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1018 0.142 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1018 0.175 pci_locker_signal_stop+0x3 [sigmatek_lrt] (lrt_ioctl_work+0x768 [sigmatek_lrt])
:| + begin 0x80000001 -1018 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1018 0.132 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1018 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1018 0.628 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1017 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1017 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1017 0.242 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1017 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1016 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1016 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1016 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1016 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1016 0.117 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1016 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1016 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1016 0.203 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1015 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1015 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1015 0.145 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1015 0.126 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1015 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1015 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1015 0.151 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -1014 0.124 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -1014 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1014 0.121 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1014 0.132 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -1014 0.163 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -1014 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -1014 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1014 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1013 0.213 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1013 0.158 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1013 0.155 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1013 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 584] realtim 95 -1013 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1013 0.294 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1012 0.400 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 582] cyclic: 30 -1012 0.184 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1012 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1012 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1011 0.402 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1011 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1011 0.161 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1011 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -1011 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1011 0.121 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1010 0.231 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -1010 0.179 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1010 0.147 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1010 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1010+ 1.202 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1009 0.115 __ipipe_syscall_root+0x5 (system_call+0x30)
: + func -1008 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1008 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1008 0.124 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1008 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1008 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1008 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1008 0.147 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1007 0.120 xnshadow_relax+0xc (hisyscall_event+0x201)
:| + begin 0x80000000 -1007 0.148 hard_local_irq_disable+0x15 (xnshadow_relax+0x30)
:| # func -1007 0.209 rpi_push+0x9 (xnshadow_relax+0x4d)
:| # func -1007 0.144 xnsched_renice_root+0x7 (rpi_push+0x8e)
:| # func -1007 0.139 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| # [ 0] -<?>- 30 -1007 0.151 xnsched_renice_root+0x82 (rpi_push+0x8e)
:| # func -1007 0.145 schedule_linux_call+0x9 (xnshadow_relax+0x5b)
:| # func -1006 0.200 __ipipe_set_irq_pending+0x6 (schedule_linux_call+0x83)
:| # func -1006 0.184 xnpod_suspend_thread+0x9 (xnshadow_relax+0x8c)
:| # func -1006 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1006 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000000 -1006 0.148 hard_local_irq_disable+0x15 (xnpod_suspend_thread+0x17d)
:| # func -1006 0.173 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 582] cyclic: 30 -1005 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1005 0.270 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1005 0.316 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- 30 -1005 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1004 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1004 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1004 0.175 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1004 0.341 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1004 0.194 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1004 0.148 __ipipe_do_sync_stage+0x9 (__ipipe_do_sync_pipeline+0x5e)
:| #end 0x80000000 -1003 0.115 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -1003 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -1003 0.149 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
: #func -1003 0.118 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -1003 0.142 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -1003 0.129 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -1003 0.148 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -1002 0.145 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -1002 0.129 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -1002 0.133 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -1002 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1002 0.129 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1002 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1002 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1002 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1001 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1001 0.117 local_bh_disable+0x3 (irq_enter+0x39)
: #func -1001 0.115 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -1001 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1001 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1001 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1001 0.112 tick_check_idle+0x8 (irq_enter+0x40)
: #func -1000 0.160 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -1000 0.213 ktime_get+0x9 (tick_check_idle+0x31)
: #func -1000 0.115 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -1000 0.115 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -1000 0.138 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -1000 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -1000 0.158 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -999 0.115 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -999 0.114 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -999 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -999 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -999 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -999 0.112 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -999 0.139 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -999 0.115 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -998 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -998 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -998 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -998 0.118 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -998 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -998 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -998 0.147 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -997 0.118 wait_domain_nrt_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xff)
: #func -997 0.115 __wake_up+0x8 (wait_domain_nrt_handler+0x33 [sigmatek_lrt])
: #func -997 0.141 _raw_spin_lock_irqsave+0x5 (__wake_up+0x1a)
: #func -997 0.121 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -997 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -997 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -997 0.170 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -996 0.118 __wake_up_common+0x9 (__wake_up+0x2e)
: #func -996 0.117 ipipe_root_only+0x5 (__wake_up_common+0x1b)
:| #begin 0x80000001 -996 0.184 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -996 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -996 0.112 __ipipe_spin_unlock_debug+0x3 (__wake_up+0x35)
: #func -996 0.126 _raw_spin_unlock_irqrestore+0x4 (__wake_up+0x3e)
: #func -996 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -996 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -995 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -995 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -995 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -995 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -995 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -995 0.136 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -995 0.127 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -994 0.115 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -994 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -994 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -994 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -994 0.144 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -994 0.126 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -994 0.148 idle_cpu+0x3 (irq_exit+0x50)
: #func -994 0.123 tick_nohz_irq_exit+0x4 (irq_exit+0x79)
:| #begin 0x80000001 -993 0.152 debug_smp_processor_id+0x36 (tick_nohz_irq_exit+0x13)
:| #end 0x80000001 -993 0.129 debug_smp_processor_id+0x82 (tick_nohz_irq_exit+0x13)
: #func -993 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_irq_exit+0x27)
:| #begin 0x80000001 -993 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -993 0.127 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -993 0.155 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -993 0.152 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -992 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -992 0.142 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -992 0.169 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -992 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -992 0.190 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -992 0.133 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -991 0.145 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -991 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -991 0.130 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -991 0.120 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -991 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -991 0.129 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -991 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -991 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -990 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -990 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -990 0.147 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -990 0.142 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -990 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -990 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -990 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -989 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -989 0.120 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -989 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -989 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -989 0.142 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -989 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -989 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -989 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -988 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -988 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -988 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -988 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -988 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -988 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -988 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -987 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -987 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -987 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -987 0.172 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -987 0.151 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -987 0.141 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -987 0.130 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -986 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -986 0.151 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -986 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -986 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -986 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -986 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -986 0.155 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -986 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
: #func -985 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #begin 0x80000000 -985 0.163 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| #end 0x80000000 -985 0.114 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -985 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -985 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
: #func -985 0.118 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -985 0.142 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -984 0.127 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -984 0.147 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -984 0.142 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -984 0.129 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -984 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -984 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -984 0.129 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -983 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -983 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -983 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -983 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -983 0.114 local_bh_disable+0x3 (irq_enter+0x39)
: #func -983 0.115 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -983 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -983 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -982 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -982 0.114 tick_check_idle+0x8 (irq_enter+0x40)
: #func -982 0.117 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -982 0.147 ktime_get+0x9 (tick_check_idle+0x31)
: #func -982 0.117 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -982 0.114 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -982 0.129 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -982 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -981 0.157 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -981 0.115 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -981 0.114 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -981 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -981 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -981 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -981 0.115 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -981 0.130 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -980 0.114 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -980 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -980 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -980 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -980 0.117 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -980 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -980 0.169 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -979 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -979 0.167 rthal_apc_handler+0x6 (__ipipe_do_sync_stage+0xff)
: #func -979 0.123 lostage_handler+0xc (rthal_apc_handler+0x6b)
:| #begin 0x80000001 -979 0.142 debug_smp_processor_id+0x36 (lostage_handler+0x16)
:| #end 0x80000001 -979 0.149 debug_smp_processor_id+0x82 (lostage_handler+0x16)
: #func -979 0.117 xnsched_renice_root+0x7 (lostage_handler+0xbb)
:| #begin 0x80000000 -979 0.164 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *#func -978 0.130 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *#[ 0] -<?>- -1 -978 0.145 xnsched_renice_root+0x82 (lostage_handler+0xbb)
:| *#func -978 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| #end 0x80000000 -978 0.118 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -978 0.117 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #begin 0x80000001 -978 0.152 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| #end 0x80000001 -978 0.126 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: #func -977 0.118 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| #begin 0x80000001 -977 0.129 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| #func -977 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| #func -977 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #func -977 0.130 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| #func -977 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| #func -977 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #func -977 0.147 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| #func -976 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| #func -976 0.204 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #*func -976 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| #*func -976 0.157 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #*func -976 0.133 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| #*func -976 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 0] -<?>- -1 -976 0.135 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -975 0.129 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -975 0.261 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| #*func -975 0.329 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 580] Loader: 29 -975 0.184 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -975 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -974 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -974 0.354 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -974 0.145 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +*end 0x80000000 -974 0.181 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +*func -974 0.250 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x465 [sigmatek_lrt])
:| +*begin 0x80000001 -973 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +*end 0x80000001 -973 0.154 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +*begin 0x80000000 -973 0.173 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +*end 0x80000000 -973+ 1.297 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: +*func -972 0.121 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +*func -971 0.130 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +*begin 0x80000001 -971 0.204 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| +*end 0x80000001 -971 0.135 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +*func -971 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +*begin 0x80000001 -971 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +*end 0x80000001 -971 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +*func -971 0.145 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +*func -970 0.115 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: +*func -970 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: +*func -970 0.120 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| +*begin 0x80000000 -970 0.210 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| #*func -970 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +*end 0x80000000 -970 0.121 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +*begin 0x80000001 -970 0.155 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| +*end 0x80000001 -969 0.126 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: +*func -969 0.123 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: +*func -969 0.118 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: +*func -969 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| +*begin 0x80000001 -969 0.154 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| +*end 0x80000001 -969 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
: +*func -969 0.117 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: +*func -969 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| +*begin 0x80000000 -968 0.178 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| #*func -968 0.184 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| #*func -968 0.158 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| #*func -968 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 580] Loader: 29 -968 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -968 0.255 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -967 0.313 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 0] -<?>- -1 -967 0.210 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -967 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -967 0.126 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -967 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -966 0.308 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| #func -966 0.206 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -966 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #func -966 0.193 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #end 0x80000001 -966 0.160 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
: #func -965 0.117 wake_up_process+0x4 (lostage_handler+0xcb)
: #func -965 0.114 try_to_wake_up+0x9 (wake_up_process+0x2e)
: #func -965 0.158 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -965 0.112 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -965 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -965 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -965 0.161 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -965 0.129 select_task_rq_rt+0x6 (select_task_rq+0x15)
:| #begin 0x80000001 -964 0.142 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -964 0.126 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -964 0.123 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -964 0.111 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -964 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -964 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -964 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -963 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -963 0.117 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -963 0.117 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -963 0.115 enqueue_task+0x8 (activate_task+0x21)
: #func -963 0.144 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -963 0.154 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -963 0.215 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -962 0.117 enqueue_task_rt+0x7 (enqueue_task+0x28)
: #func -962 0.138 dequeue_rt_stack+0x9 (enqueue_task_rt+0x2c)
: #func -962 0.155 cpupri_set+0x6 (enqueue_task_rt+0xca)
: #func -962 0.124 update_rt_migration+0x4 (enqueue_task_rt+0xea)
: #func -962 0.115 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -962 0.144 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -962 0.148 resched_task+0x5 (check_preempt_curr+0x3c)
:| #begin 0x80000001 -962 0.152 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -961 0.130 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -961 0.123 task_woken_rt+0x4 (ttwu_do_wakeup+0x5b)
: #func -961 0.123 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -961 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -961 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -961 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -961 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -960 0.127 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -960 0.124 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -960 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -960 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -960 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -960 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -960 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -960 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -959 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -959 0.193 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -959 0.191 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -959 0.115 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -959 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -959 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -958 0.155 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -958 0.141 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -958 0.127 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -958 0.118 idle_cpu+0x3 (irq_exit+0x50)
: #func -958 0.145 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -958 0.142 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -958 0.136 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -958 0.118 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -957 0.215 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -957 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -957 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -957 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -957 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -957 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -957 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
: #func -956 0.149 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #begin 0x80000000 -956 0.193 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| +func -956 0.123 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -956 0.210 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x00000051 -956 0.277 common_interrupt+0x51 (need_resched+0x0)
:| +begin 0x80000001 -955 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x9f)
:| +end 0x80000001 -955 0.127 debug_smp_processor_id+0x82 (cpu_startup_entry+0x9f)
: +func -955 0.147 rcu_idle_exit+0x6 (cpu_startup_entry+0xb8)
:| #begin 0x80000001 -955 0.151 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -955 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -955 0.133 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -955 0.152 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -954 0.139 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -954 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -954 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -954 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -954 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -954 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -954 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -954 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -953 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -953 0.127 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
: +func -953 0.121 tick_nohz_idle_exit+0x7 (cpu_startup_entry+0x148)
:| +begin 0x80000001 -953 0.144 debug_smp_processor_id+0x36 (tick_nohz_idle_exit+0x16)
:| +end 0x80000001 -953 0.142 debug_smp_processor_id+0x82 (tick_nohz_idle_exit+0x16)
: #func -953 0.148 ktime_get+0x9 (tick_nohz_idle_exit+0x62)
: #func -953 0.117 tick_do_update_jiffies64+0x9 (tick_nohz_idle_exit+0x8b)
: #func -952 0.121 update_cpu_load_nohz+0x5 (tick_nohz_idle_exit+0x90)
:| #begin 0x80000001 -952 0.142 debug_smp_processor_id+0x36 (update_cpu_load_nohz+0x14)
:| #end 0x80000001 -952 0.129 debug_smp_processor_id+0x82 (update_cpu_load_nohz+0x14)
: #func -952 0.112 _raw_spin_lock+0x4 (update_cpu_load_nohz+0x2d)
: #func -952 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -952 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -952 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -952 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -951 0.129 __update_cpu_load+0x9 (update_cpu_load_nohz+0x40)
: #func -951 0.121 sched_avg_update+0x9 (__update_cpu_load+0xa9)
: #func -951 0.127 _raw_spin_unlock+0x3 (update_cpu_load_nohz+0x47)
: #func -951 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -951 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -951 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -951 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -950 0.121 calc_load_exit_idle+0x4 (tick_nohz_idle_exit+0x95)
:| #begin 0x80000001 -950 0.142 debug_smp_processor_id+0x36 (calc_load_exit_idle+0x13)
:| #end 0x80000001 -950 0.130 debug_smp_processor_id+0x82 (calc_load_exit_idle+0x13)
: #func -950 0.121 hrtimer_cancel+0x4 (tick_nohz_idle_exit+0xa9)
: #func -950 0.114 hrtimer_try_to_cancel+0x7 (hrtimer_cancel+0x12)
: #func -950 0.117 lock_hrtimer_base.isra.17+0x6 (hrtimer_try_to_cancel+0x1e)
: #func -950 0.139 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -950 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -949 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -949 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -949 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -949 0.142 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -949 0.129 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -949 0.124 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -949 0.129 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
: #func -949 0.111 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -948 0.124 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -948 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -948 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -948 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -948 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -948 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -948 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -947 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -947 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -947 0.121 hrtimer_forward+0xc (tick_nohz_idle_exit+0xd2)
: #func -947 0.118 ktime_add_safe+0x6 (hrtimer_forward+0x103)
: #func -947 0.120 ktime_add_safe+0x6 (hrtimer_forward+0x11d)
: #func -947 0.114 hrtimer_start_range_ns+0x3 (hrtimer_start_expires.constprop.9+0x1d)
: #func -947 0.115 __hrtimer_start_range_ns+0x9 (hrtimer_start_range_ns+0x15)
: #func -947 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -946 0.141 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -946 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -946 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -946 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -946 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -946 0.141 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -946 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -945 0.117 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -945 0.135 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -945 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -945 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -945 0.141 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -945 0.133 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -945 0.115 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -945 0.121 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -944 0.151 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -944 0.117 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -944 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -944 0.133 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -944 0.164 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -944 0.126 xntimer_next_local_shot+0x9 (xntimer_start_aperiodic+0x11a)
:| *#event tick@-194 -944 0.117 xntimer_next_local_shot+0xca (xntimer_start_aperiodic+0x11a)
:| *#func -943 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| *#func -943 0.245 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| *#func -943 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -943 0.126 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -943 0.112 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -943 0.126 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -943 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -942 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -942 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -942 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -942 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -942 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -942 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -942 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -941 0.117 account_idle_ticks+0x3 (tick_nohz_idle_exit+0x122)
: #func -941 0.130 account_idle_time+0x6 (account_idle_ticks+0xd)
:| #begin 0x80000001 -941 0.142 debug_smp_processor_id+0x36 (account_idle_time+0x1c)
:| #end 0x80000001 -941 0.133 debug_smp_processor_id+0x82 (account_idle_time+0x1c)
:| #begin 0x80000001 -941 0.152 debug_smp_processor_id+0x36 (account_idle_time+0x28)
:| #end 0x80000001 -941 0.127 debug_smp_processor_id+0x82 (account_idle_time+0x28)
: #func -941 0.121 ipipe_unstall_root+0x3 (tick_nohz_idle_exit+0x127)
:| #begin 0x80000000 -941 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -940 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -940 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -940 0.118 schedule_preempt_disabled+0x3 (cpu_startup_entry+0x14d)
: +func -940 0.115 sub_preempt_count+0x4 (schedule_preempt_disabled+0x12)
: +func -940 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -940 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -940 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -939 0.115 schedule+0x3 (schedule_preempt_disabled+0x17)
: +func -939 0.115 __schedule+0x9 (schedule+0x18)
: +func -939 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -939 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -939 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -939 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -939 0.152 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -939 0.130 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -938 0.117 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -938 0.145 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -938 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -938 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -938 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -938 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -938 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -938 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -937 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -937 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -937 0.118 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -937 0.118 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -937 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -937 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -937 0.127 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -936 0.117 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -936 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -936 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -936 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -936 0.120 pre_schedule_idle+0x3 (__schedule+0x242)
: #func -936 0.114 put_prev_task_idle+0x3 (__schedule+0x275)
: #func -936 0.121 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -936 0.120 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -935 0.130 dequeue_pushable_task+0x5 (pick_next_task_rt+0x5b)
: #func -935 0.127 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -935 0.130 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -935 0.120 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -935 0.154 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -935 0.148 rt_task_set_mode+0x7 (taskSwitchLinux+0xbf [sigmatek_lrt])
: #func -935 0.115 __ipipe_notify_kevent+0x9 (__schedule+0x391)
: #func -934 0.117 ipipe_root_only+0x5 (__ipipe_notify_kevent+0x1b)
:| #begin 0x80000001 -934 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -934 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -934 0.144 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x20)
:| #end 0x80000001 -934 0.124 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x49)
: #func -934 0.124 ipipe_kevent_hook+0x4 (__ipipe_notify_kevent+0x54)
: #func -934 0.154 schedule_event+0x9 (ipipe_kevent_hook+0x1c)
: #func -934 0.121 xnsched_renice_root+0x7 (schedule_event+0x1f0)
:| #begin 0x80000000 -933 0.154 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *#func -933 0.130 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *#[ 0] -<?>- 30 -933 0.147 xnsched_renice_root+0x82 (schedule_event+0x1f0)
:| *#func -933 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| #end 0x80000000 -933 0.117 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -933 0.120 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #begin 0x80000001 -933 0.141 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| #end 0x80000001 -932 0.126 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: #func -932 0.118 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| #begin 0x80000001 -932 0.129 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| #func -932 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| #func -932 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #func -932 0.130 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| #func -932 0.120 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| #func -932 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #func -931 0.141 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| #func -931 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| #func -931 0.201 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #*func -931 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| #*func -931 0.147 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #*func -931 0.132 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| #*func -931 0.173 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 0] -<?>- 30 -930 0.136 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -930 0.283 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #func -930 0.175 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -930 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #func -930 0.178 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #end 0x80000001 -929 0.133 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
:| #begin 0x80000001 -929 0.139 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -929 0.139 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -929 0.206 __schedule+0x3a3 (schedule+0x18)
:| #func -929 0.381 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #func -928 0.135 ipipe_unstall_head+0x4 (xnshadow_relax+0x91)
:| #end 0x80000000 -928 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_head+0x2e)
: #func -928 0.123 __ipipe_reenter_root+0x4 (xnshadow_relax+0xa2)
:| #begin 0x80000001 -928 0.141 debug_smp_processor_id+0x36 (__ipipe_reenter_root+0x13)
:| #end 0x80000001 -928 0.127 debug_smp_processor_id+0x82 (__ipipe_reenter_root+0x13)
: #func -928 0.114 finish_task_switch+0x7 (__ipipe_reenter_root+0x28)
: #func -928 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -928 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -927 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -927 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -927 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -927 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -927 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -927 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -927 0.170 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -926 0.115 sub_preempt_count+0x4 (__ipipe_reenter_root+0x42)
: +func -926 0.126 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -926 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -926 0.181 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -926 0.210 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -926 0.124 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -926 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -925 0.157 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -925 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -925 0.118 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| +begin 0x80000001 -925 0.181 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -925 0.147 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +begin 0x80000000 -925 0.179 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +end 0x80000000 -925 0.142 __ipipe_syscall_root+0x9e (system_call+0x30)
: +func -924 0.114 SyS_select+0x9 (syscall_call+0x7)
: +func -924 0.114 _copy_from_user+0x7 (SyS_select+0x39)
: +func -924 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -924 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -924 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -924 0.167 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -924 0.123 poll_select_set_timeout+0x6 (SyS_select+0x60)
: +func -923 0.118 core_sys_select+0xc (SyS_select+0x80)
: +func -923 0.123 __rcu_read_lock+0x3 (core_sys_select+0x31)
: +func -923 0.126 __rcu_read_unlock+0x4 (core_sys_select+0x47)
: +func -923 0.114 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -923 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -923 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -923 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -923 0.133 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -922 0.115 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -922 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -922 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -922 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -922 0.184 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -922 0.114 do_select+0xc (core_sys_select+0x12b)
: +func -922 0.144 __rcu_read_lock+0x3 (do_select+0x23)
: +func -921 0.120 __rcu_read_unlock+0x4 (do_select+0xe5)
: +func -921 0.161 poll_initwait+0x3 (do_select+0xfd)
: +func -921 0.120 fget_light+0x6 (do_select+0x2aa)
: +func -921 0.136 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -921 0.136 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -921 0.115 eventfd_poll+0x6 (do_select+0x30f)
: +func -921 0.151 _raw_spin_lock_irqsave+0x5 (eventfd_poll+0x2f)
: #func -921 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -920 0.124 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -920 0.154 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -920 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -920 0.123 __ipipe_spin_unlock_debug+0x3 (eventfd_poll+0x68)
: #func -920 0.132 _raw_spin_unlock_irqrestore+0x4 (eventfd_poll+0x71)
: #func -920 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -920 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -919 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -919 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -919 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -919 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -919 0.190 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -919 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -919 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -919 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -918 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -918 0.136 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -918 0.166 fput+0x5 (do_select+0x327)
: +func -918 0.114 fget_light+0x6 (do_select+0x2aa)
: +func -918 0.141 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -918 0.151 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -918 0.130 sock_poll+0x5 (do_select+0x30f)
: +func -917 0.132 tcp_poll+0x6 (sock_poll+0x17)
: +func -917 0.127 fput+0x5 (do_select+0x327)
: +func -917 0.127 _cond_resched+0x3 (do_select+0x3f1)
: +func -917 0.124 poll_freewait+0x9 (do_select+0x4ab)
: +func -917 0.133 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -917 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -917 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -916 0.138 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -916 0.118 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -916 0.179 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -916 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -916 0.141 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -916! 189.808 poll_select_copy_remaining+0x9 (SyS_select+0x90)
: +func -726 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +func -726 0.118 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +begin 0x80000001 -726 0.198 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -726 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -725 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -725 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -725 0.138 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -725 0.135 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| + begin 0x80000001 -725 0.221 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -725 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -724 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -724 0.155 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -724 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -724 0.151 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +func -724 0.117 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: +func -724 0.114 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: +func -724 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| +begin 0x80000000 -724 0.212 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -723 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -723 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000001 -723 0.142 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| +end 0x80000001 -723 0.144 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: +func -723 0.115 lrt_non_rt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x51)
: +func -723 0.151 lrt_non_rt_ioctl_work+0xc [sigmatek_lrt] (lrt_non_rt_ioctl+0x10 [sigmatek_lrt])
: +func -723 0.120 xnshadow_harden+0x9 (losyscall_event+0x88)
: +func -722 0.115 down_interruptible+0x6 (xnshadow_harden+0x4f)
: +func -722 0.142 _raw_spin_lock_irqsave+0x5 (down_interruptible+0x12)
: #func -722 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -722 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -722 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -722 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -722 0.132 __ipipe_spin_unlock_debug+0x3 (down_interruptible+0x33)
: #func -721 0.121 _raw_spin_unlock_irqrestore+0x4 (down_interruptible+0x3c)
: #func -721 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -721 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -721 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -721 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -721 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -721 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -721 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -720 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -720 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -720 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -720 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -720 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -720 0.114 add_preempt_count+0x4 (xnshadow_harden+0x76)
: +func -720 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -720 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -719 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -719 0.124 wake_up_process+0x4 (xnshadow_harden+0xe1)
: +func -719 0.111 try_to_wake_up+0x9 (wake_up_process+0x2e)
: +func -719 0.141 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -719 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -719 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -719 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -718 0.166 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -718 0.139 select_task_rq_rt+0x6 (select_task_rq+0x15)
:| #begin 0x80000001 -718 0.141 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -718 0.127 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -718 0.124 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -718 0.111 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -718 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -718 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -717 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -717 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -717 0.117 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -717 0.115 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -717 0.117 enqueue_task+0x8 (activate_task+0x21)
: #func -717 0.138 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -717 0.155 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -716 0.145 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -716 0.114 enqueue_task_rt+0x7 (enqueue_task+0x28)
: #func -716 0.127 dequeue_rt_stack+0x9 (enqueue_task_rt+0x2c)
: #func -716 0.151 cpupri_set+0x6 (enqueue_task_rt+0xca)
: #func -716 0.124 update_rt_migration+0x4 (enqueue_task_rt+0xea)
: #func -716 0.114 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -716 0.190 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -715 0.118 check_preempt_curr_rt+0x5 (check_preempt_curr+0x2c)
: #func -715 0.139 resched_task+0x5 (check_preempt_curr_rt+0x82)
:| #begin 0x80000001 -715 0.142 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -715 0.136 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -715 0.117 task_woken_rt+0x4 (ttwu_do_wakeup+0x5b)
: #func -715 0.123 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -715 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -715 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -714 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -714 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -714 0.130 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -714 0.121 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -714 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -714 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -714 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -714 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -713 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -713 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -713 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -713 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -713 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -713 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -713 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -712 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -712 0.120 schedule+0x3 (xnshadow_harden+0xe6)
: +func -712 0.115 __schedule+0x9 (schedule+0x18)
: +func -712 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -712 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -712 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -712 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -711 0.152 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -711 0.130 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -711 0.114 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -711 0.142 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -711 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -711 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -711 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -711 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -710 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -710 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -710 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -710 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -710 0.130 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -710 0.118 sub_preempt_count+0x4 (__schedule+0x54)
: +func -710 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -709 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -709 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -709 0.117 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -709 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -709 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -709 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -709 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -709 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -708 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -708 0.147 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -708 0.112 deactivate_task+0x4 (__schedule+0x10e)
: #func -708 0.117 dequeue_task+0x8 (deactivate_task+0x21)
: #func -708 0.190 update_rq_clock+0x6 (dequeue_task+0x1a)
: #func -708 0.120 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -708 0.126 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -707 0.120 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -707 0.144 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -707 0.117 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -707 0.132 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -707 0.135 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -707 0.123 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -707 0.123 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -707 0.121 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -706 0.117 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -706 0.136 dequeue_pushable_task+0x5 (pick_next_task_rt+0x5b)
: #func -706 0.147 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -706 0.139 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -706 0.120 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -706 0.218 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -706 0.145 rt_task_set_mode+0x7 (taskSwitchLinux+0xbf [sigmatek_lrt])
: #func -705 0.115 __ipipe_notify_kevent+0x9 (__schedule+0x391)
: #func -705 0.117 ipipe_root_only+0x5 (__ipipe_notify_kevent+0x1b)
:| #begin 0x80000001 -705 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -705 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -705 0.144 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x20)
:| #end 0x80000001 -705 0.124 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x49)
: #func -705 0.115 ipipe_kevent_hook+0x4 (__ipipe_notify_kevent+0x54)
: #func -704 0.141 schedule_event+0x9 (ipipe_kevent_hook+0x1c)
:| #begin 0x80000000 -704 0.160 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *#func -704 0.160 xnsched_peek_rpi+0x3 (schedule_event+0x14c)
:| *#func -704 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -704 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| #begin 0x80000001 -704 0.126 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -704 0.138 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -703 0.178 __schedule+0x3a3 (schedule+0x18)
:| #func -703 0.166 __switch_to+0x9 (__schedule+0x50a)
:| #func -703 0.138 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -703 0.133 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -703 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -703 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -703 0.114 finish_task_switch+0x7 (__schedule+0x533)
: #func -702 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -702 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -702 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -702 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -702 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -702 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -702 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -702 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -701 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -701 0.152 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -701 0.129 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -701 0.115 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -701 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -701 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -701 0.188 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -700 0.123 kthread_should_stop+0x3 (gatekeeper_thread+0xc3)
: +func -700 0.118 rpi_pop+0x9 (gatekeeper_thread+0xf0)
:| +begin 0x80000000 -700 0.170 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -700 0.170 xnsched_peek_rpi+0x3 (rpi_pop+0x6e)
:| *+func -700 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -700 0.117 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -700 0.120 xnsched_renice_root+0x7 (rpi_pop+0xb4)
:| +begin 0x80000000 -699 0.216 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *+func -699 0.121 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *+[ 0] -<?>- -1 -699 0.147 xnsched_renice_root+0x82 (rpi_pop+0xb4)
:| *+func -699 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| +end 0x80000000 -699 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000000 -699 0.157 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -699 0.145 xnpod_resume_thread+0x9 (gatekeeper_thread+0x11a)
:| *+[ 582] cyclic: 30 -698 0.185 xnpod_resume_thread+0x39 (gatekeeper_thread+0x11a)
:| *+func -698 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -698 0.121 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -698 0.120 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| +begin 0x80000001 -698 0.142 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| +end 0x80000001 -698 0.132 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: +func -698 0.120 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| +begin 0x80000001 -697 0.141 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| +func -697 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| +func -697 0.144 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -697 0.141 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| +func -697 0.120 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -697 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -697 0.145 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -696 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -696 0.206 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -696 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -696 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -696 0.132 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| # func -696 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -696 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -695 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -695 0.292 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -695 0.253 __switch_to+0x9 (__schedule+0x50a)
:| # func -695 0.190 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| # func -695 0.245 xnpod_switch_fpu+0x3 (xnshadow_harden+0x133)
:| # func -694 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -694 0.132 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -694 0.118 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: + func -694 0.126 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -694 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -694 0.212 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -694 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -693 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -693 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -693 0.147 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -693 0.120 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -693 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -693 0.151 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -693 0.126 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -692 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -692 0.120 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -692 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -692 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -692 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -692 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -692 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -692 0.155 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -691 0.169 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -691 0.144 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -691 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 582] cyclic: 30 -691 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -691 0.262 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -691 0.353 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 23] gatekee -1 -690 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -690 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -690 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -690 0.120 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -690 0.328 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -689 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -689 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -689 0.198 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x80000001 -689 0.170 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
: +func -689 0.118 up+0x5 (gatekeeper_thread+0xb9)
: +func -689 0.141 _raw_spin_lock_irqsave+0x5 (up+0x11)
: #func -688 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -688 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -688 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -688 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -688 0.121 __ipipe_spin_unlock_debug+0x3 (up+0x2e)
: #func -688 0.123 _raw_spin_unlock_irqrestore+0x4 (up+0x37)
: #func -688 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -687 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -687 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -687 0.126 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -687 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -687 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -687 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -687 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -687 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -686 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -686 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -686 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -686 0.115 schedule+0x3 (gatekeeper_thread+0xbe)
: +func -686 0.115 __schedule+0x9 (schedule+0x18)
: +func -686 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -686 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -686 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -685 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -685 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -685 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -685 0.115 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -685 0.144 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -685 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -685 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -684 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -684 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -684 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -684 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -684 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -684 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -684 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -683 0.117 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -683 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -683 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -683 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -683 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -683 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -683 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -683 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -682 0.114 deactivate_task+0x4 (__schedule+0x10e)
: #func -682 0.115 dequeue_task+0x8 (deactivate_task+0x21)
: #func -682 0.149 update_rq_clock+0x6 (dequeue_task+0x1a)
:| #begin 0x80000001 -682 0.141 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -682 0.142 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -682 0.114 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -682 0.126 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -681 0.118 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -681 0.141 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -681 0.172 cpupri_set+0x6 (dequeue_rt_stack+0x104)
: #func -681 0.121 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -681 0.121 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -681 0.114 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -681 0.121 pull_rt_task+0x9 (pre_schedule_rt+0x18)
: #func -681 0.132 idle_balance+0x9 (__schedule+0x252)
: #func -680 0.124 _raw_spin_unlock+0x3 (idle_balance+0x5f)
: #func -680 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -680 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -680 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -680 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -680 0.118 __rcu_read_lock+0x3 (idle_balance+0x64)
: #func -680 0.111 __rcu_read_unlock+0x4 (idle_balance+0xee)
: #func -680 0.114 _raw_spin_lock+0x4 (idle_balance+0xf5)
: #func -679 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -679 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -679 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -679 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -679 0.118 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -679 0.124 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -679 0.114 pick_next_task_fair+0x7 (__schedule+0x28c)
: #func -679 0.117 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -678 0.115 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -678 0.121 pick_next_task_fair+0x7 (__schedule+0x299)
: #func -678 0.129 pick_next_task_idle+0x3 (__schedule+0x299)
: #func -678 0.118 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -678 0.149 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -678 0.129 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -678 0.151 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -678 0.136 rt_task_set_mode+0x7 (taskSwitchLinux+0xbf [sigmatek_lrt])
:| #begin 0x80000000 -677 0.148 __schedule+0x3a3 (schedule+0x18)
:| #func -677 0.164 __switch_to+0x9 (__schedule+0x50a)
:| #func -677 0.139 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -677 0.126 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -677 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -677 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -677 0.114 finish_task_switch+0x7 (__schedule+0x533)
: #func -676 0.126 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -676 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -676 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -676 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -676 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -676 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -676 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -675 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -675 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -675 0.142 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -675 0.130 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -675 0.139 _raw_spin_lock_irqsave+0x5 (post_schedule.part.48+0xc)
: #func -675 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -675 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -675 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -674 0.155 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -674 0.114 post_schedule_idle+0x3 (post_schedule.part.48+0x22)
: #func -674 0.129 __ipipe_spin_unlock_debug+0x3 (post_schedule.part.48+0x29)
: #func -674 0.123 _raw_spin_unlock_irqrestore+0x4 (post_schedule.part.48+0x32)
: #func -674 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -674 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -674 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -673 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -673 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -673 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -673 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -673 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -673 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -673 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -673 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -672 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -672 0.114 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -672 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -672 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -672 0.166 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -672 0.118 add_preempt_count+0x4 (schedule_preempt_disabled+0x21)
: +func -672 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -671 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -671 0.169 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -671 0.127 tick_nohz_idle_enter+0x4 (cpu_startup_entry+0x24)
: +func -671 0.121 set_cpu_sd_state_idle+0x4 (tick_nohz_idle_enter+0x37)
:| +begin 0x80000001 -671 0.141 debug_smp_processor_id+0x36 (set_cpu_sd_state_idle+0xe)
:| +end 0x80000001 -671 0.126 debug_smp_processor_id+0x82 (set_cpu_sd_state_idle+0xe)
: +func -671 0.118 __rcu_read_lock+0x3 (set_cpu_sd_state_idle+0x15)
: +func -670 0.138 __rcu_read_unlock+0x4 (set_cpu_sd_state_idle+0x4d)
:| #begin 0x80000001 -670 0.141 debug_smp_processor_id+0x36 (tick_nohz_idle_enter+0x46)
:| #end 0x80000001 -670 0.127 debug_smp_processor_id+0x82 (tick_nohz_idle_enter+0x46)
: #func -670 0.121 __tick_nohz_idle_enter+0x9 (tick_nohz_idle_enter+0x5b)
:| #begin 0x80000001 -670 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -670 0.129 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -670 0.160 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -669 0.142 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -669 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -669 0.148 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -669 0.149 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -669 0.154 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -669 0.141 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -669 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -668 0.142 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -668 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -668 0.132 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -668 0.120 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -668 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -668 0.129 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -668 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -668 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -667 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -667 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -667 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -667 0.127 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -667 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -667 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -667 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -666 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -666 0.121 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -666 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -666 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -666 0.142 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -666 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -666 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -666 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -665 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -665 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -665 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -665 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -665 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -665 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -665 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -664 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -664 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -664 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -664 0.187 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -664 0.130 nohz_balance_enter_idle+0x3 (__tick_nohz_idle_enter+0x24a)
: #func -664 0.124 calc_load_enter_idle+0x4 (__tick_nohz_idle_enter+0x24f)
:| #begin 0x80000001 -664 0.142 debug_smp_processor_id+0x36 (calc_load_enter_idle+0x13)
:| #end 0x80000001 -663 0.132 debug_smp_processor_id+0x82 (calc_load_enter_idle+0x13)
: #func -663 0.115 hrtimer_start+0x3 (__tick_nohz_idle_enter+0x2e1)
: #func -663 0.115 __hrtimer_start_range_ns+0x9 (hrtimer_start+0x14)
: #func -663 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -663 0.141 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -663 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -663 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -663 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -662 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -662 0.148 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -662 0.129 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -662 0.126 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -662 0.130 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
:| #begin 0x80000001 -662 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -662 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -661 0.117 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -661 0.133 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -661 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -661 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -661 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -661 0.135 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -661 0.114 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -661 0.124 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -660 0.151 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -660 0.118 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -660 0.170 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -660 0.129 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -660 0.141 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -660 0.127 xntimer_next_local_shot+0x9 (xntimer_start_aperiodic+0x11a)
:| *#event tick@732804 -660 0.117 xntimer_next_local_shot+0xca (xntimer_start_aperiodic+0x11a)
:| *#func -659 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| *#func -659 0.239 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| *#func -659 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -659 0.148 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -659 0.117 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -659 0.126 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -659 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -658 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -658 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -658 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -658 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -658 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -658 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -658 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -657 0.120 ipipe_unstall_root+0x3 (tick_nohz_idle_enter+0x60)
:| #begin 0x80000000 -657 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -657 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -657 0.130 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x80000001 -657 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -657 0.151 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -657 0.114 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -657 0.120 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -656 0.148 rcu_idle_enter+0x5 (cpu_startup_entry+0x70)
:| #begin 0x80000001 -656 0.151 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -656 0.133 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -656 0.118 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -656 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -656 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -656 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -655 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -655 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -655 0.155 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -655 0.152 debug_smp_processor_id+0x36 (cpu_startup_entry+0x75)
:| #end 0x80000001 -655 0.127 debug_smp_processor_id+0x82 (cpu_startup_entry+0x75)
: #func -655 0.120 ipipe_unstall_root+0x3 (cpu_startup_entry+0x8d)
:| #begin 0x80000000 -655 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -654 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -654! 586.646 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x00000051 -68 0.173 common_interrupt+0x39 (cpu_startup_entry+0x91)
:| +func -67 0.148 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -67 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -67 0.253 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -67 0.164 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -67 0.138 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -67 0.155 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -66 0.118 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -66 0.129 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -66 0.124 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -66 0.112 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -66! 46.038 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -20 0.121 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -20 0.242 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -20 0.114 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -19+ 9.077 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -10 0.167 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -10 0.120 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -10 0.299 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -10 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -10 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -10 0.144 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -9 0.120 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -9 0.160 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x5a [sigmatek_lrt])
:| # func -9+ 4.286 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -5 0.163 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -5+ 4.623 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x13d [sigmatek_lrt])
:| # func 0 0.142 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1a4 [sigmatek_lrt])
:| # func 0 0.194 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1d2 [sigmatek_lrt])
:| # func 0 0.178 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x1e8 [sigmatek_lrt])
<| # freeze 0x000106b1 0 0.256 irqhandler_fpga+0x1ff [sigmatek_lrt] (xnintr_irq_handler+0x78)
| # func 0 0.268 printk+0x9 (irqhandler_fpga+0x214 [sigmatek_lrt])
| # func 0 1.579 __ipipe_spin_lock_irqsave+0x6 (printk+0x140)
| # func 2 0.160 __ipipe_spin_unlock_irqrestore+0x4 (printk+0x17b)
| # func 2 0.138 ipipe_raise_irq+0x8 (printk+0x189)
| # func 2 0.141 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
| # func 2 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
| # func 2 0.139 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
| # func 2 0.121 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
| # func 2 0.167 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
| # func 3 0.000 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 13:41 ` Johann Obermayr
@ 2015-07-13 14:02 ` Gilles Chanteperdrix
2015-07-13 14:56 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 14:02 UTC (permalink / raw)
To: Johann Obermayr; +Cc: Xenomai
On Mon, Jul 13, 2015 at 03:41:55PM +0200, Johann Obermayr wrote:
> Am 10.07.2015 um 12:49 schrieb Gilles Chanteperdrix:
> >On Fri, Jul 10, 2015 at 12:43:29PM +0200, Johann Obermayr wrote:
> >>Am 10.07.2015 um 11:46 schrieb Gilles Chanteperdrix:
> >>>On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
> >>>>Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
> >>>>>On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
> >>>>>>Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
> >>>>>>>On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
> >>>>>>>>Hello,
> >>>>>>>>
> >>>>>>>>we have follow situation
> >>>>>>>>
> >>>>>>>>u64 lrtdrv_time_of_irq_ns ;
> >>>>>>>>void worker_task()
> >>>>>>>>{
> >>>>>>>>while(1)
> >>>>>>>> {
> >>>>>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
> >>>>>>>>RTDM_TIMERMODE_ABSOLUTE);
> >>>>>>>> do_something();
> >>>>>>>> }
> >>>>>>>>}
> >>>>>>>>
> >>>>>>>>_kernel_rtdm_irq_handler()
> >>>>>>>>{
> >>>>>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
> >>>>>>>>}
> >>>>>>>>
> >>>>>>>>the _kernel_irq_callback() is called every 1ms.
> >>>>>>>>we will , that the worker_task begin 50us before next irq
> >>>>>>>>
> >>>>>>>>But sometime, the worker task start ~50us after irq. Why ?
> >>>>>>>Best way to know, enable the I-pipe tracer, setup sufficient back
> >>>>>>>trace points, and when the wrong wake up happens, trigger an I-pipe
> >>>>>>>trace.
> >>>>>>>
> >>>>>>i will enable i-pipe tracer.
> >>>>>>found out, that
> >>>>>>
> >>>>>>rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
> >>>>>>u64 calc = rtdm_clock_read_monotonic();
> >>>>>>
> >>>>>>sometime the diff between xxx & calc is biger than 61000ns.
> >>>>>This is called "kernel task scheduling latency". It can be measured by
> >>>>>the latency -t 1 test.
> >>>>Ok, i will test it with this test.
> >>>>Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
> >>>>rtdm_clock_read_monotonic
> >>>Yeah, latency -t 1 does that too. Only much better.
> >>>
> >>Result a) only latency -t 1
> >>RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
> >>RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat
> >>worst
> >>RTD| -2.618| -2.481| -2.384| 0| 0| -2.618| -2.384
> >>RTD| -2.506| -2.477| -2.301| 0| 0| -2.618| -2.301
> >>RTD| -2.590| -2.479| -2.192| 0| 0| -2.618| -2.192
> >>RTD| -2.517| -2.489| -2.324| 0| 0| -2.618| -2.192
> >>RTD| -2.612| -2.490| -2.335| 0| 0| -2.618| -2.192
> >>RTD| -2.723| -2.490| -2.300| 0| 0| -2.723| -2.192
> >>RTD| -2.521| -2.491| -2.332| 0| 0| -2.723| -2.192
> >>RTD| -2.624| -2.492| -2.346| 0| 0| -2.723| -2.192
> >>RTD| -2.520| -2.490| -2.341| 0| 0| -2.723| -2.192
> >>RTD| -2.584| -2.490| -2.353| 0| 0| -2.723| -2.192
> >>RTD| -2.521| -2.487| -2.333| 0| 0| -2.723| -2.192
> >>RTD| -2.521| -2.490| -2.329| 0| 0| -2.723| -2.192
> >>RTD| -2.519| -2.488| -1.946| 0| 0| -2.723| -1.946
> >>RTD| -2.521| -2.490| -2.353| 0| 0| -2.723| -1.946
> >>RTD| -2.605| -2.490| -2.343| 0| 0| -2.723| -1.946
> >>RTD| -2.521| -2.490| -2.328| 0| 0| -2.723| -1.946
> >>RTD| -2.568| -2.489| -2.337| 0| 0| -2.723| -1.946
> >>RTD| -2.520| -2.490| -2.329| 0| 0| -2.723| -1.946
> >>RTD| -2.553| -2.491| -2.337| 0| 0| -2.723| -1.946
> >>RTD| -2.521| -2.491| -2.325| 0| 0| -2.723| -1.946
> >>RTD| -2.521| -2.490| -2.341| 0| 0| -2.723| -1.946
> >>
> >>Result b) with our application, that copy with a releaxed task from/to a
> >>SRAM on a PCI Card.
> >>RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
> >>RTH|---lat min|---lat avg|----lat max|-overrun|---msw|---lat best|--lat
> >>worst
> >>RTD| -2.680| -1.537| 716.496| 38| 0| -2.680| 716.496
> >>RTD| -2.680| -1.414| 691.232| 81| 0| -2.680| 716.496
> >>RTD| -3.051| -0.877| 739.939| 120| 0| -3.051| 739.939
> >>RTD| -2.662| -1.422| 738.747| 160| 0| -3.051| 739.939
> >>RTD| -2.713| -1.637| 726.880| 191| 0| -3.051| 739.939
> >>RTD| -2.656| -1.044| 675.880| 218| 0| -3.051| 739.939
> >>RTD| -2.655| -1.269| 753.354| 251| 0| -3.051| 753.354
> >>RTD| -2.661| -1.344| 633.684| 301| 0| -3.051| 753.354
> >>RTD| -2.663| -1.626| 698.008| 333| 0| -3.051| 753.354
> >>RTD| -2.662| -0.579| 751.352| 379| 0| -3.051| 753.354
> >>RTD| -2.669| -1.455| 733.275| 410| 0| -3.051| 753.354
> >>RTD| -2.758| -1.670| 629.569| 434| 0| -3.051| 753.354
> >>RTD| -2.760| -1.686| 690.451| 445| 0| -3.051| 753.354
> >>RTD| -2.668| -1.032| 753.099| 483| 0| -3.051| 753.354
> >>RTD| -2.661| -1.476| 735.519| 518| 0| -3.051| 753.354
> >>RTD| -2.660| -1.647| 725.828| 552| 0| -3.051| 753.354
> >>RTD| -2.661| -1.236| 250.879| 556| 0| -3.051| 753.354
> >Your application (and more probably driver) has a problem, it
> >probably creates large interrupts masking section. Once again, to
> >understand what the problem is, I suggest using the I-pipe tracer.
> >
> Hello,
>
> i have add a ipipe trace. But i don't understand it.
>
> the function lrtdrv_monitoring_irq has no mutex, only a atomic funtion and
> write result into a ring buffer.
> (see your trace screen. red lines)
>
> http://www.pic-upload.de/view-27670244/Trace001.png.html
Neither do I. We do not know when the interrupt was expected, so we
basically can not say how much it is late.
So, please:
1- run the latency test but generate load with the dohell script
instead of running your application.
If you get large latencies, then it is probable that the issue does
not come from your application
2- Enable the I-pipe tracer, and run the latency test with the -f
argument, generating with whatever load (dohell script, or your
application) which causes high latencies.
Regards.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 14:02 ` Gilles Chanteperdrix
@ 2015-07-13 14:56 ` Johann Obermayr
2015-07-13 15:24 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 14:56 UTC (permalink / raw)
To: Xenomai
Am 13.07.2015 um 16:02 schrieb Gilles Chanteperdrix:
> On Mon, Jul 13, 2015 at 03:41:55PM +0200, Johann Obermayr wrote:
>> Am 10.07.2015 um 12:49 schrieb Gilles Chanteperdrix:
>>> On Fri, Jul 10, 2015 at 12:43:29PM +0200, Johann Obermayr wrote:
>>>> Am 10.07.2015 um 11:46 schrieb Gilles Chanteperdrix:
>>>>> On Fri, Jul 10, 2015 at 11:34:59AM +0200, Johann Obermayr wrote:
>>>>>> Am 09.07.2015 um 17:51 schrieb Gilles Chanteperdrix:
>>>>>>> On Thu, Jul 09, 2015 at 05:43:03PM +0200, Johann Obermayr wrote:
>>>>>>>> Am 09.07.2015 um 16:19 schrieb Gilles Chanteperdrix:
>>>>>>>>> On Thu, Jul 09, 2015 at 02:15:55PM +0200, Johann Obermayr wrote:
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> we have follow situation
>>>>>>>>>>
>>>>>>>>>> u64 lrtdrv_time_of_irq_ns ;
>>>>>>>>>> void worker_task()
>>>>>>>>>> {
>>>>>>>>>> while(1)
>>>>>>>>>> {
>>>>>>>>>> rtdm_task_sleep_abs(lrtdrv_time_of_irq_ns + 950000ull,
>>>>>>>>>> RTDM_TIMERMODE_ABSOLUTE);
>>>>>>>>>> do_something();
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> _kernel_rtdm_irq_handler()
>>>>>>>>>> {
>>>>>>>>>> lrtdrv_time_of_irq_ns = rtdm_clock_read_monotonic();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> the _kernel_irq_callback() is called every 1ms.
>>>>>>>>>> we will , that the worker_task begin 50us before next irq
>>>>>>>>>>
>>>>>>>>>> But sometime, the worker task start ~50us after irq. Why ?
>>>>>>>>> Best way to know, enable the I-pipe tracer, setup sufficient back
>>>>>>>>> trace points, and when the wrong wake up happens, trigger an I-pipe
>>>>>>>>> trace.
>>>>>>>>>
>>>>>>>> i will enable i-pipe tracer.
>>>>>>>> found out, that
>>>>>>>>
>>>>>>>> rtdm_task_sleep_abs(xxx, RTDM_TIMERMODE_ABSOLUTE);
>>>>>>>> u64 calc = rtdm_clock_read_monotonic();
>>>>>>>>
>>>>>>>> sometime the diff between xxx & calc is biger than 61000ns.
>>>>>>> This is called "kernel task scheduling latency". It can be measured by
>>>>>>> the latency -t 1 test.
>>>>>> Ok, i will test it with this test.
>>>>>> Here a simple list of measure from rtdm_task_sleep_abs(xxx) calc =
>>>>>> rtdm_clock_read_monotonic
>>>>> Yeah, latency -t 1 does that too. Only much better.
>>>>>
>>>> Result a) only latency -t 1
>>>> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
>>>> RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat
>>>> worst
>>>> RTD| -2.618| -2.481| -2.384| 0| 0| -2.618| -2.384
>>>> RTD| -2.506| -2.477| -2.301| 0| 0| -2.618| -2.301
>>>> RTD| -2.590| -2.479| -2.192| 0| 0| -2.618| -2.192
>>>> RTD| -2.517| -2.489| -2.324| 0| 0| -2.618| -2.192
>>>> RTD| -2.612| -2.490| -2.335| 0| 0| -2.618| -2.192
>>>> RTD| -2.723| -2.490| -2.300| 0| 0| -2.723| -2.192
>>>> RTD| -2.521| -2.491| -2.332| 0| 0| -2.723| -2.192
>>>> RTD| -2.624| -2.492| -2.346| 0| 0| -2.723| -2.192
>>>> RTD| -2.520| -2.490| -2.341| 0| 0| -2.723| -2.192
>>>> RTD| -2.584| -2.490| -2.353| 0| 0| -2.723| -2.192
>>>> RTD| -2.521| -2.487| -2.333| 0| 0| -2.723| -2.192
>>>> RTD| -2.521| -2.490| -2.329| 0| 0| -2.723| -2.192
>>>> RTD| -2.519| -2.488| -1.946| 0| 0| -2.723| -1.946
>>>> RTD| -2.521| -2.490| -2.353| 0| 0| -2.723| -1.946
>>>> RTD| -2.605| -2.490| -2.343| 0| 0| -2.723| -1.946
>>>> RTD| -2.521| -2.490| -2.328| 0| 0| -2.723| -1.946
>>>> RTD| -2.568| -2.489| -2.337| 0| 0| -2.723| -1.946
>>>> RTD| -2.520| -2.490| -2.329| 0| 0| -2.723| -1.946
>>>> RTD| -2.553| -2.491| -2.337| 0| 0| -2.723| -1.946
>>>> RTD| -2.521| -2.491| -2.325| 0| 0| -2.723| -1.946
>>>> RTD| -2.521| -2.490| -2.341| 0| 0| -2.723| -1.946
>>>>
>>>> Result b) with our application, that copy with a releaxed task from/to a
>>>> SRAM on a PCI Card.
>>>> RTT| 00:00:01 (in-kernel periodic task, 100 us period, priority 99)
>>>> RTH|---lat min|---lat avg|----lat max|-overrun|---msw|---lat best|--lat
>>>> worst
>>>> RTD| -2.680| -1.537| 716.496| 38| 0| -2.680| 716.496
>>>> RTD| -2.680| -1.414| 691.232| 81| 0| -2.680| 716.496
>>>> RTD| -3.051| -0.877| 739.939| 120| 0| -3.051| 739.939
>>>> RTD| -2.662| -1.422| 738.747| 160| 0| -3.051| 739.939
>>>> RTD| -2.713| -1.637| 726.880| 191| 0| -3.051| 739.939
>>>> RTD| -2.656| -1.044| 675.880| 218| 0| -3.051| 739.939
>>>> RTD| -2.655| -1.269| 753.354| 251| 0| -3.051| 753.354
>>>> RTD| -2.661| -1.344| 633.684| 301| 0| -3.051| 753.354
>>>> RTD| -2.663| -1.626| 698.008| 333| 0| -3.051| 753.354
>>>> RTD| -2.662| -0.579| 751.352| 379| 0| -3.051| 753.354
>>>> RTD| -2.669| -1.455| 733.275| 410| 0| -3.051| 753.354
>>>> RTD| -2.758| -1.670| 629.569| 434| 0| -3.051| 753.354
>>>> RTD| -2.760| -1.686| 690.451| 445| 0| -3.051| 753.354
>>>> RTD| -2.668| -1.032| 753.099| 483| 0| -3.051| 753.354
>>>> RTD| -2.661| -1.476| 735.519| 518| 0| -3.051| 753.354
>>>> RTD| -2.660| -1.647| 725.828| 552| 0| -3.051| 753.354
>>>> RTD| -2.661| -1.236| 250.879| 556| 0| -3.051| 753.354
>>> Your application (and more probably driver) has a problem, it
>>> probably creates large interrupts masking section. Once again, to
>>> understand what the problem is, I suggest using the I-pipe tracer.
>>>
>> Hello,
>>
>> i have add a ipipe trace. But i don't understand it.
>>
>> the function lrtdrv_monitoring_irq has no mutex, only a atomic funtion and
>> write result into a ring buffer.
>> (see your trace screen. red lines)
>>
>> http://www.pic-upload.de/view-27670244/Trace001.png.html
> Neither do I. We do not know when the interrupt was expected, so we
> basically can not say how much it is late.
>
> So, please:
> 1- run the latency test but generate load with the dohell script
> instead of running your application.
>
> If you get large latencies, then it is probable that the issue does
> not come from your application
>
> 2- Enable the I-pipe tracer, and run the latency test with the -f
> argument, generating with whatever load (dohell script, or your
> application) which causes high latencies.
>
> Regards.
>
without your application, there are no large latencies.
with your application see frozen.txt (with latency -f)
We see the problem only if one task (background) is accessing the SRAM
on your PCI-Card. if we stop this task, all is ok.
So we have a higher prior task (pci-locker), that interrupt the
background task, so that the pci bus get free.
regards
Johann
-------------- next part --------------
I-pipe frozen back-tracing service on 3.10.0/ipipe release #1
------------------------------------------------------------
CPU: 0, Freeze: 1619507745620 cycles, Trace Points: 2000 (+10)
Calibrated minimum trace-point overhead: 0.078 us
+----- Hard IRQs ('|': locked)
|+-- Xenomai
||+- Linux ('*': domain stalled, '+': current, '#': current+stalled)
||| +---------- Delay flag ('+': > 1 us, '!': > 10 us)
||| | +- NMI noise ('N')
||| | |
Type User Val. Time Delay Function (Parent)
:| + end 0x80000001 -2589 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -2589 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -2589 0.136 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -2589 0.124 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| + begin 0x80000001 -2589 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| + end 0x80000001 -2588 0.129 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
: + func -2588 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -2588 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -2588 0.155 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -2588 0.157 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -2588 0.142 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -2588 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 602] Loader: 29 -2588 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2587 0.277 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2587 0.322 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2587 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2587 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2586 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2586 0.149 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2586 0.314 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2586 0.191 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2586 0.157 __ipipe_do_sync_stage+0x9 (__ipipe_do_sync_pipeline+0x5e)
:| #end 0x80000000 -2586 0.115 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -2585 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -2585 0.127 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
: #func -2585 0.120 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -2585 0.142 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -2585 0.127 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -2585 0.146 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -2585 0.141 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -2585 0.135 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -2584 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -2584 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -2584 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -2584 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -2584 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -2584 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2584 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2583 0.114 local_bh_disable+0x3 (irq_enter+0x39)
: #func -2583 0.115 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -2583 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -2583 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2583 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2583 0.114 tick_check_idle+0x8 (irq_enter+0x40)
: #func -2583 0.151 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -2583 0.219 ktime_get+0x9 (tick_check_idle+0x31)
: #func -2582 0.115 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -2582 0.115 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -2582 0.149 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -2582 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -2582 0.158 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -2582 0.144 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -2581 0.112 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -2581 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -2581 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2581 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2581 0.117 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -2581 0.149 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -2581 0.112 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -2581 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -2580 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2580 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2580 0.117 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -2580 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -2580 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2580 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2580 0.133 wait_domain_nrt_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xff)
: #func -2579 0.133 __wake_up+0x8 (wait_domain_nrt_handler+0x33 [sigmatek_lrt])
: #func -2579 0.144 _raw_spin_lock_irqsave+0x5 (__wake_up+0x1a)
: #func -2579 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -2579 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -2579 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2579 0.169 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2579 0.117 __wake_up_common+0x9 (__wake_up+0x2e)
: #func -2578 0.118 ipipe_root_only+0x5 (__wake_up_common+0x1b)
:| #begin 0x80000001 -2578 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2578 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2578 0.123 __ipipe_spin_unlock_debug+0x3 (__wake_up+0x35)
: #func -2578 0.124 _raw_spin_unlock_irqrestore+0x4 (__wake_up+0x3e)
: #func -2578 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -2578 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -2578 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2577 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2577 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -2577 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -2577 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2577 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2577 0.129 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -2577 0.114 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -2577 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -2576 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2576 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -2576 0.142 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -2576 0.127 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -2576 0.123 idle_cpu+0x3 (irq_exit+0x50)
: #func -2576 0.118 tick_nohz_irq_exit+0x4 (irq_exit+0x79)
:| #begin 0x80000001 -2576 0.142 debug_smp_processor_id+0x36 (tick_nohz_irq_exit+0x13)
:| #end 0x80000001 -2575 0.129 debug_smp_processor_id+0x82 (tick_nohz_irq_exit+0x13)
: #func -2575 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_irq_exit+0x27)
:| #begin 0x80000001 -2575 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -2575 0.127 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -2575 0.155 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -2575 0.152 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -2575 0.146 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -2574 0.149 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -2574 0.151 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -2574 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -2574 0.188 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -2574 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -2574 0.146 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -2574 0.141 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -2573 0.133 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -2573 0.120 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -2573 0.141 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -2573 0.129 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -2573 0.115 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -2573 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -2573 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -2573 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2572 0.146 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2572 0.126 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -2572 0.114 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -2572 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -2572 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2572 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2572 0.121 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -2571 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -2571 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -2571 0.142 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -2571 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -2571 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -2571 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2571 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2570 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -2570 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -2570 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -2570 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -2570 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2570 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2570 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -2570 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -2569 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2569 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2569 0.151 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -2569 0.141 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -2569 0.130 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -2569 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -2569 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -2568 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -2568 0.120 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -2568 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -2568 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -2568 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -2568 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
: #func -2568 0.124 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #begin 0x80000000 -2568 0.198 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| +func -2567 0.120 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2567 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x00000051 -2567+ 6.297 common_interrupt+0x51 (need_resched+0x1)
:| +begin 0x000000ef -2561 0.141 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -2561 0.152 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2560 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2560 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2560 0.123 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2560 0.127 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2560 0.151 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2560 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2560 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2560 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2559 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2559 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2559 0.142 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2559 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2559 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2559 0.170 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2559 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-2461 -2558 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2558 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2558 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2558 0.179 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2558 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2558 0.148 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2557 0.319 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2557 0.332 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2557 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2557 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2557 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2556 0.231 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2556 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2556 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2556 0.195 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2556 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2555 0.145 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2555 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2555 0.331 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2555 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2555 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2555 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2554 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2554 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2554 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2554 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2554 0.136 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2554 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2554 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2554 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2553 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2553 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2553 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2553 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2553 0.298 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2553 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2552 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2552 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2552 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2552 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2552 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2551 0.187 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2551 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2551 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2551! 90.277 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -2461 0.185 apic_timer_interrupt+0x3a (cpu_startup_entry+0x91)
:| +func -2461 0.181 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2460 0.121 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2460 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2460 0.129 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2460 0.133 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2460 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2460 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2460 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2459 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2459 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2459 0.169 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2459 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2459 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2459 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2459 0.195 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2458 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-2361 -2458 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2458 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2458 0.256 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2458 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2458 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2457 0.126 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2457 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2457 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2457 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2457 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2456 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2456 0.234 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2456 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2456 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2456 0.181 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2456 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2455 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2455 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2455 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2455 0.123 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2455 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2455 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2454 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2454 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2454 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2454 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2454 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2454 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2454 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2454 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2453 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2453 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2453 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2453 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2453 0.301 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2453 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2452 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2452 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2452 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2452 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2452 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2451 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2451 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2451 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2451! 90.259 apic_timer_interrupt+0x52 (cpu_startup_entry+0x91)
:| +begin 0x000000ef -2361 0.185 apic_timer_interrupt+0x3a (cpu_startup_entry+0x91)
:| +func -2361 0.179 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2360 0.126 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2360 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2360 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2360 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2360 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2360 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2360 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2359 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2359 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2359 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2359 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2359 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2359 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2359 0.195 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2358 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-2261 -2358 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2358 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2358 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2358 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2358 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2357 0.126 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2357 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2357 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2357 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2357 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2356 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2356 0.234 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2356 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2356 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2356 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2356 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2355 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2355 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2355 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2355 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2355 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2355 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2354 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2354 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2354 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2354 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2354 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2354 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2354 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2354 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2353 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2353 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2353 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2353 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2353 0.286 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2353 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2352 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2352 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2352 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2352 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2352 0.283 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2351 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2351 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2351 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2351! 90.268 apic_timer_interrupt+0x52 (cpu_startup_entry+0x91)
:| +begin 0x000000ef -2261 0.175 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -2261 0.173 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2260 0.121 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2260 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2260 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2260 0.146 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2260 0.139 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2260 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2260 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2259 0.130 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2259 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2259 0.218 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2259 0.133 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2259 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2259 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2259 0.175 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2258 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-2161 -2258 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2258 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2258 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2258 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2258 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2257 0.126 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2257 0.328 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2257 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2257 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2257 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2256 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2256 0.227 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2256 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2256 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2256 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2256 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2255 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2255 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2255 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2255 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2255 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2255 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2254 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2254 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2254 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2254 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2254 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2254 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2254 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2253 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2253 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2253 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2253 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2253 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2253 0.285 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2252 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2252 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2252 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2252 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2252 0.130 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2252 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2251 0.188 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2251 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2251 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2251! 90.245 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -2161 0.181 apic_timer_interrupt+0x3a (cpu_startup_entry+0x91)
:| +func -2161 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2160 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2160 0.148 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2160 0.114 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2160 0.146 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2160 0.139 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2160 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2160 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2159 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2159 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2159 0.215 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2159 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2159 0.117 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2159 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2159 0.197 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2158 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-2061 -2158 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2158 0.132 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2158 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2158 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2158 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2157 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2157 0.365 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2157 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2157 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2156 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2156 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2156 0.256 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2156 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2156 0.139 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2156 0.181 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2155 0.172 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2155 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2155 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2155 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2155 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2155 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2154 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2154 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2154 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2154 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2154 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2154 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2154 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2153 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2153 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2153 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2153 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2153 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2153 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2153 0.308 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2152 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2152 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2152 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2152 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2152 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2152 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2151 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2151 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2151 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2151! 90.098 apic_timer_interrupt+0x52 (cpu_startup_entry+0x91)
:| +begin 0x000000ef -2061 0.176 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -2061 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -2060 0.123 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -2060 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -2060 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -2060 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -2060 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -2060 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -2060 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2059 0.127 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -2059 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -2059 0.173 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -2059 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2059 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2059 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -2059 0.198 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -2058 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1961 -2058 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2058 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -2058 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -2058 0.179 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -2058 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2057 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2057 0.305 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -2057 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -2057 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2057 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2056 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2056 0.216 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2056 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -2056 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -2056 0.182 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -2056 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -2055 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -2055 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -2055 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2055 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2055 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2055 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2054 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2054 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2054 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2054 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2054 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -2054 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -2054 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -2054 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -2053 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -2053 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -2053 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -2053 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -2053 0.288 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -2053 0.221 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -2052 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -2052 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -2052 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -2052 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -2052 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -2051 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -2051 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -2051 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -2051! 90.282 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1961 0.184 apic_timer_interrupt+0x3a (cpu_startup_entry+0x91)
:| +func -1961 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1960 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1960 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1960 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1960 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1960 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1960 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1960 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1959 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1959 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1959 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1959 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1959 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1959 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1959 0.195 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1958 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1861 -1958 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1958 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1958 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1958 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1958 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1957 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1957 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1957 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1957 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1957 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1956 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1956 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1956 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1956 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1956 0.181 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1956 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1955 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1955 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1955 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1955 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1955 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1955 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1954 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1954 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1954 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1954 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1954 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1954 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1954 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1954 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1953 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1953 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1953 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1953 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1953 0.285 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1953 0.221 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1952 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1952 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1952 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1952 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1952 0.282 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1951 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1951 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1951 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -1951! 71.920 apic_timer_interrupt+0x52 (cpu_startup_entry+0x91)
:| +begin 0x00000051 -1879 0.182 common_interrupt+0x39 (cpu_startup_entry+0x91)
:| +func -1879 0.144 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -1879 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1879 0.160 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1878 0.124 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1878 0.135 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -1878 0.149 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -1878 0.118 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -1878 0.120 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -1878 0.135 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -1878 0.117 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -1878+ 1.355 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -1876 0.111 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -1876 0.133 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -1876 0.130 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -1876+ 1.165 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -1875 0.166 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -1875 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1874 0.188 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1874 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1874 0.141 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1874 0.138 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1874 0.118 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -1874 0.136 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x5a [sigmatek_lrt])
:| # func -1874+ 1.655 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -1872 0.130 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -1872+ 4.561 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x13d [sigmatek_lrt])
:| # func -1867 0.117 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1a4 [sigmatek_lrt])
:| # func -1867 0.154 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1d2 [sigmatek_lrt])
:| # func -1867 0.118 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x1e8 [sigmatek_lrt])
:| # func -1867 0.118 xnarch_get_cpu_time+0x4 (irqhandler_fpga+0x225 [sigmatek_lrt])
:| # func -1867 0.120 xnarch_tsc_to_ns+0x6 (xnarch_get_cpu_time+0x13)
:| # func -1867 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x245 [sigmatek_lrt])
:| # func -1866 0.120 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x25b [sigmatek_lrt])
:| # func -1866 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x280 [sigmatek_lrt])
:| # func -1866 0.114 period_update+0x4 [sigmatek_lrt] (irqhandler_fpga+0x287 [sigmatek_lrt])
:| # func -1866 0.146 lrtdrv_monitoring_start_timed+0x4 [sigmatek_lrt] (period_update+0x1f [sigmatek_lrt])
:| # func -1866 0.212 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -1866 0.227 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -1866 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -1865 0.216 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -1865 0.149 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -1865 0.152 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -1865 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 606] realtim 95 -1865 0.225 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1864 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -1864 0.164 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -1864 0.187 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 512] VARAN_T 30 -1864 0.233 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1864 0.133 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 604] cyclic: 30 -1864 0.155 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1863 0.133 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 602] Loader: 29 -1863 0.154 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1863 0.207 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 608] backgrou 0 -1863 0.225 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -1863 0.117 ipipe_raise_irq+0x8 (period_update+0x10f [sigmatek_lrt])
:| # func -1863 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| # func -1863 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1862 0.130 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| # func -1862 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| # func -1862 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1862 0.145 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| # func -1862 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| # func -1862 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1862 0.182 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| # func -1861 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| # func -1861 0.173 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1861+ 1.658 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -1860+ 1.646 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -1858 0.124 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -1858 0.123 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -1858 0.172 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x10d [sigmatek_lrt])
:| # func -1857 0.114 rthal_irq_end+0x5 (xnintr_irq_handler+0x14d)
:| # func -1857 0.123 irq_to_desc+0x3 (rthal_irq_end+0x1d)
:| # func -1857 0.121 irq_to_desc+0x3 (rthal_irq_end+0x28)
:| # func -1857 0.120 irq_to_desc+0x3 (rthal_irq_end+0x32)
:| # func -1857 0.115 __ipipe_end_fasteoi_irq+0x3 (rthal_irq_end+0x38)
:| # func -1857 0.115 release_ioapic_irq+0x5 (__ipipe_end_fasteoi_irq+0x16)
:| # func -1857 0.151 __ipipe_spin_lock_irqsave+0x6 (release_ioapic_irq+0x17)
:| # func -1857 0.115 io_apic_modify_irq.isra.9+0x7 (release_ioapic_irq+0x29)
:| # func -1856 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| # func -1856+ 1.333 io_apic_base+0x3 (native_io_apic_read+0x10)
:| # func -1855 0.112 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| # func -1855 0.123 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| # func -1855 0.201 __ipipe_spin_unlock_irqrestore+0x4 (release_ioapic_irq+0x35)
:| # func -1855 0.230 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # func -1854 0.166 __xnlock_spin+0x4 (__xnlock_get_irqsave.constprop.43+0x3f)
:| # [ 0] -<?>- -1 -1854 0.144 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1854 0.149 ipipe_send_ipi+0x6 (__xnpod_schedule+0x7d)
:| # func -1854 0.176 default_send_IPI_mask_logical+0x6 (ipipe_send_ipi+0x40)
:| # func -1854 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1854 0.264 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1853 0.307 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 606] realtim 95 -1853 0.206 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1853 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1853 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1853 0.332 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1852 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1852 0.265 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x000000ef -1852 0.133 apic_timer_interrupt+0x3a (hard_local_irq_enable+0x18)
:| + func -1852 0.152 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -1852 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -1851 0.126 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -1851 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -1851 0.133 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -1851 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -1851 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -1851 0.191 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1851 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1850 0.132 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1850 0.219 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1850 0.148 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1850 0.118 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1850 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1850 0.212 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1850 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1761 -1849 0.114 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1849 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1849 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1849 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 606] realtim 95 -1849 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1849 0.387 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1848 0.397 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1848 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1848 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1847 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1847 0.237 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1847 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1847 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1847 0.191 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1847 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1846 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1846 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1846 0.313 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1846 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1846 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1846 0.190 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1845 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1845 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1845 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1845 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1845 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1845 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1845 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1845 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1844 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1844 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1844 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1844 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1844 0.320 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1844 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 606] realtim 95 -1843 0.200 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1843 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1843 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1843 0.283 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -1842 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -1842 0.146 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -1842 0.282 apic_timer_interrupt+0x52 (hard_local_irq_enable+0x18)
: + func -1842 0.123 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -1842 0.142 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1842 0.126 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1842 0.242 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -1841 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1841 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1841 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1841! 18.421 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1822 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1822 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1822 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1822 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1822 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1822 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1822 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1821 0.146 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1821 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1821 0.117 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1821 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1821 0.262 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1821 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1821 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1820 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1820 0.129 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1820 0.120 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1820 0.144 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1820 0.172 pci_locker_signal_stop+0x3 [sigmatek_lrt] (lrt_ioctl_work+0x768 [sigmatek_lrt])
:| + begin 0x80000001 -1820 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1820 0.132 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1819 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1819 0.307 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1819 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1819 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1819 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1819 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1818 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1818 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1818 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1818 0.141 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1818 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1818 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1818 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1818 0.203 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1817 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1817 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1817 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1817 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1817 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1817 0.118 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1817 0.151 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -1816 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -1816 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1816 0.123 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1816 0.130 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -1816 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -1816 0.129 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -1816 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1816 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1815 0.210 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1815 0.157 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1815 0.170 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1815 0.193 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 606] realtim 95 -1815 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1815 0.305 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1814 0.402 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -1814 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1814 0.126 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1814 0.141 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1813 0.362 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1813 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1813 0.154 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1813 0.120 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -1813 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1812 0.124 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1812 0.224 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -1812 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1812 0.145 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1812 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1812! 50.921 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
:| + begin 0x000000ef -1761 0.130 apic_timer_interrupt+0x3a (<102d024a>)
:| + func -1761 0.152 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -1760 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -1760 0.149 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -1760 0.118 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -1760 0.129 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -1760 0.148 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -1760 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -1760 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1760 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1759 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1759 0.250 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1759 0.151 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1759 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1759 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1759 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1758 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1661 -1758 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1758 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1758 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1758 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -1758 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1757 0.389 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1757 0.396 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1757 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1757 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1756 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1756 0.268 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1756 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1756 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1756 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1756 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1755 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1755 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1755 0.308 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1755 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1755 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1754 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1754 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1754 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1754 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1754 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1754 0.136 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1754 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1754 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1753 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1753 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1753 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1753 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1753 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1753 0.298 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1752 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -1752 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1752 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1752 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1752 0.280 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -1751 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -1751 0.146 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -1751! 90.391 apic_timer_interrupt+0x52 (<102d024a>)
:| + begin 0x000000ef -1661 0.209 apic_timer_interrupt+0x3a (<102d024a>)
:| + func -1661 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -1660 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -1660 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -1660 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -1660 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -1660 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -1660 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -1660 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1659 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1659 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1659 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1659 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1659 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1659 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1659 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1658 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1561 -1658 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1658 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1658 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1658 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -1658 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1658 0378 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1657 0.397 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1657 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1657 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1656 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1656 0.247 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1656 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1656 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1656 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1656 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1655 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1655 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1655 0.301 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1655 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1655 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1655 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1654 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1654 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1654 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1654 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1654 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1654 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1654 0.127 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1653 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1653 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1653 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1653 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1653 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1653 0.292 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1652 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -1652 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1652 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1652 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1652 0.265 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -1651 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -1651 0.146 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -1651! 97.810 apic_timer_interrupt+0x52 (<102d024a>)
:| + begin 0x000000ef -1553 0.136 apic_timer_interrupt+0x3a (<10139d1e>)
:| + func -1553 0.166 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -1553 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -1553 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -1553 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -1553 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -1553 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -1552 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -1552 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1552 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1552 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1552 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1552 0.148 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1552 0.120 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1552 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1551 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1551 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1461 -1551 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1551 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1551 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1551 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -1550 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1550 0.319 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1550 0.403 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1550 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1549 0.126 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1549 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1549 0.213 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1549 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1549 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1549 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1548 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1548 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1548 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1548 0.305 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1548 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1547 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1547 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1547 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1547 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1547 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1547 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1547 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1547 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1546 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1546 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1546 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1546 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1546 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1546 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1546 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1545 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -1545 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1545 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1545 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1545 0.261 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -1544 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -1544 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -1544! 83.896 apic_timer_interrupt+0x52 (<10139d1e>)
:| + begin 0x000000ef -1460 0.132 apic_timer_interrupt+0x3a (<1013cb63>)
:| + func -1460 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -1460 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -1460 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -1460 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -1459 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -1459 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -1459 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -1459 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1459 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1459 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1459 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1459 0.133 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1458 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1458 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1458 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1458 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1361 -1458 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1458 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1458 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1457 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -1457 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1457 0.322 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1457 0.396 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1456 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1456 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1456 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1456 0.213 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1456 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1455 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1455 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1455 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1455 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1455 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1455 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1454 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1454 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1454 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1454 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1454 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1454 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1454 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1453 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1453 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1453 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1453 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1453 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1453 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1453 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1452 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1452 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1452 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -1452 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1452 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1451 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1451 0.264 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -1451 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -1451 0.146 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -1451 0.629 apic_timer_interrupt+0x52 (<1013cb63>)
: + func -1450 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1450 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1450 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1450 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1450 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1450 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1449 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1449 0.146 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1449 0.118 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1449 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1449 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1449 0.210 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1449 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1448 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1448 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1448 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1448 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1448 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1448 0.149 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -1448 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -1447 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1447 0.121 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1447 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -1447 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -1447 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -1447 0.123 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1447 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1447 0.155 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1446 0.193 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1446 0.152 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1446 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -1446 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1446 0.265 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1446 0.408 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 602] Loader: 29 -1445 0.203 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1445 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1445 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1445 0.285 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1444 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1444 0.167 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1444 0.233 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x465 [sigmatek_lrt])
:| + begin 0x80000001 -1444 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1444 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1444 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1443+ 1.098 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1442 0.120 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1442 0.118 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1442 0.185 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1442 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1442 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1442 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1441 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1441 0.141 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1441 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -1441 0.120 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1441 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1441 0.203 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1441 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1441 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1440 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1440 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1440 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1440 0.136 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1440 0.124 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| + begin 0x80000001 -1440 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| + end 0x80000001 -1440 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
: + func -1439 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1439 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1439 0.155 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1439 0.160 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1439 0.144 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1439 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 602] Loader: 29 -1439 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1438 0.286 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1438 0.319 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1438 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1438 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1438 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1437 0.154 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1437 0.314 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1437 0.191 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1437 0.161 __ipipe_do_sync_stage+0x9 (__ipipe_do_sync_pipeline+0x5e)
:| #end 0x80000000 -1437 0.114 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -1437 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -1436 0.126 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
: #func -1436 0.120 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -1436 0.142 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -1436 0.130 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -1436 0.145 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -1436 0.142 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -1436 0.136 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -1435 0.132 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -1435 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1435 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1435 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1435 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1435 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1435 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1434 0.114 local_bh_disable+0x3 (irq_enter+0x39)
: #func -1434 0.117 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -1434 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1434 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1434 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1434 0.117 tick_check_idle+0x8 (irq_enter+0x40)
: #func -1434 0.149 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -1434 0.203 ktime_get+0x9 (tick_check_idle+0x31)
: #func -1433 0.115 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -1433 0.114 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -1433 0.129 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -1433 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -1433 0.163 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -1433 0.117 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -1433 0.114 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -1432 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1432 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1432 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1432 0.117 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -1432 0.148 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -1432 0.111 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -1432 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1432 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1431 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1431 0.117 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -1431 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1431 0.161 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1431 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1431 0.115 wait_domain_nrt_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xff)
: #func -1431 0.114 __wake_up+0x8 (wait_domain_nrt_handler+0x33 [sigmatek_lrt])
: #func -1430 0.142 _raw_spin_lock_irqsave+0x5 (__wake_up+0x1a)
: #func -1430 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1430 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1430 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1430 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1430 0.118 __wake_up_common+0x9 (__wake_up+0x2e)
: #func -1430 0.118 ipipe_root_only+0x5 (__wake_up_common+0x1b)
:| #begin 0x80000001 -1430 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1429 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1429 0.123 __ipipe_spin_unlock_debug+0x3 (__wake_up+0x35)
: #func -1429 0.124 _raw_spin_unlock_irqrestore+0x4 (__wake_up+0x3e)
: #func -1429 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1429 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1429 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1429 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1428 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1428 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1428 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1428 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1428 0.130 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -1428 0.114 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -1428 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1428 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1427 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1427 0.142 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -1427 0.126 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -1427 0.124 idle_cpu+0x3 (irq_exit+0x50)
: #func -1427 0.118 tick_nohz_irq_exit+0x4 (irq_exit+0x79)
:| #begin 0x80000001 -1427 0.142 debug_smp_processor_id+0x36 (tick_nohz_irq_exit+0x13)
:| #end 0x80000001 -1427 0.129 debug_smp_processor_id+0x82 (tick_nohz_irq_exit+0x13)
: #func -1426 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_irq_exit+0x27)
:| #begin 0x80000001 -1426 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -1426 0.129 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -1426 0.154 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -1426 0.152 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -1426 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -1426 0.149 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1425 0.155 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -1425 0.141 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -1425 0.190 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -1425 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -1425 0.145 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -1425 0.141 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -1425 0.130 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -1424 0.138 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -1424 0.144 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -1424 0.127 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -1424 0.115 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -1424 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1424 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1424 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1424 0.155 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1423 0.124 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -1423 0.118 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1423 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1423 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1423 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1423 0.121 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -1423 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -1422 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -1422 0.142 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -1422 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1422 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1422 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1422 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1422 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -1422 0.121 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -1421 0.118 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1421 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1421 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1421 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1421 0.120 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1421 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1421 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1420 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1420 0.146 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -1420 0.141 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -1420 0.130 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -1420 0.121 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -1420 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1420 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1419 0.120 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1419 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1419 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1419 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1419 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
: #func -1419 0.126 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| #begin 0x80000000 -1419 0.181 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| +func -1419 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1418 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x00000051 -1418! 57.503 common_interrupt+0x51 (cpu_startup_entry+0x91)
:| +begin 0x000000ef -1361 0.172 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1361 0.172 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1360 0.126 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1360 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1360 0.139 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1360 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1360 0.151 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1360 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1360 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1359 0.129 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1359 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1359 0.219 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1359 0.141 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1359 0.114 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1359 0.135 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1359 0.190 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1358 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1261 -1358 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1358 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1358 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1358 0.178 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1358 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1357 0.148 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1357 0.319 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1357 0.319 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1357 0.212 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1356 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1356 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1356 0.234 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1356 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1356 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1356 0.204 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1355 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1355 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1355 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1355 0.322 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1355 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1355 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1354 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1354 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1354 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1354 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1354 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1354 0.129 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1354 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1353 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1353 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1353 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1353 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1353 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1353 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1353 0.298 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1352 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1352 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1352 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1352 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1352 0.127 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1352 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1351 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1351 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1351 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -1351! 90.100 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1261 0.173 apic_timer_interrupt+0x3a (need_resched+0x1)
:| +func -1261 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1260 0.123 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1260 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1260 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1260 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1260 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1260 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1260 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1259 0.127 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1259 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1259 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1259 0.138 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1259 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1259 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1259 0.194 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1258 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1161 -1258 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1258 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1258 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1258 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1258 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1257 0.151 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1257 0.308 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1257 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1257 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1257 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1256 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1256 0.234 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1256 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1256 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1256 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1256 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1255 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1255 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1255 0.301 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1255 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1255 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1255 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1254 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1254 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1254 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1254 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1254 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1254 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1254 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1253 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1253 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1253 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1253 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1253 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1253 0.299 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1252 0.221 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1252 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1252 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1252 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1252 0.127 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1252 0.282 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1251 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1251 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1251 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -1251! 90.222 apic_timer_interrupt+0x52 (need_resched+0x1)
:| +begin 0x000000ef -1161 0.175 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1161 0.176 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1160 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1160 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1160 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1160 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1160 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1160 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1160 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1159 0.126 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1159 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1159 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1159 0.133 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1159 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1159 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1159 0.198 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1158 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1061 -1158 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1158 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1158 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1158 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1158 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1157 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1157 0.308 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1157 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1157 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1157 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1156 0.121 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1156 0.234 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1156 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1156 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1156 0.187 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1156 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1155 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1155 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1155 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1155 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1155 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1155 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1154 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1154 0.126 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1154 0.148 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1154 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1154 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1154 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1154 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1154 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1153 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1153 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1153 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1153 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1153 0.299 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1152 0.225 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1152 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1152 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1152 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1152 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1152 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1151 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1151 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1151 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -1151! 90.241 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1061 0.185 apic_timer_interrupt+0x3a (cpu_startup_entry+0x91)
:| +func -1061 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1060 0.124 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1060 0.151 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -1060 0.114 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1060 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1060 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1060 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -1060 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1059 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1059 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -1059 0.216 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1059 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1059 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1059 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -1059 0.194 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1058 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-961 -1058 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1058 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1058 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1058 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1058 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1057 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1057 0.365 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1057 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -1057 0.234 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1056 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1056 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1056 0.228 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1056 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1056 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1056 0.182 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1055 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1055 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1055 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1055 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1055 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1055 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1054 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1054 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1054 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1054 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1054 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1054 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1054 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1053 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1053 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1053 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1053 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1053 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -1053 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1053 0.299 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1052 0.218 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1052 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1052 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1052 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1052 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1052 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1051 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1051 0.118 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -1051 0.164 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -1051! 90.100 apic_timer_interrupt+0x52 (cpu_startup_entry+0x91)
:| +begin 0x000000ef -961 0.175 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -961 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -960 0.123 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -960 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -960 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -960 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -960 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -960 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -960 0.176 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -959 0.127 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -959 0.121 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -959 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -959 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -959 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -959 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -959 0.175 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -958 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-861 -958 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -958 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -958 0.256 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -958 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -958 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -958 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -957 0.311 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -957 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -957 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -957 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -956 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -956 0.216 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -956 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -956 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -956 0.182 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -956 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -955 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -955 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -955 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -955 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -955 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -955 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -954 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -954 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -954 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -954 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -954 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -954 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -954 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -954 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -953 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -953 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -953 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -953 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -953 0.288 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -953 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -952 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -952 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -952 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -952 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -952 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -951 0.191 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -951 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +func -951 0.166 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +end 0x000000ef -951! 72.348 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x00000051 -879 0.179 common_interrupt+0x39 (cpu_startup_entry+0x91)
:| +func -878 0.142 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -878 0.111 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -878 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| +func -878 0.124 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -878 0.139 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -878 0.152 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -878 0.118 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -878 0.127 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -877 0.135 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -877 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -877+ 1.361 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -876 0.112 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -876 0.132 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -876 0.132 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -875+ 1.162 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -874 0.166 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -874 0.112 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| +func -874 0.187 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -874 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -874 0.141 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -874 0.138 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -873 0.121 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -873 0.136 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x5a [sigmatek_lrt])
:| # func -873+ 1.655 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -871 0.127 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -871+ 4.564 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x13d [sigmatek_lrt])
:| # func -867 0.117 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1a4 [sigmatek_lrt])
:| # func -867 0.154 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x1d2 [sigmatek_lrt])
:| # func -867 0.118 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x1e8 [sigmatek_lrt])
:| # func -866 0.118 xnarch_get_cpu_time+0x4 (irqhandler_fpga+0x225 [sigmatek_lrt])
:| # func -866 0.118 xnarch_tsc_to_ns+0x6 (xnarch_get_cpu_time+0x13)
:| # func -866 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x245 [sigmatek_lrt])
:| # func -866 0.120 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x25b [sigmatek_lrt])
:| # func -866 0.117 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x280 [sigmatek_lrt])
:| # func -866 0.115 period_update+0x4 [sigmatek_lrt] (irqhandler_fpga+0x287 [sigmatek_lrt])
:| # func -866 0.146 lrtdrv_monitoring_start_timed+0x4 [sigmatek_lrt] (period_update+0x1f [sigmatek_lrt])
:| # func -866 0.206 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -865 0.227 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -865 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -865 0.216 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -865 0.148 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -865 0.152 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -864 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 606] realtim 95 -864 0.227 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -864 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -864 0.163 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -864 0.201 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 512] VARAN_T 30 -864 0.236 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -863 0.135 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 604] cyclic: 30 -863 0.154 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -863 0.135 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 602] Loader: 29 -863 0.250 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -863 0.132 ipipe_raise_irq+0x8 (period_update+0x10f [sigmatek_lrt])
:| # func -863 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| # func -862 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -862 0.132 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| # func -862 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| # func -862 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -862 0.146 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| # func -862 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| # func -862 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -862 0.184 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| # func -861 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| # func -861 0.175 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -861+ 1.637 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -859+ 1.646 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -858 0.124 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0xe6 [sigmatek_lrt])
:| # func -858 0.123 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -858 0.172 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irqhandler_fpga+0x10d [sigmatek_lrt])
:| # func -857 0.115 rthal_irq_end+0x5 (xnintr_irq_handler+0x14d)
:| # func -857 0.123 irq_to_desc+0x3 (rthal_irq_end+0x1d)
:| # func -857 0.121 irq_to_desc+0x3 (rthal_irq_end+0x28)
:| # func -857 0.120 irq_to_desc+0x3 (rthal_irq_end+0x32)
:| # func -857 0.115 __ipipe_end_fasteoi_irq+0x3 (rthal_irq_end+0x38)
:| # func -857 0.115 release_ioapic_irq+0x5 (__ipipe_end_fasteoi_irq+0x16)
:| # func -857 0.152 __ipipe_spin_lock_irqsave+0x6 (release_ioapic_irq+0x17)
:| # func -856 0.114 io_apic_modify_irq.isra.9+0x7 (release_ioapic_irq+0x29)
:| # func -856 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| # func -856+ 1.355 io_apic_base+0x3 (native_io_apic_read+0x10)
:| # func -855 0.112 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| # func -855 0.121 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| # func -855 0.299 __ipipe_spin_unlock_irqrestore+0x4 (release_ioapic_irq+0x35)
:| # func -854 0.250 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # func -854+ 1.716 __xnlock_spin+0x4 (__xnlock_get_irqsave.constprop.43+0x3f)
:| # [ 0] -<?>- -1 -852 0.135 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -852 0.148 ipipe_send_ipi+0x6 (__xnpod_schedule+0x7d)
:| # func -852 0.176 default_send_IPI_mask_logical+0x6 (ipipe_send_ipi+0x40)
:| # func -852 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -852 0.304 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -852 0.308 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 606] realtim 95 -851 0.206 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -851 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -851 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -851 0.335 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -850 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -850 0.267 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x000000ef -850 0.130 apic_timer_interrupt+0x3a (hard_local_irq_enable+0x18)
:| + func -850 0.149 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -850 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -850 0.123 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -849 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -849 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -849 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -849 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -849 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -849 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -849 0.132 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -849 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -848 0.154 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -848 0.120 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -848 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -848 0.212 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -848 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-761 -848 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -848 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -847 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -847 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 606] realtim 95 -847 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -847 0.329 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -847 0.399 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -846 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -846 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -846 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -846 0.242 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -845 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -845 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -845 0.187 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -845 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -845 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -845 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -844 0.307 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -844 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -844 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -844 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -844 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -844 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -843 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -843 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -843 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -843 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -843 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -843 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -843 0.219 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -843 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -842 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -842 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -842 0.301 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -842 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 606] realtim 95 -841 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -841 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -841 0.141 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -841 0.296 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -841 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -841 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -840 0.283 apic_timer_interrupt+0x52 (hard_local_irq_enable+0x18)
: + func -840 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -840 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -840 0.129 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -840 0.243 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -840 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -839 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -839 0.170 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -839! 18.387 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -821 0.120 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -821 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -820 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -820 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -820 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -820 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -820 0.129 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -820 0.169 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -820 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -819 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -819 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -819 0.262 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -819 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -819 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -819 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -819 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -818 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -818 0.155 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -818 0.169 pci_locker_signal_stop+0x3 [sigmatek_lrt] (lrt_ioctl_work+0x768 [sigmatek_lrt])
:| + begin 0x80000001 -818 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -818 0.132 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -818 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -818 0.308 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -817 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -817 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -817 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -817 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -817 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -817 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -816 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -816 0.139 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -816 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -816 0.117 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -816 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -816 0.204 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -816 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -815 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -815 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -815 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -815 0.120 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -815 0.118 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -815 0.152 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -815 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -815 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -814 0.121 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -814 0.132 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -814 0.142 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -814 0.129 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -814 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -814 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -814 0.207 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -813 0.158 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -813 0.145 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -813 0.166 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 606] realtim 95 -813 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -813 0.307 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -813 0.405 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -812 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -812 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -812 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -812 0.359 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -811 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -811 0.164 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -811 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -811 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -811 0.124 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -811 0.225 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -810 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -810 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -810 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -810! 49.213 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
:| + begin 0x000000ef -761 0.129 apic_timer_interrupt+0x3a (<102d024a>)
:| + func -761 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -760 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -760 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -760 0.118 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -760 0.129 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -760 0.148 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -760 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -760 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -760 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -759 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -759 0.219 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -759 0.138 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -759 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -759 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -759 0.182 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -759 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-661 -758 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -758 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -758 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -758 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -758 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -758 0.334 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -757 0.396 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -757 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -757 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -757 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -756 0.236 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -756 0.164 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -756 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -756 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -756 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -756 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -755 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -755 0.308 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -755 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -755 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -755 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -754 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -754 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -754 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -754 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -754 0.130 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -754 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -754 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -754 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -753 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -753 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -753 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -753 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -753 0.288 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -753 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -752 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -752 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -752 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -752 0.262 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -752 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -751 0.146 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -751! 90.605 apic_timer_interrupt+0x52 (<102d024a>)
:| + begin 0x000000ef -661 0.130 apic_timer_interrupt+0x3a (<102d0254>)
:| + func -661 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -660 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -660 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -660 0.120 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -660 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -660 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -660 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -660 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -660 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -659 0.118 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -659 0.233 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -659 0.354 __xnlock_spin+0x4 (__xnlock_get+0x2b)
:| # func -659 0.151 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -659 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -658 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -658 0.207 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -658 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -658 0.132 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -658 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -657 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -657 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -657 0.320 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -657 0.396 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -656 0.215 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -656 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -656 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -656 0.221 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -656 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -656 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -655 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -655 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -655 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -655 0.240 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -655 0.298 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -654 0.120 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -654 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -654 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -654 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -654 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -654 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -654 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -654 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -653 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -653 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -653 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -653 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -653 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -653 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 615] -<?>- 99 -653 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -652 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -652 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 604] cyclic: 30 -652 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -652 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -652 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -651 0.261 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -651 0.117 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + func -651 0.145 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52 (<102d0254>)
:| + begin 0x000000ef -9 0.160 apic_timer_interrupt+0x3a (<080ac95a>)
:| + func -9 0.169 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -9 0.115 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -9 0.154 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| + func -9 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -9 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -8 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -8 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| + func -8 0.158 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -8 0.114 irq_hook_handler+0x5 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -8 0.120 lrtdrv_monitoring_irq+0x4 [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
:| # func -8 0.221 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -8 0.148 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -7 0.118 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -7 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 615] -<?>- 99 -7 0.231 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -7 0.124 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@38 -7 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -7 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -7 0.256 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -6 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 604] cyclic: 30 -6 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -6 0.381 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -6 0.405 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 615] -<?>- 99 -5 0.221 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -5 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -5 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -5 0.221 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -4 0.237 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -4 0.142 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -4 0.191 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -4 0.184 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -4 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -4 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -3 0.913 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -3 0.130 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2 0.142 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2 0.283 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1 0.252 xnshadow_sys_trace+0x6 (hisyscall_event+0x129)
: + func -1 0.206 ipipe_trace_frozen_reset+0x5 (xnshadow_sys_trace+0x5e)
: + func -1 0.117 __ipipe_global_path_lock+0x7 (ipipe_trace_frozen_reset+0x14)
: + func -1 0.133 __ipipe_spin_lock_irqsave+0x6 (__ipipe_global_path_lock+0x16)
:| + begin 0x80000001 -1 0.748 hard_local_irq_save+0x16 (__ipipe_spin_lock_irqsave+0x12)
:| # func 0 0.148 __ipipe_spin_unlock_irqcomplete+0x4 (__ipipe_global_path_unlock+0x5f)
:| + end 0x80000001 0 0.160 hard_local_irq_restore+0x15 (__ipipe_spin_unlock_irqcomplete+0x25)
< + freeze 0x000875a1 0 0.167 xnshadow_sys_trace+0x67 (hisyscall_event+0x129)
| + begin 0x80000001 0 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
| + end 0x80000001 0 0.139 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
| + begin 0x80000000 0 0.173 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
| + end 0x80000000 0 0.201 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
+ func 0 0.120 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
+ func 0 0.129 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
| + begin 0x80000001 1 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
| + end 0x80000001 1 0.136 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
+ func 1 0.126 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
| + begin 0x80000001 1 0.000 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 14:56 ` Johann Obermayr
@ 2015-07-13 15:24 ` Gilles Chanteperdrix
2015-07-13 16:58 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 15:24 UTC (permalink / raw)
To: johann.obermayr; +Cc: xenomai
Johann Obermayr wrote:
> without your application, there are no large latencies.
> with your application see frozen.txt (with latency -f)
I am confused. You mean "our application", not "your application", right?
lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
project.
> We see the problem only if one task (background) is accessing the SRAM
> on your PCI-Card. if we stop this task, all is ok.
Again: the Xenomai project does not make PCI-card. So, you probably mean
"our PCI-Card"?
> So we have a higher prior task (pci-locker), that interrupt the
> background task, so that the pci bus get free.
I am not sure I understand your explanations. But the trace is pretty clear:
At time -658 the timer is programmed to tick at -561.
> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> (<102d0254>)
But at that point the tick is delayed for 600us. And according to the
trace, the last traced function called before that delay is the function
ltdrv_monitoring_irq.
So, I do not know what this irq is doing, but I would suggest having a
close look at it.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 15:24 ` Gilles Chanteperdrix
@ 2015-07-13 16:58 ` Johann Obermayr
2015-07-13 17:21 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 16:58 UTC (permalink / raw)
To: xenomai
Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> Johann Obermayr wrote:
>> without your application, there are no large latencies.
>> with your application see frozen.txt (with latency -f)
> I am confused. You mean "our application", not "your application", right?
> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> project.
>
>> We see the problem only if one task (background) is accessing the SRAM
>> on your PCI-Card. if we stop this task, all is ok.
> Again: the Xenomai project does not make PCI-card. So, you probably mean
> "our PCI-Card"?
yes, our PCI-Card. (sorry for my bad english)
>> So we have a higher prior task (pci-locker), that interrupt the
>> background task, so that the pci bus get free.
> I am not sure I understand your explanations. But the trace is pretty clear:
>
> At time -658 the timer is programmed to tick at -561.
>
>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>
>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>> (<102d0254>)
> But at that point the tick is delayed for 600us. And according to the
> trace, the last traced function called before that delay is the function
>
> ltdrv_monitoring_irq.
>
> So, I do not know what this irq is doing, but I would suggest having a
> close look at it.
>
>
hello,
i have disable our lrtdrv_monitoring_irq.
Only have this callback
static void irq_hook_handler(unsigned int irq, unsigned int state)
{
if (fpga_interrupt == irq && state == 0x01)
{
time_fpga_irq = rt_timer_tsc();
}
}
same latency
regards
Johann
-------------- next part --------------
I-pipe frozen back-tracing service on 3.10.0/ipipe release #1
------------------------------------------------------------
CPU: 0, Freeze: 897546816952 cycles, Trace Points: 100 (+10)
Calibrated minimum trace-point overhead: 0.078 us
+----- Hard IRQs ('|': locked)
|+-- Xenomai
||+- Linux ('*': domain stalled, '+': current, '#': current+stalled)
||| +---------- Delay flag ('+': > 1 us, '!': > 10 us)
||| | +- NMI noise ('N')
||| | |
Type User Val. Time Delay Function (Parent)
:| # func -700 0.222 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -700 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -700 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -699 0.188 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -699 0.173 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -699 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -699 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -699 0.311 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -698 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -698 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -698 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -698 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -698 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -698 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -698 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -698 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -697 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -697 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -697 0.123 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -697 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -697 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -697 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 613] -<?>- 99 -697 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -696 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -696 0.270 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -696 0.273 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -696 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -695 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -695 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -695 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -695 0.200 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -695 0.163 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -695! 682.112 apic_timer_interrupt+0x52 (<102d0251>)
:| +begin 0x000000ef -12 0.145 apic_timer_interrupt+0x3a (<10139c1a>)
:| +func -12 0.158 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -12 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -12 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -12 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -12 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -12 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -11 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -11 0.216 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -11 0.149 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -11 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -11 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 613] -<?>- 99 -11 0.273 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -10 0.135 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-5 -10 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -10 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -10 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -10 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -10 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -10 0.135 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -9 0.366 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -9 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 613] -<?>- 99 -9 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -9 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -8 0.144 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -8 0.221 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -8 0.237 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -8 0.149 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -8 0.194 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -7 0.182 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -7 0.158 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -7 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -7+ 1.064 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -6 0.126 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -6 0.130 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -6 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -5 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -5 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -5 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -5 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -5 0.288 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -5 0.242 xnshadow_sys_trace+0x6 (hisyscall_event+0x129)
: + func -4 0.368 ipipe_trace_frozen_reset+0x5 (xnshadow_sys_trace+0x5e)
:| + begin 0x000000ef -4 0.155 apic_timer_interrupt+0x3a (__ipipe_global_path_lock+0x0)
:| + func -4 0.160 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -4 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -4 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -3 0.138 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -3 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -3 0.154 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -3 0.126 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -3 0.234 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -3 0.559 __xnlock_spin+0x4 (__xnlock_get+0x2b)
:| # func -2 0.136 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -2 0.176 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -2 0.136 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@94 -2 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -2 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1 0.356 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| + func -1 0.144 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + end 0x000000ef -1 0.201 apic_timer_interrupt+0x52 (__ipipe_global_path_lock+0x0)
: + func -1 0.132 __ipipe_global_path_lock+0x7 (ipipe_trace_frozen_reset+0x14)
: + func -1 0.136 __ipipe_spin_lock_irqsave+0x6 (__ipipe_global_path_lock+0x16)
:| + begin 0x80000001 0 0.657 hard_local_irq_save+0x16 (__ipipe_spin_lock_irqsave+0x12)
:| # func 0 0.148 __ipipe_spin_unlock_irqcomplete+0x4 (__ipipe_global_path_unlock+0x5f)
:| + end 0x80000001 0 0.166 hard_local_irq_restore+0x15 (__ipipe_spin_unlock_irqcomplete+0x25)
< + freeze 0x00091117 0 0.179 xnshadow_sys_trace+0x67 (hisyscall_event+0x129)
| + begin 0x80000001 0 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
| + end 0x80000001 0 0.149 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
| + begin 0x80000000 0 0.170 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
| + end 0x80000000 0 0.218 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
+ func 0 0.130 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
+ func 1 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
| + begin 0x80000001 1 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
| + end 0x80000001 1 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
+ func 1 0.138 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
| + begin 0x80000001 1 0.000 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 16:58 ` Johann Obermayr
@ 2015-07-13 17:21 ` Gilles Chanteperdrix
2015-07-13 17:26 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 17:21 UTC (permalink / raw)
To: johann.obermayr; +Cc: xenomai
Johann Obermayr wrote:
> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>> Johann Obermayr wrote:
>>> without your application, there are no large latencies.
>>> with your application see frozen.txt (with latency -f)
>> I am confused. You mean "our application", not "your application",
>> right?
>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>> project.
>>
>>> We see the problem only if one task (background) is accessing the SRAM
>>> on your PCI-Card. if we stop this task, all is ok.
>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>> "our PCI-Card"?
> yes, our PCI-Card. (sorry for my bad english)
>>> So we have a higher prior task (pci-locker), that interrupt the
>>> background task, so that the pci bus get free.
>> I am not sure I understand your explanations. But the trace is pretty
>> clear:
>>
>> At time -658 the timer is programmed to tick at -561.
>>
>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>
>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>> (<102d0254>)
>> But at that point the tick is delayed for 600us. And according to the
>> trace, the last traced function called before that delay is the function
>>
>> ltdrv_monitoring_irq.
>>
>> So, I do not know what this irq is doing, but I would suggest having a
>> close look at it.
>>
>>
> hello,
>
> i have disable our lrtdrv_monitoring_irq.
> Only have this callback
> static void irq_hook_handler(unsigned int irq, unsigned int state)
> {
> if (fpga_interrupt == irq && state == 0x01)
> {
> time_fpga_irq = rt_timer_tsc();
> }
> }
> same latency
Maybe, but your trace does not contain enough points to see it. The trace
should at least contain the "tick@" event which gets missed, so that we
can see how much the interrupt is delayed, and what was happening at the
time.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 17:21 ` Gilles Chanteperdrix
@ 2015-07-13 17:26 ` Johann Obermayr
2015-07-13 19:58 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 17:26 UTC (permalink / raw)
To: xenomai
Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> Johann Obermayr wrote:
>> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>>> Johann Obermayr wrote:
>>>> without your application, there are no large latencies.
>>>> with your application see frozen.txt (with latency -f)
>>> I am confused. You mean "our application", not "your application",
>>> right?
>>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>>> project.
>>>
>>>> We see the problem only if one task (background) is accessing the SRAM
>>>> on your PCI-Card. if we stop this task, all is ok.
>>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>>> "our PCI-Card"?
>> yes, our PCI-Card. (sorry for my bad english)
>>>> So we have a higher prior task (pci-locker), that interrupt the
>>>> background task, so that the pci bus get free.
>>> I am not sure I understand your explanations. But the trace is pretty
>>> clear:
>>>
>>> At time -658 the timer is programmed to tick at -561.
>>>
>>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>>> (<102d0254>)
>>> But at that point the tick is delayed for 600us. And according to the
>>> trace, the last traced function called before that delay is the function
>>>
>>> ltdrv_monitoring_irq.
>>>
>>> So, I do not know what this irq is doing, but I would suggest having a
>>> close look at it.
>>>
>>>
>> hello,
>>
>> i have disable our lrtdrv_monitoring_irq.
>> Only have this callback
>> static void irq_hook_handler(unsigned int irq, unsigned int state)
>> {
>> if (fpga_interrupt == irq && state == 0x01)
>> {
>> time_fpga_irq = rt_timer_tsc();
>> }
>> }
>> same latency
> Maybe, but your trace does not contain enough points to see it. The trace
> should at least contain the "tick@" event which gets missed, so that we
> can see how much the interrupt is delayed, and what was happening at the
> time.
>
>
Hi,
Sorry, here with more points.
regards
-------------- next part --------------
I-pipe frozen back-tracing service on 3.10.0/ipipe release #1
------------------------------------------------------------
CPU: 0, Freeze: 2447758391360 cycles, Trace Points: 2000 (+10)
Calibrated minimum trace-point overhead: 0.078 us
+----- Hard IRQs ('|': locked)
|+-- Xenomai
||+- Linux ('*': domain stalled, '+': current, '#': current+stalled)
||| +---------- Delay flag ('+': > 1 us, '!': > 10 us)
||| | +- NMI noise ('N')
||| | |
Type User Val. Time Delay Function (Parent)
:| + begin 0x80000001 -1829 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1829 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1828 0.132 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1828 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1828 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1828 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1828 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1828 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1828 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1828 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1827 0.307 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1827 0.265 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -1827 0.200 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1827 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1827 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1826 0.133 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1826 0.283 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1826 0.195 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1826 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1826! 90.465 apic_timer_interrupt+0x52 (<102d0251>)
:| +begin 0x000000ef -1735 0.142 apic_timer_interrupt+0x3a (<102d0251>)
:| +func -1735 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1735 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1735 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1735 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1734 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1734 0.169 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1734 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1734 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1734 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1734 0.114 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1734 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1734 0.185 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1733 0.129 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1636 -1733 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1733 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1733 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1733 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -1733 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1732 0.142 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1732 0.308 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1732 0.299 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1732 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1731 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1731 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1731 0.212 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1731 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1731 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1731 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1731 0.172 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1730 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1730 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1730 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1730 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1730 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1730 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1729 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1729 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1729 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1729 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1729 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1729 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1729 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1728 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1728 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1728 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1728 0.166 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1728 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1728 0.301 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1727 0.270 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -1727 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1727 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1727 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1727 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1727 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1726 0.194 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1726 0.164 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1726! 96.228 apic_timer_interrupt+0x52 (<102d0251>)
:| +begin 0x000000ef -1630 0.152 apic_timer_interrupt+0x3a (<10139c1a>)
:| +func -1630 0.158 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1629 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1629 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1629 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1629 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1629 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1629 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1629 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1628 0.149 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1628 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1628 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1628 0.185 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1628 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1536 -1628 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1628 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1627 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1627 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -1627 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1627 0.144 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1627 0.311 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1626 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1626 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1626 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1626 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1626 0.212 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1626 0.164 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1625 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1625 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1625 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1625 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1625 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1625 0.302 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1624 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1624 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1624 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1624 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1624 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1624 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1623 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1623 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1623 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1623 0.117 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1623 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1623 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1623 0.146 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1623 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1622 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1622 0.293 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1622 0.273 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -1622 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1621 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1621 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1621 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1621 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1621 0.194 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1621 0.163 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1620! 88.816 apic_timer_interrupt+0x52 (<10139c1a>)
:| +begin 0x000000ef -1532 0.142 apic_timer_interrupt+0x3a (<080ac951>)
:| +func -1532 0.149 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1531 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1531 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1531 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1531 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1531 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1531 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1531 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1530 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1530 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1530 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1530 0.185 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1530 0.124 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1436 -1530 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1530 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1529 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1529 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -1529 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1529 0.142 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1529 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1528 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1528 0.200 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1528 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1528 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1528 0.213 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1528 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1527 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1527 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1527 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1527 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1527 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1527 0.302 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1526 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1526 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1526 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1526 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1526 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1526 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1525 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1525 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1525 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1525 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1525 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1525 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1525 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1525 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1524 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1524 0.289 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1524 0.264 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -1524 0.200 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1523 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1523 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1523 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1523 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1523 0.194 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1523 0.164 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1522! 46.627 apic_timer_interrupt+0x52 (<080ac951>)
: +func -1476 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +func -1476 0.126 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +begin 0x80000001 -1476 0.190 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1475 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1475 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1475 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1475 0.127 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1475 0.152 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| + begin 0x80000001 -1475 0.221 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -1475 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -1474 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -1474 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -1474 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -1474 0.155 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +func -1474 0.117 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: +func -1474 0.114 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: +func -1474 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| +begin 0x80000000 -1473 0.233 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1473 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1473 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000001 -1473 0.160 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| +end 0x80000001 -1473 0.145 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: +func -1473 0.114 lrt_non_rt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x51)
: +func -1473 0.127 lrt_non_rt_ioctl_work+0xc [sigmatek_lrt] (lrt_non_rt_ioctl+0x10 [sigmatek_lrt])
: +func -1472 0.118 xnshadow_harden+0x9 (losyscall_event+0x88)
: +func -1472 0.112 down_interruptible+0x6 (xnshadow_harden+0x4f)
: +func -1472 0.149 _raw_spin_lock_irqsave+0x5 (down_interruptible+0x12)
: #func -1472 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1472 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1472 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1472 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1472 0.130 __ipipe_spin_unlock_debug+0x3 (down_interruptible+0x33)
: #func -1471 0.126 _raw_spin_unlock_irqrestore+0x4 (down_interruptible+0x3c)
: #func -1471 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1471 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1471 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1471 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1471 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1471 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1470 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1470 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1470 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1470 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1470 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1470 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1470 0.114 add_preempt_count+0x4 (xnshadow_harden+0x76)
: +func -1470 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1469 0.170 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1469 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1469 0.129 wake_up_process+0x4 (xnshadow_harden+0xe1)
: +func -1469 0.111 try_to_wake_up+0x9 (wake_up_process+0x2e)
: +func -1469 0.141 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -1469 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1469 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1468 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1468 0.164 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1468 0.141 select_task_rq_rt+0x6 (select_task_rq+0x15)
:| #begin 0x80000001 -1468 0.151 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -1468 0.127 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -1468 0.124 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -1468 0.111 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -1467 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1467 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1467 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1467 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1467 0.117 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -1467 0.115 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -1467 0.115 enqueue_task+0x8 (activate_task+0x21)
: #func -1467 0.155 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -1466 0.141 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1466 0.139 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1466 0.114 enqueue_task_rt+0x7 (enqueue_task+0x28)
: #func -1466 0.129 dequeue_rt_stack+0x9 (enqueue_task_rt+0x2c)
: #func -1466 0.158 cpupri_set+0x6 (enqueue_task_rt+0xca)
: #func -1466 0.124 update_rt_migration+0x4 (enqueue_task_rt+0xea)
: #func -1466 0.114 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -1466 0.138 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -1465 0.117 check_preempt_curr_rt+0x5 (check_preempt_curr+0x2c)
: #func -1465 0.141 resched_task+0x5 (check_preempt_curr_rt+0x82)
:| #begin 0x80000001 -1465 0.141 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -1465 0.135 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -1465 0.117 task_woken_rt+0x4 (ttwu_do_wakeup+0x5b)
: #func -1465 0.123 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -1465 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1464 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1464 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1464 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1464 0.127 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -1464 0.123 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -1464 0.138 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1464 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1464 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1463 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1463 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1463 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1463 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1463 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1463 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1463 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1462 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1462 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1462 0.123 schedule+0x3 (xnshadow_harden+0xe6)
: +func -1462 0.112 __schedule+0x9 (schedule+0x18)
: +func -1462 0.115 add_preempt_count+0x4 (__schedule+0x23)
: +func -1462 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1462 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1462 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1461 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1461 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1461 0.114 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1461 0.145 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1461 0.121 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -1461 0.111 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1461 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1460 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1460 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1460 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1460 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1460 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1460 0.132 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1460 0.117 sub_preempt_count+0x4 (__schedule+0x54)
: +func -1460 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1459 0.170 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1459 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1459 0.123 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1459 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1459 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1459 0.127 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1459 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1458 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1458 0.176 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1458 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1458 0.112 deactivate_task+0x4 (__schedule+0x10e)
: #func -1458 0.115 dequeue_task+0x8 (deactivate_task+0x21)
: #func -1458 0.142 update_rq_clock+0x6 (dequeue_task+0x1a)
: #func -1458 0.123 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -1457 0.133 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -1457 0.118 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -1457 0.142 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -1457 0.117 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -1457 0.123 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -1457 0.139 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -1457 0.126 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -1457 0.123 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -1456 0.120 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -1456 0.124 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -1456 0.132 dequeue_pushable_task+0x5 (pick_next_task_rt+0x5b)
: #func -1456 0.146 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1456 0.132 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1456 0.121 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1456 0.121 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1456 0.160 rt_task_set_mode+0x7 (taskSwitchLinux+0x8d [sigmatek_lrt])
: #func -1455 0.123 __ipipe_notify_kevent+0x9 (__schedule+0x391)
: #func -1455 0.118 ipipe_root_only+0x5 (__ipipe_notify_kevent+0x1b)
:| #begin 0x80000001 -1455 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1455 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1455 0.144 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x20)
:| #end 0x80000001 -1455 0.124 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x49)
: #func -1455 0.118 ipipe_kevent_hook+0x4 (__ipipe_notify_kevent+0x54)
: #func -1454 0.139 schedule_event+0x9 (ipipe_kevent_hook+0x1c)
:| #begin 0x80000000 -1454 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *#func -1454 0.160 xnsched_peek_rpi+0x3 (schedule_event+0x14c)
:| *#func -1454 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1454 0.130 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| #begin 0x80000001 -1454 0.126 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -1454 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -1453 0.201 __schedule+0x3a3 (schedule+0x18)
:| #func -1453 0.163 __switch_to+0x9 (__schedule+0x50a)
:| #func -1453 0.141 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1453 0.126 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1453 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1453 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1453 0.121 finish_task_switch+0x7 (__schedule+0x533)
: #func -1452 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1452 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1452 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1452 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1452 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1452 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1452 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1452 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1451 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1451 0.144 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1451 0.126 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1451 0.115 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1451 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1451 0.172 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1451 0.175 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1450 0.130 kthread_should_stop+0x3 (gatekeeper_thread+0xc3)
: +func -1450 0.117 rpi_pop+0x9 (gatekeeper_thread+0xf0)
:| +begin 0x80000000 -1450 0.170 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1450 0.158 xnsched_peek_rpi+0x3 (rpi_pop+0x6e)
:| *+func -1450 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1450 0.115 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -1450 0.120 xnsched_renice_root+0x7 (rpi_pop+0xb4)
:| +begin 0x80000000 -1449 0.209 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *+func -1449 0.127 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *+[ 603] cyclic: -1 -1449 0.146 xnsched_renice_root+0x82 (rpi_pop+0xb4)
:| *+func -1449 0.144 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| +end 0x80000000 -1449 0.132 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +begin 0x80000000 -1449 0.169 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| *+func -1449 0.141 xnpod_resume_thread+0x9 (gatekeeper_thread+0x11a)
:| *+[ 603] cyclic: 30 -1448 0.185 xnpod_resume_thread+0x39 (gatekeeper_thread+0x11a)
:| *+func -1448 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +end 0x80000000 -1448 0.115 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +func -1448 0.118 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| +begin 0x80000001 -1448 0.144 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| +end 0x80000001 -1448 0.132 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: +func -1448 0.118 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| +begin 0x80000001 -1447 0.136 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| +func -1447 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| +func -1447 0.129 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| +func -1447 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1447 0.148 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1447 0.172 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1447 0.129 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1446 0.133 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| # func -1446 0.182 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: -1 -1446 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1446 0.138 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1446 0.304 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1446 0.195 __switch_to+0x9 (__schedule+0x50a)
:| # func -1445 0.187 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| # func -1445 0.250 xnpod_switch_fpu+0x3 (xnshadow_harden+0x133)
:| # func -1445 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1445 0.132 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -1445 0.124 sys_rtdm_ioctl+0x3 (losyscall_event+0xb1)
: + func -1445 0.126 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -1444 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -1444 0.209 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -1444 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1444 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1444 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -1444 0.139 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -1444 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -1443 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -1443 0.172 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -1443 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -1443 0.120 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -1443 0.120 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -1443 0.130 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -1443 0.151 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -1443 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -1442 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -1442 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -1442 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -1442 0.166 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -1442 0.144 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -1442 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -1441 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1441 0.274 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1441 0.316 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 23] gatekee -1 -1441 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1441 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1440 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1440 0.130 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1440 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1440 0.195 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1440 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x80000001 -1440 0.185 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
: +func -1439 0.118 up+0x5 (gatekeeper_thread+0xb9)
: +func -1439 0.142 _raw_spin_lock_irqsave+0x5 (up+0x11)
: #func -1439 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1439 0.126 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1439 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1439 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1439 0.121 __ipipe_spin_unlock_debug+0x3 (up+0x2e)
: #func -1438 0.123 _raw_spin_unlock_irqrestore+0x4 (up+0x37)
: #func -1438 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1438 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1438 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1438 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1438 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1438 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1438 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1437 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1437 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1437 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1437 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1437 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1437 0.114 schedule+0x3 (gatekeeper_thread+0xbe)
: +func -1437 0.115 __schedule+0x9 (schedule+0x18)
: +func -1436 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -1436 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1436 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1436 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1436 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1436 0.127 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1436 0.117 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1436 0.145 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1435 0.262 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
:| #begin 0x000000ef -1435 0.149 apic_timer_interrupt+0x3a (__ipipe_trace+0x40b)
:| #func -1435 0.176 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| #func -1435 0.121 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| #func -1435 0.133 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| #func -1435 0.138 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| #func -1434 0.145 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| #func -1434 0.190 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| #*func -1434 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| #*func -1434 0.179 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| #*func -1434 0.149 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| #*func -1434 0.124 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| #*func -1433 0.144 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| #*[ 620] -<?>- 99 -1433 0.207 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| #*func -1433 0.133 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| #*event tick@-1336 -1433 0.121 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| #*func -1433 0.142 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| #*func -1433 0.267 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| #*func -1432 0.185 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 23] gatekee -1 -1432 0.129 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -1432 0.141 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -1432 0.328 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| #*func -1432 0.345 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 620] -<?>- 99 -1431 0.231 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -1431 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -1431 0.130 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -1431 0.225 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -1431 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| #*func -1430 0.146 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +*end 0x80000000 -1430 0.228 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +*begin 0x80000001 -1430 0.191 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +*end 0x80000001 -1430 0.163 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +*begin 0x80000000 -1430 0.172 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +*end 0x80000000 -1430 0.350 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: +*func -1429 0.132 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +*func -1429 0.138 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +*begin 0x80000001 -1429 0.197 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| +*end 0x80000001 -1429 0.136 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +*func -1429 0.127 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +*begin 0x80000001 -1428 0.155 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +*end 0x80000001 -1428 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +*func -1428 0.141 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +*func -1428 0.126 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: +*func -1428 0.127 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: +*func -1428 0.132 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| +*begin 0x80000000 -1428 0.187 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| #*func -1427 0.161 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| #*func -1427 0.187 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 620] -<?>- 99 -1427 0.130 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -1427 0.295 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -1427 0.239 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 23] gatekee -1 -1426 0.206 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -1426 0.130 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -1426 0.130 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -1426 0.146 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -1426 0.296 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| #func -1426 0.195 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -1425 0.161 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #end 0x000000ef -1425 0.247 apic_timer_interrupt+0x52 (__ipipe_trace+0x40b)
: #func -1425 0.123 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1425 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1425 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1425 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1424 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1424 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1424 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1424 0.133 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1424 0.117 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1424 0.176 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1424 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1423 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1423 0.114 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1423 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1423 0.172 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1423 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1423 0.115 deactivate_task+0x4 (__schedule+0x10e)
: #func -1423 0.117 dequeue_task+0x8 (deactivate_task+0x21)
: #func -1423 0.145 update_rq_clock+0x6 (dequeue_task+0x1a)
:| #begin 0x80000001 -1422 0.142 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1422 0.145 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1422 0.112 dequeue_task_rt+0x5 (dequeue_task+0x28)
: #func -1422 0.127 update_curr_rt+0x9 (dequeue_task_rt+0x13)
: #func -1422 0.118 sched_avg_update+0x9 (update_curr_rt+0xc9)
: #func -1422 0.144 dequeue_rt_stack+0x9 (dequeue_task_rt+0x1b)
: #func -1422 0.176 cpupri_set+0x6 (dequeue_rt_stack+0x104)
: #func -1421 0.120 update_rt_migration+0x4 (dequeue_rt_stack+0x121)
: #func -1421 0.141 dequeue_pushable_task+0x5 (dequeue_task_rt+0x24)
: #func -1421 0.124 pre_schedule_rt+0x3 (__schedule+0x242)
: #func -1421 0.120 pull_rt_task+0x9 (pre_schedule_rt+0x18)
: #func -1421 0.127 idle_balance+0x9 (__schedule+0x252)
: #func -1421 0.124 _raw_spin_unlock+0x3 (idle_balance+0x5f)
: #func -1421 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1421 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1420 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1420 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1420 0.118 __rcu_read_lock+0x3 (idle_balance+0x64)
: #func -1420 0.112 __rcu_read_unlock+0x4 (idle_balance+0xee)
: #func -1420 0.114 _raw_spin_lock+0x4 (idle_balance+0xf5)
: #func -1420 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1420 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1420 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1419 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1419 0.117 put_prev_task_rt+0x5 (__schedule+0x275)
: #func -1419 0.126 update_curr_rt+0x9 (put_prev_task_rt+0x14)
: #func -1419 0.124 pick_next_task_fair+0x7 (__schedule+0x28c)
: #func -1419 0.120 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -1419 0.117 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -1419 0.120 pick_next_task_fair+0x7 (__schedule+0x299)
: #func -1419 0.124 pick_next_task_idle+0x3 (__schedule+0x299)
: #func -1418 0.117 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1418 0.149 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1418 0.138 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1418 0.121 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1418 0.133 rt_task_set_mode+0x7 (taskSwitchLinux+0x8d [sigmatek_lrt])
:| #begin 0x80000000 -1418 0.146 __schedule+0x3a3 (schedule+0x18)
:| #func -1418 0.190 __switch_to+0x9 (__schedule+0x50a)
:| #func -1417 0.139 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1417 0.124 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1417 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1417 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1417 0.121 finish_task_switch+0x7 (__schedule+0x533)
: #func -1417 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1417 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1416 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1416 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1416 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1416 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1416 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1416 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1416 0.164 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1415 0.152 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1415 0.135 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1415 0.141 _raw_spin_lock_irqsave+0x5 (post_schedule.part.48+0xc)
: #func -1415 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1415 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1415 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1415 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1415 0.114 post_schedule_idle+0x3 (post_schedule.part.48+0x22)
: #func -1414 0.120 __ipipe_spin_unlock_debug+0x3 (post_schedule.part.48+0x29)
: #func -1414 0.120 _raw_spin_unlock_irqrestore+0x4 (post_schedule.part.48+0x32)
: #func -1414 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1414 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1414 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1414 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1414 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1414 0.130 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1413 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1413 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1413 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1413 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1413 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1413 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1413 0.114 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1412 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1412 0.172 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1412 0.172 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1412 0.117 add_preempt_count+0x4 (schedule_preempt_disabled+0x21)
: +func -1412 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1412 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1412 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1411 0.138 tick_nohz_idle_enter+0x4 (cpu_startup_entry+0x24)
: +func -1411 0.121 set_cpu_sd_state_idle+0x4 (tick_nohz_idle_enter+0x37)
:| +begin 0x80000001 -1411 0.142 debug_smp_processor_id+0x36 (set_cpu_sd_state_idle+0xe)
:| +end 0x80000001 -1411 0.127 debug_smp_processor_id+0x82 (set_cpu_sd_state_idle+0xe)
: +func -1411 0.114 __rcu_read_lock+0x3 (set_cpu_sd_state_idle+0x15)
: +func -1411 0.141 __rcu_read_unlock+0x4 (set_cpu_sd_state_idle+0x4d)
:| #begin 0x80000001 -1411 0.152 debug_smp_processor_id+0x36 (tick_nohz_idle_enter+0x46)
:| #end 0x80000001 -1410 0.130 debug_smp_processor_id+0x82 (tick_nohz_idle_enter+0x46)
: #func -1410 0.121 __tick_nohz_idle_enter+0x9 (tick_nohz_idle_enter+0x5b)
:| #begin 0x80000001 -1410 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -1410 0.129 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -1410 0.190 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -1410 0.151 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -1410 0.144 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -1409 0.160 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1409 0.148 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -1409 0.141 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -1409 0.148 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -1409 0.136 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -1409 0.141 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -1409 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -1408 0.130 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -1408 0.120 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -1408 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -1408 0.127 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -1408 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -1408 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1408 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1408 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1407 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1407 0.129 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -1407 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1407 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1407 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1407 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1407 0.124 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -1406 0.151 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -1406 0.129 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -1406 0.141 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -1406 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1406 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1406 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1406 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1405 0.124 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -1405 0.124 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -1405 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1405 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1405 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1405 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1405 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1405 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1404 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1404 0.170 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1404 0.135 nohz_balance_enter_idle+0x3 (__tick_nohz_idle_enter+0x24a)
: #func -1404 0.124 calc_load_enter_idle+0x4 (__tick_nohz_idle_enter+0x24f)
:| #begin 0x80000001 -1404 0.160 debug_smp_processor_id+0x36 (calc_load_enter_idle+0x13)
:| #end 0x80000001 -1404 0.139 debug_smp_processor_id+0x82 (calc_load_enter_idle+0x13)
: #func -1404 0.112 hrtimer_start+0x3 (__tick_nohz_idle_enter+0x2e1)
: #func -1403 0.115 __hrtimer_start_range_ns+0x9 (hrtimer_start+0x14)
: #func -1403 0.118 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -1403 0.139 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1403 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1403 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1403 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1403 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1403 0.142 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -1402 0.129 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -1402 0.130 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -1402 0.130 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
:| #begin 0x80000001 -1402 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1402 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1402 0.115 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1402 0.132 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -1401 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -1401 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -1401 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -1401 0.133 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -1401 0.114 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -1401 0.124 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -1401 0.149 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -1401 0.118 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -1400 0.154 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -1400 0.129 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -1400 0.181 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -1400 0.145 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1400 0.126 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -1400 0.115 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1400 0.123 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1399 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1399 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1399 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1399 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1399 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1399 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1399 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1398 0.146 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1398 0.118 ipipe_unstall_root+0x3 (tick_nohz_idle_enter+0x60)
:| #begin 0x80000000 -1398 0.139 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1398 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1398 0.132 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x80000001 -1398 0.158 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1398 0.151 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1397 0.115 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1397 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1397 0.149 rcu_idle_enter+0x5 (cpu_startup_entry+0x70)
:| #begin 0x80000001 -1397 0.152 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1397 0.130 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1397 0.121 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1397 0.151 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1396 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1396 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1396 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1396 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1396 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1396 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x75)
:| #end 0x80000001 -1396 0.129 debug_smp_processor_id+0x82 (cpu_startup_entry+0x75)
: #func -1396 0.120 ipipe_unstall_root+0x3 (cpu_startup_entry+0x8d)
:| #begin 0x80000000 -1395 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1395 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1395! 59.960 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x000000ef -1335 0.155 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1335 0.176 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1335 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1335 0.135 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1335 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1334 0.157 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1334 0.190 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1334 0.136 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1334 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1334 0.152 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1334 0.120 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1333 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1333 0.188 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1333 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1236 -1333 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1333 0.142 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1333 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1333 0.184 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 23] gatekee -1 -1332 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1332 0.152 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1332 0.316 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1332 0.317 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1331 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1331 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1331 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1331 0.255 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1331 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1331 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1330 0.195 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1330 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1330 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1330 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1330 0.313 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1329 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1329 0.129 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1329 0.198 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1329 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1329 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1329 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1329 0.142 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1328 0.144 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1328 0.117 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1328 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1328 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1328 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1328 0.146 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1328 0.166 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1327 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1327 0.305 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1327 0.213 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1327 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1327 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1327 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1326 0.142 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1326 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1326 0.193 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1326 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1326! 90.505 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1235 0.158 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1235 0.178 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1235 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1235 0.120 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1235 0.141 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1234 0.142 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1234 0.188 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1234 0.126 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1234 0.215 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1234 0.161 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1234 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1233 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1233 0.198 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1233 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1136 -1233 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1233 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1233 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1233 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1232 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1232 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1232 0.304 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1232 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1231 0.231 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1231 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1231 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1231 0.219 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1231 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1231 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1230 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1230 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1230 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1230 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1230 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1230 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1229 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1229 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1229 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1229 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1229 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1229 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1229 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1228 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1228 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1228 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1228 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1228 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1228 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1228 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1227 0.299 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1227 0.216 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1227 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1227 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1227 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1227 0.142 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1226 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1226 0.187 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1226 0.164 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1226! 90.630 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1135 0.160 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1135 0.175 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1135 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1135 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1135 0.144 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1134 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1134 0.187 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1134 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1134 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1134 0.141 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1134 0.114 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1134 0.135 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1133 0.175 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1133 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1036 -1133 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1133 0.132 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1133 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1133 0.179 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1132 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1132 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1132 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1132 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1132 0.201 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1131 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1131 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1131 0.216 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1131 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1131 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1131 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1130 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1130 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1130 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1130 0.301 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1130 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1130 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1129 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1129 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1129 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1129 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1129 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1129 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1129 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1128 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1128 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1128 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1128 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1128 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1128 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1128 0.285 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1127 0.216 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1127 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1127 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1127 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1127 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1127 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1126 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1126 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1126! 90.810 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x000000ef -1035 0.160 apic_timer_interrupt+0x3a (need_resched+0x0)
:| +func -1035 0.176 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1035 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1035 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1035 0.142 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1034 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1034 0.187 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1034 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1034 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1034 0.141 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1034 0.114 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1034 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -1033 0.172 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1033 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-936 -1033 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1033 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1033 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1033 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1032 0.121 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1032 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1032 0.308 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1032 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -1032 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1031 0.129 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1031 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1031 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1031 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1031 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1031 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1030 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1030 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1030 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1030 0.301 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1030 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1030 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1029 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1029 0.132 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1029 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1029 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1029 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1029 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1029 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1028 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1028 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1028 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1028 0.149 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1028 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -1028 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1028 0.283 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1027 0.216 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1027 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1027 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1027 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1027 0.127 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1027 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1026 0.188 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1026 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1026! 32.816 apic_timer_interrupt+0x52 (need_resched+0x0)
:| +begin 0x00000051 -993 0.155 common_interrupt+0x39 (need_resched+0x0)
:| +func -993 0.136 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -993 0.145 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -993 0.132 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -993 0.141 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -992 0.170 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -992 0.124 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -992 0.130 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -992 0.129 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -992 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -992+ 1.357 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -990 0.109 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -990 0.127 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -990 0.142 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -990+ 1.161 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -989 0.166 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -989 0.184 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -988 0.144 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -988 0.144 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -988 0.146 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -988+ 6.133 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -982 0.146 xnarch_get_cpu_time+0x4 (irqhandler_fpga+0x132 [sigmatek_lrt])
:| # func -982 0.123 xnarch_tsc_to_ns+0x6 (xnarch_get_cpu_time+0x13)
:| # func -982 0.120 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x14c [sigmatek_lrt])
:| # func -982 0.118 period_update+0x4 [sigmatek_lrt] (irqhandler_fpga+0x163 [sigmatek_lrt])
:| # func -981 0.182 lrtdrv_monitoring_start_timed+0x4 [sigmatek_lrt] (period_update+0x1f [sigmatek_lrt])
:| # func -981 0.204 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -981 0.225 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -981 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -981 0.234 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -980 0.149 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -980 0.161 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -980 0.146 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 605] realtim 95 -980 0.230 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -980 0.144 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -980 0.158 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -979 0.178 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 511] VARAN_T 30 -979 0.233 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -979 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 603] cyclic: 30 -979 0.172 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -979 0.157 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 601] Loader: 29 -979 0.227 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -978 0.117 ipipe_raise_irq+0x8 (period_update+0x10f [sigmatek_lrt])
:| # func -978 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| # func -978 0.132 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| # func -978 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| # func -978 0.148 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| # func -978 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| # func -978 0.185 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| # func -977 0.161 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| # func -977+ 1.665 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -976+ 1.645 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -974 0.175 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -974 0.115 rthal_irq_end+0x5 (xnintr_irq_handler+0x14d)
:| # func -974 0.135 irq_to_desc+0x3 (rthal_irq_end+0x1d)
:| # func -973 0.120 irq_to_desc+0x3 (rthal_irq_end+0x28)
:| # func -973 0.120 irq_to_desc+0x3 (rthal_irq_end+0x32)
:| # func -973 0.114 __ipipe_end_fasteoi_irq+0x3 (rthal_irq_end+0x38)
:| # func -973 0.121 release_ioapic_irq+0x5 (__ipipe_end_fasteoi_irq+0x16)
:| # func -973 0.152 __ipipe_spin_lock_irqsave+0x6 (release_ioapic_irq+0x17)
:| # func -973 0.114 io_apic_modify_irq.isra.9+0x7 (release_ioapic_irq+0x29)
:| # func -973 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| # func -973+ 1.342 io_apic_base+0x3 (native_io_apic_read+0x10)
:| # func -971 0.111 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| # func -971 0.123 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| # func -971 0.201 __ipipe_spin_unlock_irqrestore+0x4 (release_ioapic_irq+0x35)
:| # func -971 0.178 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -971 0.145 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -971 0.155 ipipe_send_ipi+0x6 (__xnpod_schedule+0x7d)
:| # func -970 0.188 default_send_IPI_mask_logical+0x6 (ipipe_send_ipi+0x40)
:| # func -970 0.138 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -970 0.264 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -970 0.314 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 605] realtim 95 -969 0.261 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -969 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -969 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -969 0.302 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -969 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -969 0.173 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -968 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -968 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -968 0.123 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -968 0.265 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -968 0.191 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -968 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -967 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -967! 18.485 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -949 0.132 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -949 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -948 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -948 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -948 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -948 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -948 0.124 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -948 0.148 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -948 0.118 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -947 0.117 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -947 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -947 0.261 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -947 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -947 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -947 0.142 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -947 0.129 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -946 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -946 0.154 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -946 0.173 pci_locker_signal_stop+0x3 [sigmatek_lrt] (lrt_ioctl_work+0x768 [sigmatek_lrt])
:| + begin 0x80000001 -946 0.181 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -946 0.130 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -946 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -946 0.331 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -945 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -945 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -945 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -945 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -945 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -945 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -944 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -944 0.139 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -944 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -944 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -944 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -944 0.201 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -944 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -943 0.123 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -943 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -943 0.127 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -943 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -943 0.133 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -943 0.152 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -943 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -943 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -942 0.124 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -942 0.132 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -942 0.167 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -942 0.127 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -942 0.120 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -942 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -942 0.219 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -941 0.161 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -941 0.152 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -941 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 605] realtim 95 -941 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -941 0.337 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -940 0.418 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -940 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -940 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -940 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -940 0.322 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -939 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -939 0.154 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -939 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -939 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -939 0.123 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -939 0.236 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -938 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -938 0.164 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -938 0.175 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -938 0.947 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -937 0.126 __ipipe_syscall_root+0x5 (system_call+0x30)
: + func -937 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -937 0.190 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -936 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -936 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -936 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -936 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -936 0.146 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -936 0.127 xnshadow_relax+0xc (hisyscall_event+0x201)
:| + begin 0x80000000 -936 0.158 hard_local_irq_disable+0x15 (xnshadow_relax+0x30)
:| # func -935 0.207 rpi_push+0x9 (xnshadow_relax+0x4d)
:| # func -935 0.145 xnsched_renice_root+0x7 (rpi_push+0x8e)
:| # func -935 0.139 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| # [ 0] -<?>- 30 -935 0.148 xnsched_renice_root+0x82 (rpi_push+0x8e)
:| # func -935 0.166 schedule_linux_call+0x9 (xnshadow_relax+0x5b)
:| # func -935 0.197 __ipipe_set_irq_pending+0x6 (schedule_linux_call+0x83)
:| # func -934 0.182 xnpod_suspend_thread+0x9 (xnshadow_relax+0x8c)
:| # func -934 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -934 0.267 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x000000ef -934 0.139 apic_timer_interrupt+0x3a (hard_local_irq_enable+0x18)
:| + func -934 0.160 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -934 0.133 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -933 0.121 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -933 0.135 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -933 0.146 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -933 0.161 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -933 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -933 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -933 0.144 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -932 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -932 0.135 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -932 0.207 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -932 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-836 -932 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -932 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -932 0.258 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -931 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -931 0.135 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -931 0.341 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -931 0.418 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -930 0.209 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -930 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -930 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -930 0.233 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -930 0.166 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -929 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -929 0.190 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -929 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -929 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -929 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -929 0.322 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -928 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -928 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -928 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -928 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -928 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -928 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -928 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -927 0.130 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -927 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -927 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -927 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -927 0.178 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -927 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -927 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -926 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -926 0.307 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -926 0.242 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- 30 -926 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -926 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -925 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -925 0.148 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -925 0.301 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -925 0.191 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -925 0.148 __ipipe_do_sync_stage+0x9 (__ipipe_do_sync_pipeline+0x5e)
:| #end 0x80000000 -925 0.114 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -924 0.133 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -924 0.120 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -924 0.142 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -924 0.130 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -924 0.145 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -924 0.142 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -924 0.129 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -923 0.136 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -923 0.141 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -923 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -923 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -923 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -923 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -923 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -922 0.115 local_bh_disable+0x3 (irq_enter+0x39)
: #func -922 0.115 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -922 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -922 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -922 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -922 0.114 tick_check_idle+0x8 (irq_enter+0x40)
: #func -922 0.151 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -922 0.215 ktime_get+0x9 (tick_check_idle+0x31)
: #func -921 0.115 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -921 0.114 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -921 0.141 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -921 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -921 0.158 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -921 0.144 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -921 0.111 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -920 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -920 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -920 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -920 0.115 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -920 0.130 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -920 0.114 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -920 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -920 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -919 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -919 0.118 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -919 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -919 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -919 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -919 0.117 wait_domain_nrt_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xff)
: #func -919 0.114 __wake_up+0x8 (wait_domain_nrt_handler+0x33 [sigmatek_lrt])
: #func -918 0.141 _raw_spin_lock_irqsave+0x5 (__wake_up+0x1a)
: #func -918 0.123 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -918 0.115 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -918 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -918 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -918 0.118 __wake_up_common+0x9 (__wake_up+0x2e)
: #func -918 0.117 ipipe_root_only+0x5 (__wake_up_common+0x1b)
:| #begin 0x80000001 -917 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -917 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -917 0.114 __ipipe_spin_unlock_debug+0x3 (__wake_up+0x35)
: #func -917 0.124 _raw_spin_unlock_irqrestore+0x4 (__wake_up+0x3e)
: #func -917 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -917 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -917 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -917 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -916 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -916 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -916 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -916 0.136 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -916 0.127 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -916 0.127 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -916 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -916 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -915 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -915 0.142 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -915 0.127 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -915 0.144 idle_cpu+0x3 (irq_exit+0x50)
: #func -915 0.121 tick_nohz_irq_exit+0x4 (irq_exit+0x79)
:| #begin 0x80000001 -915 0.152 debug_smp_processor_id+0x36 (tick_nohz_irq_exit+0x13)
:| #end 0x80000001 -915 0.129 debug_smp_processor_id+0x82 (tick_nohz_irq_exit+0x13)
: #func -914 0.129 __tick_nohz_idle_enter+0x9 (tick_nohz_irq_exit+0x27)
:| #begin 0x80000001 -914 0.151 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -914 0.133 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -914 0.155 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -914 0.142 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -914 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -914 0.148 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -913 0.148 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -913 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -913 0.190 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -913 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -913 0.157 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -913 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -912 0.132 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -912 0.121 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -912 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -912 0.127 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -912 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -912 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -912 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -912 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -911 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -911 0.151 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -911 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -911 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -911 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -911 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -911 0.121 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -910 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -910 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -910 0.141 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -910 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -910 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -910 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -910 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -909 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -909 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -909 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -909 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -909 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -909 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -909 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -909 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -908 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -908 0.161 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -908 0.146 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -908 0.141 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -908 0.130 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -908 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -908 0.151 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -907 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -907 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -907 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -907 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -907 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -907 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
:| #begin 0x80000000 -907 0.173 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| #end 0x80000000 -906 0.114 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -906 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xe4)
: #func -906 0.120 irq_enter+0x4 (__ipipe_do_sync_stage+0xe9)
:| #begin 0x80000001 -906 0.141 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -906 0.127 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -906 0.146 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -906 0.142 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -906 0.130 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -905 0.133 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -905 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -905 0.129 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -905 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -905 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -905 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -905 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -904 0.115 local_bh_disable+0x3 (irq_enter+0x39)
: #func -904 0.114 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -904 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -904 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -904 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -904 0.114 tick_check_idle+0x8 (irq_enter+0x40)
: #func -904 0.118 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -904 0.145 ktime_get+0x9 (tick_check_idle+0x31)
: #func -903 0.117 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -903 0.114 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -903 0.129 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -903 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -903 0.157 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -903 0.115 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -903 0.114 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -903 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -902 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -902 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -902 0.115 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -902 0.130 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -902 0.114 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -902 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -902 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -902 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -901 0.118 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -901 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -901 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -901 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -901 0.160 rthal_apc_handler+0x6 (__ipipe_do_sync_stage+0xff)
: #func -901 0.123 lostage_handler+0xc (rthal_apc_handler+0x6b)
:| #begin 0x80000001 -901 0.142 debug_smp_processor_id+0x36 (lostage_handler+0x16)
:| #end 0x80000001 -900 0.142 debug_smp_processor_id+0x82 (lostage_handler+0x16)
: #func -900 0.115 xnsched_renice_root+0x7 (lostage_handler+0xbb)
:| #begin 0x80000000 -900 0.231 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *#func -900 0.126 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *#[ 0] -<?>- -1 -900 0.151 xnsched_renice_root+0x82 (lostage_handler+0xbb)
:| *#func -900 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| #end 0x80000000 -899 0.120 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -899 0.120 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #begin 0x80000001 -899 0.142 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| #end 0x80000001 -899 0.133 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: #func -899 0.118 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| #begin 0x80000001 -899 0.141 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| #func -899 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| #func -899 0.142 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| #func -898 0.118 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| #func -898 0.149 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| #func -898 0.173 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| #*func -898 0.127 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| #*func -898 0.145 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| #*func -898 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 0] -<?>- -1 -898 0.130 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -897 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -897 0.268 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| #*func -897 0.368 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 601] Loader: 29 -897 0.212 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -896 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -896 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -896 0.326 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -896 0.144 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +*end 0x80000000 -896 0.193 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: +*func -896 0.242 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x465 [sigmatek_lrt])
:| +*begin 0x80000001 -895 0.182 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +*end 0x80000001 -895 0.141 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +*begin 0x80000000 -895 0.172 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +*end 0x80000000 -895+ 1.254 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: +*func -894 0.135 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: +*func -893 0.129 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| +*begin 0x80000001 -893 0.197 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| +*end 0x80000001 -893 0.136 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +*func -893 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +*begin 0x80000001 -893 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +*end 0x80000001 -893 0.133 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +*func -893 0.152 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: +*func -892 0.118 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: +*func -892 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: +*func -892 0.123 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| +*begin 0x80000000 -892 0.219 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| #*func -892 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| +*end 0x80000000 -892 0.133 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| +*begin 0x80000001 -892 0.154 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| +*end 0x80000001 -891 0.126 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: +*func -891 0.117 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: +*func -891 0.118 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: +*func -891 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| +*begin 0x80000001 -891 0.164 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
:| +*end 0x80000001 -891 0.146 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x41d [sigmatek_lrt])
: +*func -891 0.127 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: +*func -890 0.120 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| +*begin 0x80000000 -890 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| #*func -890 0.184 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| #*func -890 0.155 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| #*func -890 0.181 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 601] Loader: 29 -890 0.124 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -890 0.261 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #*func -889 0.329 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #*[ 0] -<?>- -1 -889 0.212 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| #*func -889 0.130 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| #*func -889 0.135 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| #*func -888 0.149 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| #*func -888 0.292 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| #func -888 0.197 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -888 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #end 0x80000001 -888 0.161 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
: #func -888 0.123 wake_up_process+0x4 (lostage_handler+0xcb)
: #func -887 0.123 try_to_wake_up+0x9 (wake_up_process+0x2e)
: #func -887 0.141 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -887 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -887 0.126 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -887 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -887 0.160 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -887 0.129 select_task_rq_rt+0x6 (select_task_rq+0x15)
:| #begin 0x80000001 -886 0.144 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -886 0.127 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -886 0.133 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -886 0.111 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -886 0.117 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -886 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -886 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -886 0.146 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -885 0.117 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -885 0.115 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -885 0.117 enqueue_task+0x8 (activate_task+0x21)
: #func -885 0.170 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -885 0.141 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -885 0.158 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -885 0.114 enqueue_task_rt+0x7 (enqueue_task+0x28)
: #func -884 0.129 dequeue_rt_stack+0x9 (enqueue_task_rt+0x2c)
: #func -884 0.176 cpupri_set+0x6 (enqueue_task_rt+0xca)
: #func -884 0.123 update_rt_migration+0x4 (enqueue_task_rt+0xea)
: #func -884 0.115 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -884 0.123 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -884 0.145 resched_task+0x5 (check_preempt_curr+0x3c)
:| #begin 0x80000001 -884 0.142 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -883 0.135 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -883 0.123 task_woken_rt+0x4 (ttwu_do_wakeup+0x5b)
: #func -883 0.124 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -883 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -883 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -883 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -883 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -883 0.114 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -882 0.126 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -882 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -882 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -882 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -882 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -882 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -882 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -882 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -881 0.187 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -881 0.132 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -881 0.114 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -881 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -881 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -881 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -880 0.142 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -880 0.126 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -880 0.136 idle_cpu+0x3 (irq_exit+0x50)
: #func -880 0.146 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -880 0.141 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -880 0.136 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -880 0.117 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -880 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -879 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -879 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -879 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -879 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -879 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -879 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
:| #begin 0x80000000 -879 0.194 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| +func -878 0.166 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x00000051 -878 0.237 common_interrupt+0x51 (need_resched+0x0)
:| +begin 0x80000001 -878 0.151 debug_smp_processor_id+0x36 (cpu_startup_entry+0x9f)
:| +end 0x80000001 -878 0.132 debug_smp_processor_id+0x82 (cpu_startup_entry+0x9f)
: +func -878 0.152 rcu_idle_exit+0x6 (cpu_startup_entry+0xb8)
:| #begin 0x80000001 -878 0.154 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -877 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -877 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -877 0.152 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -877 0.139 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -877 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -877 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -877 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -876 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -876 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -876 0.148 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -876 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -876 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -876 0.130 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
: +func -876 0.127 tick_nohz_idle_exit+0x7 (cpu_startup_entry+0x148)
:| +begin 0x80000001 -875 0.152 debug_smp_processor_id+0x36 (tick_nohz_idle_exit+0x16)
:| +end 0x80000001 -875 0.145 debug_smp_processor_id+0x82 (tick_nohz_idle_exit+0x16)
: #func -875 0.149 ktime_get+0x9 (tick_nohz_idle_exit+0x62)
: #func -875 0.114 tick_do_update_jiffies64+0x9 (tick_nohz_idle_exit+0x8b)
: #func -875 0.121 update_cpu_load_nohz+0x5 (tick_nohz_idle_exit+0x90)
:| #begin 0x80000001 -875 0.151 debug_smp_processor_id+0x36 (update_cpu_load_nohz+0x14)
:| #end 0x80000001 -875 0.127 debug_smp_processor_id+0x82 (update_cpu_load_nohz+0x14)
: #func -875 0.114 _raw_spin_lock+0x4 (update_cpu_load_nohz+0x2d)
: #func -874 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -874 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -874 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -874 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -874 0.129 __update_cpu_load+0x9 (update_cpu_load_nohz+0x40)
: #func -874 0.121 sched_avg_update+0x9 (__update_cpu_load+0xa9)
: #func -874 0.124 _raw_spin_unlock+0x3 (update_cpu_load_nohz+0x47)
: #func -873 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -873 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -873 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -873 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -873 0.124 calc_load_exit_idle+0x4 (tick_nohz_idle_exit+0x95)
:| #begin 0x80000001 -873 0.152 debug_smp_processor_id+0x36 (calc_load_exit_idle+0x13)
:| #end 0x80000001 -873 0.132 debug_smp_processor_id+0x82 (calc_load_exit_idle+0x13)
: #func -873 0.114 hrtimer_cancel+0x4 (tick_nohz_idle_exit+0xa9)
: #func -872 0.112 hrtimer_try_to_cancel+0x7 (hrtimer_cancel+0x12)
: #func -872 0.117 lock_hrtimer_base.isra.17+0x6 (hrtimer_try_to_cancel+0x1e)
: #func -872 0.139 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -872 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -872 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -872 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -872 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -871 0.142 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -871 0.129 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -871 0.126 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -871 0.135 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
: #func -871 0.111 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -871 0.124 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -871 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -871 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -870 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -870 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -870 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -870 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -870 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -870 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -870 0.123 hrtimer_forward+0xc (tick_nohz_idle_exit+0xd2)
: #func -870 0.121 ktime_add_safe+0x6 (hrtimer_forward+0x103)
: #func -869 0.120 ktime_add_safe+0x6 (hrtimer_forward+0x11d)
: #func -869 0.114 hrtimer_start_range_ns+0x3 (hrtimer_start_expires.constprop.9+0x1d)
: #func -869 0.127 __hrtimer_start_range_ns+0x9 (hrtimer_start_range_ns+0x15)
: #func -869 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -869 0.142 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -869 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -869 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -869 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -868 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -868 0.151 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -868 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -868 0.124 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -868 0.130 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -868 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -868 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -867 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -867 0.135 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -867 0.121 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -867 0.123 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -867 0.149 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -867 0.118 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -867 0.154 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -866 0.133 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -866 0.184 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -866 0.146 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -866 0.126 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -866 0.114 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -866 0.124 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -866 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -865 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -865 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -865 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -865 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -865 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -865 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -865 0.161 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -865 0.118 account_idle_ticks+0x3 (tick_nohz_idle_exit+0x122)
: #func -864 0.129 account_idle_time+0x6 (account_idle_ticks+0xd)
:| #begin 0x80000001 -864 0.141 debug_smp_processor_id+0x36 (account_idle_time+0x1c)
:| #end 0x80000001 -864 0.136 debug_smp_processor_id+0x82 (account_idle_time+0x1c)
:| #begin 0x80000001 -864 0.151 debug_smp_processor_id+0x36 (account_idle_time+0x28)
:| #end 0x80000001 -864 0.129 debug_smp_processor_id+0x82 (account_idle_time+0x28)
: #func -864 0.121 ipipe_unstall_root+0x3 (tick_nohz_idle_exit+0x127)
:| #begin 0x80000000 -864 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -863 0.193 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -863 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -863 0.117 schedule_preempt_disabled+0x3 (cpu_startup_entry+0x14d)
: +func -863 0.115 sub_preempt_count+0x4 (schedule_preempt_disabled+0x12)
: +func -863 0.126 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -863 0.170 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -863 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -862 0.115 schedule+0x3 (schedule_preempt_disabled+0x17)
: +func -862 0.115 __schedule+0x9 (schedule+0x18)
: +func -862 0.114 add_preempt_count+0x4 (__schedule+0x23)
: +func -862 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -862 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -862 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -862 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -862 0.135 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -861 0.120 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -861 0.145 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -861 0.120 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -861 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -861 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -861 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -861 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -860 0.124 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -860 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -860 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -860 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -860 0.118 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -860 0.176 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -860 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -860 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -859 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -859 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -859 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -859 0.169 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -859 0.130 pre_schedule_idle+0x3 (__schedule+0x242)
: #func -859 0.115 put_prev_task_idle+0x3 (__schedule+0x275)
: #func -859 0.130 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -858 0.132 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -858 0.148 dequeue_pushable_task+0x5 (pick_next_task_rt+0x5b)
: #func -858 0.132 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -858 0.127 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -858 0.121 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -858 0.127 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -858 0.133 rt_task_set_mode+0x7 (taskSwitchLinux+0x8d [sigmatek_lrt])
: #func -858 0.111 __ipipe_notify_kevent+0x9 (__schedule+0x391)
: #func -857 0.126 ipipe_root_only+0x5 (__ipipe_notify_kevent+0x1b)
:| #begin 0x80000001 -857 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -857 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -857 0.142 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x20)
:| #end 0x80000001 -857 0.133 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x49)
: #func -857 0.115 ipipe_kevent_hook+0x4 (__ipipe_notify_kevent+0x54)
: #func -857 0.141 schedule_event+0x9 (ipipe_kevent_hook+0x1c)
: #func -856 0.121 xnsched_renice_root+0x7 (schedule_event+0x1f0)
:| #begin 0x80000000 -856 0.167 hard_local_irq_disable+0x15 (xnsched_renice_root+0x1e)
:| *#func -856 0.130 xnsched_track_policy+0x6 (xnsched_renice_root+0x6a)
:| *#[ 0] -<?>- 30 -856 0.146 xnsched_renice_root+0x82 (schedule_event+0x1f0)
:| *#func -856 0.135 __ipipe_restore_head+0x4 (xnsched_renice_root+0xd6)
:| #end 0x80000000 -856 0.117 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -856 0.118 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #begin 0x80000001 -856 0.141 ipipe_get_current_domain+0x1d (__xnpod_schedule+0x15)
:| #end 0x80000001 -855 0.124 ipipe_get_current_domain+0x67 (__xnpod_schedule+0x15)
: #func -855 0.120 ipipe_raise_irq+0x8 (__xnpod_schedule+0x26)
:| #begin 0x80000001 -855 0.129 hard_local_irq_save+0x16 (ipipe_raise_irq+0x14)
:| #func -855 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| #func -855 0.132 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| #func -855 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| #func -855 0.141 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| #func -854 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| #*func -854 0.127 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| #*func -854 0.133 xnpod_schedule_handler+0x3 (__ipipe_dispatch_irq+0x182)
:| #*func -854 0.173 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| #*[ 0] -<?>- 30 -854 0.129 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| #*func -854 0.282 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| #func -853 0.172 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -853 0.154 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #end 0x80000001 -853 0.132 hard_local_irq_restore+0x15 (ipipe_raise_irq+0x45)
:| #begin 0x80000001 -853 0.127 hard_local_irq_save+0x16 (__ipipe_notify_kevent+0x5b)
:| #end 0x80000001 -853 0.135 hard_local_irq_restore+0x15 (__ipipe_notify_kevent+0x6b)
:| #begin 0x80000000 -853 0.206 __schedule+0x3a3 (schedule+0x18)
:| #func -853 0.418 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| #func -852 0.197 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| #func -852 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| #end 0x000000ef -852 0.246 apic_timer_interrupt+0x52 (hard_local_irq_enable+0x18)
:| #begin 0x80000000 -852 0.173 hard_local_irq_disable+0x15 (xnpod_suspend_thread+0x17d)
:| *#func -851 0.145 ipipe_unstall_head+0x4 (xnshadow_relax+0x91)
:| #end 0x80000000 -851 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_head+0x2e)
: #func -851 0.124 __ipipe_reenter_root+0x4 (xnshadow_relax+0xa2)
:| #begin 0x80000001 -851 0.141 debug_smp_processor_id+0x36 (__ipipe_reenter_root+0x13)
:| #end 0x80000001 -851 0.130 debug_smp_processor_id+0x82 (__ipipe_reenter_root+0x13)
: #func -851 0.114 finish_task_switch+0x7 (__ipipe_reenter_root+0x28)
: #func -851 0.126 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -850 0.121 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -850 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -850 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -850 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -850 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -850 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -850 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -849 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -849 0.118 sub_preempt_count+0x4 (__ipipe_reenter_root+0x42)
: +func -849 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -849 0.170 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -849 0.194 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -849 0.210 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -849 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: +func -848 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| +begin 0x80000001 -848 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| +end 0x80000001 -848 0.135 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: +func -848 0.121 losyscall_event+0x9 (ipipe_syscall_hook+0x6b)
:| +begin 0x80000001 -848 0.184 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| +end 0x80000001 -848 0.145 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| +begin 0x80000000 -848 0.179 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| +end 0x80000000 -847 0.141 __ipipe_syscall_root+0x9e (system_call+0x30)
: +func -847 0.120 SyS_select+0x9 (syscall_call+0x7)
: +func -847 0.115 _copy_from_user+0x7 (SyS_select+0x39)
: +func -847 0.117 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -847 0.176 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -847 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -847 0.166 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -846 0.126 poll_select_set_timeout+0x6 (SyS_select+0x60)
: +func -846 0.118 core_sys_select+0xc (SyS_select+0x80)
: +func -846 0.123 __rcu_read_lock+0x3 (core_sys_select+0x31)
: +func -846 0.129 __rcu_read_unlock+0x4 (core_sys_select+0x47)
: +func -846 0.114 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -846 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -846 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -846 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -845 0.138 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -845 0.118 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -845 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -845 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -845 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -845 0.193 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -845 0.118 do_select+0xc (core_sys_select+0x12b)
: +func -844 0.146 __rcu_read_lock+0x3 (do_select+0x23)
: +func -844 0.120 __rcu_read_unlock+0x4 (do_select+0xe5)
: +func -844 0.173 poll_initwait+0x3 (do_select+0xfd)
: +func -844 0.118 fget_light+0x6 (do_select+0x2aa)
: +func -844 0.136 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -844 0.138 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -844 0.120 eventfd_poll+0x6 (do_select+0x30f)
: +func -843 0.149 _raw_spin_lock_irqsave+0x5 (eventfd_poll+0x2f)
: #func -843 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -843 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -843 0.158 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -843 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -843 0.123 __ipipe_spin_unlock_debug+0x3 (eventfd_poll+0x68)
: #func -843 0.132 _raw_spin_unlock_irqrestore+0x4 (eventfd_poll+0x71)
: #func -843 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -842 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -842 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -842 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -842 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -842 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -842 0.191 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -842 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -841 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -841 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -841 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -841 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -841 0.160 fput+0x5 (do_select+0x327)
: +func -841 0.115 fget_light+0x6 (do_select+0x2aa)
: +func -841 0.142 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -840 0.136 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -840 0.126 sock_poll+0x5 (do_select+0x30f)
: +func -840 0.145 tcp_poll+0x6 (sock_poll+0x17)
: +func -840 0.123 fput+0x5 (do_select+0x327)
: +func -840 0.136 _cond_resched+0x3 (do_select+0x3f1)
: +func -840 0.124 poll_freewait+0x9 (do_select+0x4ab)
: +func -840 0.132 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -840 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -839 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -839 0.141 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -839 0.126 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -839 0.176 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -839 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -839 0.142 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -839+ 3.370 poll_select_copy_remaining+0x9 (SyS_select+0x90)
:| +begin 0x000000ef -835 0.127 apic_timer_interrupt+0x3a (<102d024a>)
:| +func -835 0.155 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -835 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -835 0.121 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -835 0.141 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -834 0.155 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -834 0.172 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -834 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -834 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -834 0.136 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -834 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -834 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -833 0.194 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -833 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-736 -833 0.115 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -833 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -833 0.258 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -833 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- 30 -832 0.130 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -832 0.142 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -832 0.334 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -832 0.311 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -832 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -831 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -831 0.124 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -831 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -831 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -831 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -831 0.206 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -830 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -830 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -830 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -830 0.368 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -830 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -829 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -829 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -829 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -829 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -829 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -829 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -829 0.133 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -828 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -828 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -828 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -828 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -828 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -828 0.166 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -828 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -828 0.302 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -827 0.285 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -827 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -827 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -827 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -827 0.135 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -826 0.276 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -826 0.195 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -826 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -826! 90.572 apic_timer_interrupt+0x52 (<102d024a>)
:| +begin 0x000000ef -735 0.127 apic_timer_interrupt+0x3a (<102d0251>)
:| +func -735 0.152 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -735 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -735 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -735 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -735 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -734 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -734 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -734 0.215 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -734 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -734 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -734 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -733 0.188 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -733 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-636 -733 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -733 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -733 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -733 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -732 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -732 0.141 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -732 0.360 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -732 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -732 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -731 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -731 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -731 0.231 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -731 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -731 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -731 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -730 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -730 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -730 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -730 0.301 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -730 0.121 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -730 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -729 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -729 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -729 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -729 0.144 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -729 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -729 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -729 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -728 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -728 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -728 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -728 0.146 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -728 0.166 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 620] -<?>- 99 -728 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -728 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -727 0.265 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 603] cyclic: 30 -727 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -727 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -727 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -727 0.144 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -726 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -726 0.195 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -726 0.163 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -726! 717.050 apic_timer_interrupt+0x52 (<102d0251>)
:| +begin 0x000000ef -9 0.158 apic_timer_interrupt+0x3a (<10139c1a>)
:| +func -9 0.158 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -8 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -8 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -8 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -8 0.146 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -8 0.169 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -8 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -8 0.218 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -7 0.149 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -7 0.115 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -7 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 620] -<?>- 99 -7 0.298 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -7 0.123 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@63 -7 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -7 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -6 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -6 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 603] cyclic: 30 -6 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -6 0.141 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -6 0.372 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -5 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 620] -<?>- 99 -5 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -5 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -5 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -5 0.231 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -4 0.219 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -4 0.141 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -4 0.198 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -4 0.181 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -4 0.152 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -3 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -3 0.877 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -2 0.124 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1 0.273 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1 0.233 xnshadow_sys_trace+0x6 (hisyscall_event+0x129)
: + func -1 0.212 ipipe_trace_frozen_reset+0x5 (xnshadow_sys_trace+0x5e)
: + func -1 0.124 __ipipe_global_path_lock+0x7 (ipipe_trace_frozen_reset+0x14)
: + func -1 0.120 __ipipe_spin_lock_irqsave+0x6 (__ipipe_global_path_lock+0x16)
:| + begin 0x80000001 0 0.648 hard_local_irq_save+0x16 (__ipipe_spin_lock_irqsave+0x12)
:| # func 0 0.148 __ipipe_spin_unlock_irqcomplete+0x4 (__ipipe_global_path_unlock+0x5f)
:| + end 0x80000001 0 0.155 hard_local_irq_restore+0x15 (__ipipe_spin_unlock_irqcomplete+0x25)
< + freeze 0x0009990f 0 0.161 xnshadow_sys_trace+0x67 (hisyscall_event+0x129)
| + begin 0x80000001 0 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
| + end 0x80000001 0 0.139 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
| + begin 0x80000000 0 0.169 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
| + end 0x80000000 0 0.203 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
+ func 0 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
+ func 0 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
| + begin 0x80000001 1 0.190 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
| + end 0x80000001 1 0.149 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
+ func 1 0.126 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
| + begin 0x80000001 1 0.000 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 17:26 ` Johann Obermayr
@ 2015-07-13 19:58 ` Gilles Chanteperdrix
2015-07-13 20:23 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 19:58 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
> Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> >Johann Obermayr wrote:
> >>Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> >>>Johann Obermayr wrote:
> >>>>without your application, there are no large latencies.
> >>>>with your application see frozen.txt (with latency -f)
> >>>I am confused. You mean "our application", not "your application",
> >>>right?
> >>>lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> >>>project.
> >>>
> >>>>We see the problem only if one task (background) is accessing the SRAM
> >>>>on your PCI-Card. if we stop this task, all is ok.
> >>>Again: the Xenomai project does not make PCI-card. So, you probably mean
> >>>"our PCI-Card"?
> >>yes, our PCI-Card. (sorry for my bad english)
> >>>>So we have a higher prior task (pci-locker), that interrupt the
> >>>>background task, so that the pci bus get free.
> >>>I am not sure I understand your explanations. But the trace is pretty
> >>>clear:
> >>>
> >>>At time -658 the timer is programmed to tick at -561.
> >>>
> >>>>:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> >>>>:| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> >>>>[sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> >>>>:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> >>>>(<102d0254>)
> >>>But at that point the tick is delayed for 600us. And according to the
> >>>trace, the last traced function called before that delay is the function
> >>>
> >>>ltdrv_monitoring_irq.
> >>>
> >>>So, I do not know what this irq is doing, but I would suggest having a
> >>>close look at it.
> >>>
> >>>
> >>hello,
> >>
> >>i have disable our lrtdrv_monitoring_irq.
> >>Only have this callback
> >>static void irq_hook_handler(unsigned int irq, unsigned int state)
> >>{
> >> if (fpga_interrupt == irq && state == 0x01)
> >> {
> >> time_fpga_irq = rt_timer_tsc();
> >> }
> >>}
> >>same latency
> >Maybe, but your trace does not contain enough points to see it. The trace
> >should at least contain the "tick@" event which gets missed, so that we
> >can see how much the interrupt is delayed, and what was happening at the
> >time.
> >
> >
> Hi,
>
> Sorry, here with more points.
Ok, what is irq_hook_handler ?
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 19:58 ` Gilles Chanteperdrix
@ 2015-07-13 20:23 ` Johann Obermayr
2015-07-13 20:31 ` Gilles Chanteperdrix
2015-07-13 20:35 ` Gilles Chanteperdrix
0 siblings, 2 replies; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 20:23 UTC (permalink / raw)
Cc: xenomai
Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
> On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
>> Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
>>> Johann Obermayr wrote:
>>>> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>>>>> Johann Obermayr wrote:
>>>>>> without your application, there are no large latencies.
>>>>>> with your application see frozen.txt (with latency -f)
>>>>> I am confused. You mean "our application", not "your application",
>>>>> right?
>>>>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>>>>> project.
>>>>>
>>>>>> We see the problem only if one task (background) is accessing the SRAM
>>>>>> on your PCI-Card. if we stop this task, all is ok.
>>>>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>>>>> "our PCI-Card"?
>>>> yes, our PCI-Card. (sorry for my bad english)
>>>>>> So we have a higher prior task (pci-locker), that interrupt the
>>>>>> background task, so that the pci bus get free.
>>>>> I am not sure I understand your explanations. But the trace is pretty
>>>>> clear:
>>>>>
>>>>> At time -658 the timer is programmed to tick at -561.
>>>>>
>>>>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>>>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>>>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>>>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>>>>> (<102d0254>)
>>>>> But at that point the tick is delayed for 600us. And according to the
>>>>> trace, the last traced function called before that delay is the function
>>>>>
>>>>> ltdrv_monitoring_irq.
>>>>>
>>>>> So, I do not know what this irq is doing, but I would suggest having a
>>>>> close look at it.
>>>>>
>>>>>
>>>> hello,
>>>>
>>>> i have disable our lrtdrv_monitoring_irq.
>>>> Only have this callback
>>>> static void irq_hook_handler(unsigned int irq, unsigned int state)
>>>> {
>>>> if (fpga_interrupt == irq && state == 0x01)
>>>> {
>>>> time_fpga_irq = rt_timer_tsc();
>>>> }
>>>> }
>>>> same latency
>>> Maybe, but your trace does not contain enough points to see it. The trace
>>> should at least contain the "tick@" event which gets missed, so that we
>>> can see how much the interrupt is delayed, and what was happening at the
>>> time.
>>>
>>>
>> Hi,
>>
>> Sorry, here with more points.
> Ok, what is irq_hook_handler ?
>
Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to
the PC. But internal in the FPGA there
are different IRQ sources. One of them is our Tick.
So we measure the time from __ipipe_handle_irq to the our
rtdm_irq_request handler.
In our handler we can check, if it our Tick and than we can calc the
correct time to start our pci_locker task 50us before next Tick-irq.
It's a callback function from some irq function ipipe_raise_irq,
__ipipe_do_IRQ, __ipipe_handle_irq
for our own tracing and it save the fpga irq time.
Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the
end of the function.
Regards
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:23 ` Johann Obermayr
@ 2015-07-13 20:31 ` Gilles Chanteperdrix
2015-07-13 20:45 ` Johann Obermayr
2015-07-13 20:35 ` Gilles Chanteperdrix
1 sibling, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 20:31 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
> Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
> >On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
> >>Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> >>>Johann Obermayr wrote:
> >>>>Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> >>>>>Johann Obermayr wrote:
> >>>>>>without your application, there are no large latencies.
> >>>>>>with your application see frozen.txt (with latency -f)
> >>>>>I am confused. You mean "our application", not "your application",
> >>>>>right?
> >>>>>lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> >>>>>project.
> >>>>>
> >>>>>>We see the problem only if one task (background) is accessing the SRAM
> >>>>>>on your PCI-Card. if we stop this task, all is ok.
> >>>>>Again: the Xenomai project does not make PCI-card. So, you probably mean
> >>>>>"our PCI-Card"?
> >>>>yes, our PCI-Card. (sorry for my bad english)
> >>>>>>So we have a higher prior task (pci-locker), that interrupt the
> >>>>>>background task, so that the pci bus get free.
> >>>>>I am not sure I understand your explanations. But the trace is pretty
> >>>>>clear:
> >>>>>
> >>>>>At time -658 the timer is programmed to tick at -561.
> >>>>>
> >>>>>>:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> >>>>>>:| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> >>>>>>[sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> >>>>>>:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> >>>>>>(<102d0254>)
> >>>>>But at that point the tick is delayed for 600us. And according to the
> >>>>>trace, the last traced function called before that delay is the function
> >>>>>
> >>>>>ltdrv_monitoring_irq.
> >>>>>
> >>>>>So, I do not know what this irq is doing, but I would suggest having a
> >>>>>close look at it.
> >>>>>
> >>>>>
> >>>>hello,
> >>>>
> >>>>i have disable our lrtdrv_monitoring_irq.
> >>>>Only have this callback
> >>>>static void irq_hook_handler(unsigned int irq, unsigned int state)
> >>>>{
> >>>> if (fpga_interrupt == irq && state == 0x01)
> >>>> {
> >>>> time_fpga_irq = rt_timer_tsc();
> >>>> }
> >>>>}
> >>>>same latency
> >>>Maybe, but your trace does not contain enough points to see it. The trace
> >>>should at least contain the "tick@" event which gets missed, so that we
> >>>can see how much the interrupt is delayed, and what was happening at the
> >>>time.
> >>>
> >>>
> >>Hi,
> >>
> >>Sorry, here with more points.
> >Ok, what is irq_hook_handler ?
> >
> Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
> PC. But internal in the FPGA there
> are different IRQ sources. One of them is our Tick.
> So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
> handler.
> In our handler we can check, if it our Tick and than we can calc the correct
> time to start our pci_locker task 50us before next Tick-irq.
>
> It's a callback function from some irq function ipipe_raise_irq,
> __ipipe_do_IRQ, __ipipe_handle_irq
> for our own tracing and it save the fpga irq time.
> Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
> of the function.
I see two weird things in your traces:
- irq_hook_handler which is taking a lot of time
- or some APIC related functions taking a lot of time.
Are you sure your system is not one of those which disable the APIC
during idle period. Is your system UP or SMP?
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:23 ` Johann Obermayr
2015-07-13 20:31 ` Gilles Chanteperdrix
@ 2015-07-13 20:35 ` Gilles Chanteperdrix
1 sibling, 0 replies; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 20:35 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
> Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
> >On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
> >>Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> >>>Johann Obermayr wrote:
> >>>>Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> >>>>>Johann Obermayr wrote:
> >>>>>>without your application, there are no large latencies.
> >>>>>>with your application see frozen.txt (with latency -f)
> >>>>>I am confused. You mean "our application", not "your application",
> >>>>>right?
> >>>>>lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> >>>>>project.
> >>>>>
> >>>>>>We see the problem only if one task (background) is accessing the SRAM
> >>>>>>on your PCI-Card. if we stop this task, all is ok.
> >>>>>Again: the Xenomai project does not make PCI-card. So, you probably mean
> >>>>>"our PCI-Card"?
> >>>>yes, our PCI-Card. (sorry for my bad english)
> >>>>>>So we have a higher prior task (pci-locker), that interrupt the
> >>>>>>background task, so that the pci bus get free.
> >>>>>I am not sure I understand your explanations. But the trace is pretty
> >>>>>clear:
> >>>>>
> >>>>>At time -658 the timer is programmed to tick at -561.
> >>>>>
> >>>>>>:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> >>>>>>:| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> >>>>>>[sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> >>>>>>:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> >>>>>>(<102d0254>)
> >>>>>But at that point the tick is delayed for 600us. And according to the
> >>>>>trace, the last traced function called before that delay is the function
> >>>>>
> >>>>>ltdrv_monitoring_irq.
> >>>>>
> >>>>>So, I do not know what this irq is doing, but I would suggest having a
> >>>>>close look at it.
> >>>>>
> >>>>>
> >>>>hello,
> >>>>
> >>>>i have disable our lrtdrv_monitoring_irq.
> >>>>Only have this callback
> >>>>static void irq_hook_handler(unsigned int irq, unsigned int state)
> >>>>{
> >>>> if (fpga_interrupt == irq && state == 0x01)
> >>>> {
> >>>> time_fpga_irq = rt_timer_tsc();
> >>>> }
> >>>>}
> >>>>same latency
> >>>Maybe, but your trace does not contain enough points to see it. The trace
> >>>should at least contain the "tick@" event which gets missed, so that we
> >>>can see how much the interrupt is delayed, and what was happening at the
> >>>time.
> >>>
> >>>
> >>Hi,
> >>
> >>Sorry, here with more points.
> >Ok, what is irq_hook_handler ?
> >
> Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
> PC. But internal in the FPGA there
> are different IRQ sources.
The proper way to handle an interrupt which generates multiple IRQ
sources is with a "chained handler".
> One of them is our Tick.
> So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
> handler.
> In our handler we can check, if it our Tick and than we can calc the correct
> time to start our pci_locker task 50us before next Tick-irq.
>
> It's a callback function from some irq function ipipe_raise_irq,
> __ipipe_do_IRQ, __ipipe_handle_irq
Ok, this is a bit ugly, and probably useless. The latency is not
between __ipipe_handle_irq and your handler, it is between the hardware
irq and __ipipe_handle_irq. And due to this latency, you must
calculate the "correct" time with some margin taking the latency
into account. And you must realize that you can not do anything from
such callback, such as calling xenomai functions.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:31 ` Gilles Chanteperdrix
@ 2015-07-13 20:45 ` Johann Obermayr
2015-07-13 20:54 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 20:45 UTC (permalink / raw)
To: xenomai
Am 13.07.2015 um 22:31 schrieb Gilles Chanteperdrix:
> On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
>> Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
>>> On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
>>>> Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
>>>>> Johann Obermayr wrote:
>>>>>> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>>>>>>> Johann Obermayr wrote:
>>>>>>>> without your application, there are no large latencies.
>>>>>>>> with your application see frozen.txt (with latency -f)
>>>>>>> I am confused. You mean "our application", not "your application",
>>>>>>> right?
>>>>>>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>>>>>>> project.
>>>>>>>
>>>>>>>> We see the problem only if one task (background) is accessing the SRAM
>>>>>>>> on your PCI-Card. if we stop this task, all is ok.
>>>>>>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>>>>>>> "our PCI-Card"?
>>>>>> yes, our PCI-Card. (sorry for my bad english)
>>>>>>>> So we have a higher prior task (pci-locker), that interrupt the
>>>>>>>> background task, so that the pci bus get free.
>>>>>>> I am not sure I understand your explanations. But the trace is pretty
>>>>>>> clear:
>>>>>>>
>>>>>>> At time -658 the timer is programmed to tick at -561.
>>>>>>>
>>>>>>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>>>>>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>>>>>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>>>>>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>>>>>>> (<102d0254>)
>>>>>>> But at that point the tick is delayed for 600us. And according to the
>>>>>>> trace, the last traced function called before that delay is the function
>>>>>>>
>>>>>>> ltdrv_monitoring_irq.
>>>>>>>
>>>>>>> So, I do not know what this irq is doing, but I would suggest having a
>>>>>>> close look at it.
>>>>>>>
>>>>>>>
>>>>>> hello,
>>>>>>
>>>>>> i have disable our lrtdrv_monitoring_irq.
>>>>>> Only have this callback
>>>>>> static void irq_hook_handler(unsigned int irq, unsigned int state)
>>>>>> {
>>>>>> if (fpga_interrupt == irq && state == 0x01)
>>>>>> {
>>>>>> time_fpga_irq = rt_timer_tsc();
>>>>>> }
>>>>>> }
>>>>>> same latency
>>>>> Maybe, but your trace does not contain enough points to see it. The trace
>>>>> should at least contain the "tick@" event which gets missed, so that we
>>>>> can see how much the interrupt is delayed, and what was happening at the
>>>>> time.
>>>>>
>>>>>
>>>> Hi,
>>>>
>>>> Sorry, here with more points.
>>> Ok, what is irq_hook_handler ?
>>>
>> Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
>> PC. But internal in the FPGA there
>> are different IRQ sources. One of them is our Tick.
>> So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
>> handler.
>> In our handler we can check, if it our Tick and than we can calc the correct
>> time to start our pci_locker task 50us before next Tick-irq.
>>
>> It's a callback function from some irq function ipipe_raise_irq,
>> __ipipe_do_IRQ, __ipipe_handle_irq
>> for our own tracing and it save the fpga irq time.
>> Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
>> of the function.
> I see two weird things in your traces:
> - irq_hook_handler which is taking a lot of time
> - or some APIC related functions taking a lot of time.
>
> Are you sure your system is not one of those which disable the APIC
> during idle period. Is your system UP or SMP?
>
It's a SMP (Dual core Celeron)
Kernel cmdline
nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1
console=ttyS0,115200 BOOT_IMAGE=/bzImage FirstUsbDrive=E
console=/dev/null noconsole root=/dev/sda2 rw
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:45 ` Johann Obermayr
@ 2015-07-13 20:54 ` Gilles Chanteperdrix
2015-07-13 21:30 ` Johann Obermayr
2015-07-13 21:58 ` Johann Obermayr
0 siblings, 2 replies; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 20:54 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Mon, Jul 13, 2015 at 10:45:17PM +0200, Johann Obermayr wrote:
> Am 13.07.2015 um 22:31 schrieb Gilles Chanteperdrix:
> >On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
> >>Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
> >>>On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
> >>>>Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> >>>>>Johann Obermayr wrote:
> >>>>>>Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> >>>>>>>Johann Obermayr wrote:
> >>>>>>>>without your application, there are no large latencies.
> >>>>>>>>with your application see frozen.txt (with latency -f)
> >>>>>>>I am confused. You mean "our application", not "your application",
> >>>>>>>right?
> >>>>>>>lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> >>>>>>>project.
> >>>>>>>
> >>>>>>>>We see the problem only if one task (background) is accessing the SRAM
> >>>>>>>>on your PCI-Card. if we stop this task, all is ok.
> >>>>>>>Again: the Xenomai project does not make PCI-card. So, you probably mean
> >>>>>>>"our PCI-Card"?
> >>>>>>yes, our PCI-Card. (sorry for my bad english)
> >>>>>>>>So we have a higher prior task (pci-locker), that interrupt the
> >>>>>>>>background task, so that the pci bus get free.
> >>>>>>>I am not sure I understand your explanations. But the trace is pretty
> >>>>>>>clear:
> >>>>>>>
> >>>>>>>At time -658 the timer is programmed to tick at -561.
> >>>>>>>
> >>>>>>>>:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> >>>>>>>>:| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> >>>>>>>>[sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> >>>>>>>>:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> >>>>>>>>(<102d0254>)
> >>>>>>>But at that point the tick is delayed for 600us. And according to the
> >>>>>>>trace, the last traced function called before that delay is the function
> >>>>>>>
> >>>>>>>ltdrv_monitoring_irq.
> >>>>>>>
> >>>>>>>So, I do not know what this irq is doing, but I would suggest having a
> >>>>>>>close look at it.
> >>>>>>>
> >>>>>>>
> >>>>>>hello,
> >>>>>>
> >>>>>>i have disable our lrtdrv_monitoring_irq.
> >>>>>>Only have this callback
> >>>>>>static void irq_hook_handler(unsigned int irq, unsigned int state)
> >>>>>>{
> >>>>>> if (fpga_interrupt == irq && state == 0x01)
> >>>>>> {
> >>>>>> time_fpga_irq = rt_timer_tsc();
> >>>>>> }
> >>>>>>}
> >>>>>>same latency
> >>>>>Maybe, but your trace does not contain enough points to see it. The trace
> >>>>>should at least contain the "tick@" event which gets missed, so that we
> >>>>>can see how much the interrupt is delayed, and what was happening at the
> >>>>>time.
> >>>>>
> >>>>>
> >>>>Hi,
> >>>>
> >>>>Sorry, here with more points.
> >>>Ok, what is irq_hook_handler ?
> >>>
> >>Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
> >>PC. But internal in the FPGA there
> >>are different IRQ sources. One of them is our Tick.
> >>So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
> >>handler.
> >>In our handler we can check, if it our Tick and than we can calc the correct
> >>time to start our pci_locker task 50us before next Tick-irq.
> >>
> >>It's a callback function from some irq function ipipe_raise_irq,
> >>__ipipe_do_IRQ, __ipipe_handle_irq
> >>for our own tracing and it save the fpga irq time.
> >>Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
> >>of the function.
> >I see two weird things in your traces:
> >- irq_hook_handler which is taking a lot of time
> >- or some APIC related functions taking a lot of time.
> >
> >Are you sure your system is not one of those which disable the APIC
> >during idle period. Is your system UP or SMP?
> >
> It's a SMP (Dual core Celeron)
Real dual core, or hyperthreaded ?
> Kernel cmdline
> nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1 console=ttyS0,115200
> BOOT_IMAGE=/bzImage FirstUsbDrive=E console=/dev/null noconsole
> root=/dev/sda2 rw
Do you have the same problem without these options ?
nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:54 ` Gilles Chanteperdrix
@ 2015-07-13 21:30 ` Johann Obermayr
2015-07-13 21:58 ` Johann Obermayr
1 sibling, 0 replies; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 21:30 UTC (permalink / raw)
Cc: xenomai
Am 13.07.2015 um 22:54 schrieb Gilles Chanteperdrix:
> On Mon, Jul 13, 2015 at 10:45:17PM +0200, Johann Obermayr wrote:
>> Am 13.07.2015 um 22:31 schrieb Gilles Chanteperdrix:
>>> On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
>>>> Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
>>>>> On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
>>>>>> Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
>>>>>>> Johann Obermayr wrote:
>>>>>>>> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>>>>>>>>> Johann Obermayr wrote:
>>>>>>>>>> without your application, there are no large latencies.
>>>>>>>>>> with your application see frozen.txt (with latency -f)
>>>>>>>>> I am confused. You mean "our application", not "your application",
>>>>>>>>> right?
>>>>>>>>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>>>>>>>>> project.
>>>>>>>>>
>>>>>>>>>> We see the problem only if one task (background) is accessing the SRAM
>>>>>>>>>> on your PCI-Card. if we stop this task, all is ok.
>>>>>>>>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>>>>>>>>> "our PCI-Card"?
>>>>>>>> yes, our PCI-Card. (sorry for my bad english)
>>>>>>>>>> So we have a higher prior task (pci-locker), that interrupt the
>>>>>>>>>> background task, so that the pci bus get free.
>>>>>>>>> I am not sure I understand your explanations. But the trace is pretty
>>>>>>>>> clear:
>>>>>>>>>
>>>>>>>>> At time -658 the timer is programmed to tick at -561.
>>>>>>>>>
>>>>>>>>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>>>>>>>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>>>>>>>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>>>>>>>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>>>>>>>>> (<102d0254>)
>>>>>>>>> But at that point the tick is delayed for 600us. And according to the
>>>>>>>>> trace, the last traced function called before that delay is the function
>>>>>>>>>
>>>>>>>>> ltdrv_monitoring_irq.
>>>>>>>>>
>>>>>>>>> So, I do not know what this irq is doing, but I would suggest having a
>>>>>>>>> close look at it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> hello,
>>>>>>>>
>>>>>>>> i have disable our lrtdrv_monitoring_irq.
>>>>>>>> Only have this callback
>>>>>>>> static void irq_hook_handler(unsigned int irq, unsigned int state)
>>>>>>>> {
>>>>>>>> if (fpga_interrupt == irq && state == 0x01)
>>>>>>>> {
>>>>>>>> time_fpga_irq = rt_timer_tsc();
>>>>>>>> }
>>>>>>>> }
>>>>>>>> same latency
>>>>>>> Maybe, but your trace does not contain enough points to see it. The trace
>>>>>>> should at least contain the "tick@" event which gets missed, so that we
>>>>>>> can see how much the interrupt is delayed, and what was happening at the
>>>>>>> time.
>>>>>>>
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Sorry, here with more points.
>>>>> Ok, what is irq_hook_handler ?
>>>>>
>>>> Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
>>>> PC. But internal in the FPGA there
>>>> are different IRQ sources. One of them is our Tick.
>>>> So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
>>>> handler.
>>>> In our handler we can check, if it our Tick and than we can calc the correct
>>>> time to start our pci_locker task 50us before next Tick-irq.
>>>>
>>>> It's a callback function from some irq function ipipe_raise_irq,
>>>> __ipipe_do_IRQ, __ipipe_handle_irq
>>>> for our own tracing and it save the fpga irq time.
>>>> Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
>>>> of the function.
>>> I see two weird things in your traces:
>>> - irq_hook_handler which is taking a lot of time
>>> - or some APIC related functions taking a lot of time.
>>>
>>> Are you sure your system is not one of those which disable the APIC
>>> during idle period. Is your system UP or SMP?
>>>
>> It's a SMP (Dual core Celeron)
> Real dual core, or hyperthreaded ?
Real dual core, no hyperthreading
>
>> Kernel cmdline
>> nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1 console=ttyS0,115200
>> BOOT_IMAGE=/bzImage FirstUsbDrive=E console=/dev/null noconsole
>> root=/dev/sda2 rw
> Do you have the same problem without these options ?
> nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1
>
same result. but higher cpu load.
core 0 is reserver for our highprio task (see screenshot. realtime,
cyclic) all other on core 1 (linux & visu).
we have on old mainboard with Intel 6 / C200 series. this board run
without ACPI.
Our new board (Intel 8 / C220 series) does not run without ACPI. so we
have enable ACPI in the kernel .config.
We also enable MSI, because of IRQ sharing.
-------------- next part --------------
I-pipe frozen back-tracing service on 3.10.0/ipipe release #1
------------------------------------------------------------
CPU: 0, Freeze: 749274436391 cycles, Trace Points: 2000 (+10)
Calibrated minimum trace-point overhead: 0.078 us
+----- Hard IRQs ('|': locked)
|+-- Xenomai
||+- Linux ('*': domain stalled, '+': current, '#': current+stalled)
||| +---------- Delay flag ('+': > 1 us, '!': > 10 us)
||| | +- NMI noise ('N')
||| | |
Type User Val. Time Delay Function (Parent)
:| #end 0x80000001 -1670 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1670 0.115 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -1670 0.144 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -1670 0.118 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -1670 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1670 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1670 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1670 0.117 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -1669 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1669 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1669 0.132 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1669 0.135 wait_domain_nrt_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0xff)
: #func -1669 0.124 __wake_up+0x8 (wait_domain_nrt_handler+0x33 [sigmatek_lrt])
: #func -1669 0.142 _raw_spin_lock_irqsave+0x5 (__wake_up+0x1a)
: #func -1669 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1668 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1668 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1668 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1668 0.118 __wake_up_common+0x9 (__wake_up+0x2e)
: #func -1668 0.117 ipipe_root_only+0x5 (__wake_up_common+0x1b)
:| #begin 0x80000001 -1668 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1668 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1667 0.115 __ipipe_spin_unlock_debug+0x3 (__wake_up+0x35)
: #func -1667 0.124 _raw_spin_unlock_irqrestore+0x4 (__wake_up+0x3e)
: #func -1667 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1667 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1667 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1667 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1667 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1667 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1666 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1666 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1666 0.127 irq_exit+0x3 (__ipipe_do_sync_stage+0x104)
: #func -1666 0.115 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -1666 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1666 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1666 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1665 0.152 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -1665 0.127 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -1665 0.175 idle_cpu+0x3 (irq_exit+0x50)
: #func -1665 0.121 tick_nohz_irq_exit+0x4 (irq_exit+0x79)
:| #begin 0x80000001 -1665 0.152 debug_smp_processor_id+0x36 (tick_nohz_irq_exit+0x13)
:| #end 0x80000001 -1665 0.129 debug_smp_processor_id+0x82 (tick_nohz_irq_exit+0x13)
: #func -1665 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_irq_exit+0x27)
:| #begin 0x80000001 -1665 0.151 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -1664 0.130 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -1664 0.154 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -1664 0.152 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -1664 0.145 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -1664 0.154 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1664 0.176 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -1663 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -1663 0.193 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -1663 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -1663 0.152 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -1663 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -1663 0.133 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -1663 0.136 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -1662 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -1662 0.129 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -1662 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -1662 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1662 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1662 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1662 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1661 0.127 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -1661 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1661 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1661 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1661 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1661 0.121 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -1661 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -1661 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -1660 0.141 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -1660 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1660 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1660 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1660 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1660 0.115 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -1660 0.124 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -1660 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1659 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1659 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1659 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1659 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1659 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1659 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1659 0.182 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1658 0.115 hrtimer_start+0x3 (__tick_nohz_idle_enter+0x2e1)
: #func -1658 0.114 __hrtimer_start_range_ns+0x9 (hrtimer_start+0x14)
: #func -1658 0.120 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -1658 0.138 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1658 0.114 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1658 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1658 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1657 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1657 0.141 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -1657 0.129 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -1657 0.135 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
:| #begin 0x80000001 -1657 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1657 0.138 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1657 0.117 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1657 0.146 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
: #func -1656 0.112 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1656 0.126 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1656 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1656 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1656 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1656 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1656 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1655 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1655 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1655 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1655 0.148 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -1655 0.151 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -1655 0.132 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -1655 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -1654 0.152 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1654 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1654 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1654 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1654 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1654 0.167 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1654 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0x117)
:| #begin 0x80000000 -1654 0.193 hard_local_irq_disable+0x17 (__ipipe_do_sync_stage+0x11c)
:| +func -1653 0.179 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x00000051 -1653 0.231 common_interrupt+0x51 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1653 0.152 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1653 0.184 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1653 0.145 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1652 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1652 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1652 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1652 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1652 0.141 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1652 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1652 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1652 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1651 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1651 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1651 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1651 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1651 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1651 0.133 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1651 0.144 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1650 0.144 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1650 0.117 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1650 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1650 0.123 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1650 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1650 0.132 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1650 0.146 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1649 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1649 0.130 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1649 0.121 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1649 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1649 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1649 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1649 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1649 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1648 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1648 0.118 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1648 0.120 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1648 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1648 0.127 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1648 0.121 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1648 0.144 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1647+ 9.725 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1638 0.132 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1638 0.158 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1637 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1637 0.121 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1637 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1637 0.149 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1637 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1637 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1637 0.173 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1636 0.149 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1636 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1636 0.139 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1636 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1636 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1538 -1636 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1636 0.152 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1635 0.259 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1635 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1635 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1635 0.152 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1635 0.313 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1634 0.322 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1634 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1634 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1634 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1634 0.237 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1633 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1633 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1633 0.224 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1633 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1633 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1633 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1632 0.351 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1632 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1632 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1632 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1632 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1632 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1631 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1631 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1631 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1631 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1631 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1631 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1631 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1631 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1630 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1630 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1630 0.299 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1630 0.221 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1630 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1629 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1629 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1629 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1629 0.280 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1629 0.201 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1629 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1628 0.215 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1628 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1628 0.158 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1628 0.146 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1628 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1628 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1627 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1627 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1627 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1627 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1627 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1627 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1627 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1626 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1626 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1626 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1626 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1626 0.133 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1626 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1626 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1625 0.115 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1625 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1625 0.121 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1625 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1625 0.133 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1625 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1625 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1625 0.130 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1624 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1624 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1624 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1624 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1624 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1624 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1624 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1623 0.115 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1623 0.123 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1623 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1623 0.129 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1623 0.121 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1623 0.142 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1623! 84.994 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1538 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1538 0.149 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1537 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1537 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1537 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1537 0.139 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1537 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1537 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1537 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1536 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1536 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1536 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1536 0.207 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1536 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1438 -1536 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1536 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1536 0.255 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1535 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1535 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1535 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1535 0.310 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1535 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1534 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1534 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1534 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1534 0.216 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1534 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1533 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1533 0.209 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1533 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1533 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1533 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1533 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1532 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1532 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1532 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1532 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1532 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1532 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1531 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1531 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1531 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1531 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1531 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1531 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1531 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1531 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1530 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1530 0.288 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1530 0.218 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1530 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1530 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1529 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1529 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1529 0.277 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1529 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1529 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1529 0.215 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1528 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1528 0.158 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1528 0.148 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1528 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1528 0.132 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1528 0.133 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1527 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1527 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1527 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1527 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1527 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1527 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1527 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1527 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1526 0.193 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1526 0.121 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1526 0.139 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1526 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1526 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1526 0.114 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1526 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1525 0.123 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1525 0.145 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1525 0.132 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1525 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1525 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1525 0.129 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1525 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1525 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1524 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1524 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1524 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1524 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1524 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1524 0.115 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1524 0.120 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1523 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1523 0.129 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1523 0.118 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1523 0.144 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1523! 85.187 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1438 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1438 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1437 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1437 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1437 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1437 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1437 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1437 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1437 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1436 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1436 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1436 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1436 0.209 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1436 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1338 -1436 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1436 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1436 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1435 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1435 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1435 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1435 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1435 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1434 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1434 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1434 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1434 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1434 0.160 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1433 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1433 0.209 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1433 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1433 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1433 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1433 0.302 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1432 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1432 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1432 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1432 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1432 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1432 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1431 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1431 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1431 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1431 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1431 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1431 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1431 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1431 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1430 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1430 0.288 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1430 0.218 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1430 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1430 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1429 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1429 0.130 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1429 0.277 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1429 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1429 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1429 0.215 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1428 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1428 0.160 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1428 0.148 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1428 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1428 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1428 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1427 0.141 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1427 0.132 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1427 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1427 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1427 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1427 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1427 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1427 0.126 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1426 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1426 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1426 0.135 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1426 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1426 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1426 0.114 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1426 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1425 0.123 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1425 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1425 0.132 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1425 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1425 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1425 0.129 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1425 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1425 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1424 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1424 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1424 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1424 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1424 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1424 0.115 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1424 0.120 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1423 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1423 0.129 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1423 0.118 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1423 0.142 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1423! 85.218 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1338 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1338 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1337 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1337 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1337 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1337 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1337 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1337 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1337 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1337 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1336 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1336 0.135 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1336 0.198 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1336 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1238 -1336 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1336 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1336 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1335 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1335 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1335 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1335 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1335 0.301 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1334 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1334 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1334 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1334 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1334 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1333 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1333 0.207 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1333 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1333 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1333 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1333 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1332 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1332 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1332 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1332 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1332 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1332 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1332 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1331 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1331 0.118 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1331 0.117 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1331 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1331 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1331 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1331 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1330 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1330 0.291 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1330 0.219 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1330 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1330 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1329 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1329 0.132 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1329 0.282 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1329 0.190 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1329 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1329 0.215 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1328 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1328 0.158 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1328 0.148 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1328 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1328 0.132 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1328 0.133 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1328 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1327 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1327 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1327 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1327 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1327 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1327 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1327 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1326 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1326 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1326 0.135 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1326 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1326 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1326 0.114 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1326 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1325 0.123 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1325 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1325 0.132 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1325 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1325 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1325 0.129 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1325 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1325 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1324 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1324 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1324 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1324 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1324 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1324 0.114 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1324 0.121 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1323 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1323 0.127 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1323 0.120 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1323 0.142 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1323! 85.220 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1238 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1238 0.149 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1237 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1237 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1237 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1237 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1237 0.169 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1237 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1237 0.172 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1237 0.135 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1236 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1236 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1236 0.187 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1236 0.139 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1138 -1236 0.114 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1236 0.132 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1236 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1235 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1235 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1235 0.129 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1235 0.307 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1235 0.302 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1234 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1234 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1234 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1234 0.218 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1234 0.163 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1233 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1233 0.207 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1233 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1233 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1233 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1233 0.304 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1232 0.117 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1232 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1232 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1232 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1232 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1232 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1232 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1231 0.126 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1231 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1231 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1231 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1231 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1231 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1231 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1230 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1230 0.135 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1230 0.142 xntimer_next_local_shot+0x9 (__xnpod_schedule+0x1d6)
:| # event tick@-1212 -1230 0.114 xntimer_next_local_shot+0xca (__xnpod_schedule+0x1d6)
:| # func -1230 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1230 0.366 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1229 0.218 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1229 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1229 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1229 0.115 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1229 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1229 0.282 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1228 0.187 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1228 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1228 0.215 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1228 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1228 0.158 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1227 0.148 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1227 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1227 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1227 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1227 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1227 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1227 0.114 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1227 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1226 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1226 0.138 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1226 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1226 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1226 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1226 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1226 0.135 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1225 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1225 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1225 0.115 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1225 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1225 0.121 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1225 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1225 0.130 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1224 0.146 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1224 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1224 0.129 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1224 0.121 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1224 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1224 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1224 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1224 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1223 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1223 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1223 0.114 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1223 0.121 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1223 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1223 0.126 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1223 0.120 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1223 0.144 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1222! 11.448 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1211 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1211 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1211 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1211 0.115 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1210 0.130 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1210 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1210 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1210 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1210 0.169 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1210 0.151 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1210 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1138 -1209 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1209 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1209 0.268 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1209 0.135 xnintr_host_tick+0x3 (xnintr_clock_handler+0x152)
:| # func -1209 0.230 __ipipe_set_irq_pending+0x6 (xnarch_relay_tick+0x44)
:| +func -1209 0.187 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1208 0.152 __ipipe_do_sync_stage+0x9 (__ipipe_do_sync_pipeline+0x5e)
:| #end 0x80000000 -1208 0.126 hard_local_irq_enable+0x16 (__ipipe_do_sync_stage+0xa1)
: #func -1208 0.142 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_sync_stage+0x146)
: #func -1208 0.115 __ipipe_do_IRQ+0x6 (__ipipe_do_sync_stage+0x15c)
: #func -1208 0.115 __ipipe_get_ioapic_irq_vector+0x3 (__ipipe_do_IRQ+0x20)
: #func -1208 0.121 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_do_IRQ+0x38)
: #func -1208 0.112 smp_apic_timer_interrupt+0x5 (__ipipe_do_IRQ+0x58)
: #func -1207 0.120 irq_enter+0x4 (smp_apic_timer_interrupt+0x21)
:| #begin 0x80000001 -1207 0.154 debug_smp_processor_id+0x36 (irq_enter+0xe)
:| #end 0x80000001 -1207 0.130 debug_smp_processor_id+0x82 (irq_enter+0xe)
: #func -1207 0.146 rcu_irq_enter+0x7 (irq_enter+0x15)
:| #begin 0x80000001 -1207 0.142 debug_smp_processor_id+0x36 (rcu_irq_enter+0x1e)
:| #end 0x80000001 -1207 0.130 debug_smp_processor_id+0x82 (rcu_irq_enter+0x1e)
: #func -1207 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_irq_enter+0x70)
:| #begin 0x80000001 -1206 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1206 0.139 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1206 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1206 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1206 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1206 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1206 0.114 local_bh_disable+0x3 (irq_enter+0x39)
: #func -1206 0.117 add_preempt_count+0x4 (local_bh_disable+0x12)
: #func -1205 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1205 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1205 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1205 0.114 tick_check_idle+0x8 (irq_enter+0x40)
: #func -1205 0.118 tick_check_oneshot_broadcast+0x3 (tick_check_idle+0x19)
: #func -1205 0.149 ktime_get+0x9 (tick_check_idle+0x31)
: #func -1205 0.115 tick_nohz_stop_idle+0x4 (tick_check_idle+0x48)
: #func -1205 0.115 update_ts_time_stats+0x9 (tick_nohz_stop_idle+0x20)
: #func -1204 0.129 nr_iowait_cpu+0x3 (update_ts_time_stats+0x31)
:| #begin 0x80000001 -1204 0.142 debug_smp_processor_id+0x36 (tick_check_idle+0x53)
:| #end 0x80000001 -1204 0.160 debug_smp_processor_id+0x82 (tick_check_idle+0x53)
: #func -1204 0.117 tick_do_update_jiffies64+0x9 (tick_check_idle+0x84)
: #func -1204 0.114 ipipe_restore_root+0x4 (tick_check_idle+0x97)
: #func -1204 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1204 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1203 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1203 0.117 _local_bh_enable+0x3 (irq_enter+0x45)
: #func -1203 0.130 __local_bh_enable+0x4 (_local_bh_enable+0x12)
: #func -1203 0.114 sub_preempt_count+0x4 (__local_bh_enable+0x6a)
: #func -1203 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1203 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1203 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1203 0.115 add_preempt_count+0x4 (irq_enter+0x4f)
: #func -1202 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1202 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1202 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1202 0.142 debug_smp_processor_id+0x36 (smp_apic_timer_interrupt+0x26)
:| #end 0x80000001 -1202 0.154 debug_smp_processor_id+0x82 (smp_apic_timer_interrupt+0x26)
: #func -1202 0.124 hrtimer_interrupt+0x9 (smp_apic_timer_interrupt+0x58)
:| #begin 0x80000001 -1202 0.152 debug_smp_processor_id+0x36 (hrtimer_interrupt+0x1a)
:| #end 0x80000001 -1201 0.133 debug_smp_processor_id+0x82 (hrtimer_interrupt+0x1a)
: #func -1201 0.111 _raw_spin_lock+0x4 (hrtimer_interrupt+0x41)
: #func -1201 0.124 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1201 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1201 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1201 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1201 0.163 ktime_get_update_offsets+0x9 (hrtimer_interrupt+0x61)
: #func -1200 0.132 __run_hrtimer+0x9 (hrtimer_interrupt+0x101)
: #func -1200 0.164 __remove_hrtimer+0x9 (__run_hrtimer+0x4b)
: #func -1200 0.127 _raw_spin_unlock+0x3 (__run_hrtimer+0x59)
: #func -1200 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1200 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1200 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1200 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1199 0.117 hrtimer_wakeup+0x3 (__run_hrtimer+0x8e)
: #func -1199 0.118 wake_up_process+0x4 (hrtimer_wakeup+0x1d)
: #func -1199 0.112 try_to_wake_up+0x9 (wake_up_process+0x2e)
: #func -1199 0.142 _raw_spin_lock_irqsave+0x5 (try_to_wake_up+0x25)
: #func -1199 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1199 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1199 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1199 0.160 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1198 0.151 task_waking_fair+0x6 (try_to_wake_up+0xc7)
: #func -1198 0.120 select_task_rq_fair+0x9 (select_task_rq+0x15)
:| #begin 0x80000001 -1198 0.151 debug_smp_processor_id+0x36 (select_task_rq_fair+0x1c)
:| #end 0x80000001 -1198 0.141 debug_smp_processor_id+0x82 (select_task_rq_fair+0x1c)
: #func -1198 0.138 __rcu_read_lock+0x3 (select_task_rq_fair+0x72)
: #func -1198 0.120 idle_cpu+0x3 (select_task_rq_fair+0x250)
: #func -1198 0.139 __rcu_read_unlock+0x4 (select_task_rq_fair+0x4ac)
:| #begin 0x80000001 -1197 0.142 debug_smp_processor_id+0x36 (try_to_wake_up+0x106)
:| #end 0x80000001 -1197 0.126 debug_smp_processor_id+0x82 (try_to_wake_up+0x106)
: #func -1197 0.120 cpus_share_cache+0x4 (try_to_wake_up+0x10d)
: #func -1197 0.112 _raw_spin_lock+0x4 (try_to_wake_up+0x15a)
: #func -1197 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1197 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1197 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1197 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1196 0.117 ttwu_do_activate.constprop.91+0x5 (try_to_wake_up+0x163)
: #func -1196 0.121 activate_task+0x4 (ttwu_do_activate.constprop.91+0x2b)
: #func -1196 0.115 enqueue_task+0x8 (activate_task+0x21)
: #func -1196 0.132 update_rq_clock+0x6 (enqueue_task+0x1a)
:| #begin 0x80000001 -1196 0.154 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1196 0.227 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1196 0.124 enqueue_task_fair+0x9 (enqueue_task+0x28)
: #func -1195 0.120 update_curr+0x9 (enqueue_task_fair+0x59)
: #func -1195 0.121 place_entity+0x9 (enqueue_task_fair+0xa4)
: #func -1195 0.118 __enqueue_entity+0x7 (enqueue_task_fair+0xb2)
: #func -1195 0.118 hrtick_update+0x5 (enqueue_task_fair+0xc7)
: #func -1195 0.117 ttwu_do_wakeup+0x6 (ttwu_do_activate.constprop.91+0x50)
: #func -1195 0.121 check_preempt_curr+0x7 (ttwu_do_wakeup+0x14)
: #func -1195 0.141 resched_task+0x5 (check_preempt_curr+0x3c)
:| #begin 0x80000001 -1195 0.151 debug_smp_processor_id+0x36 (resched_task+0x48)
:| #end 0x80000001 -1194 0.136 debug_smp_processor_id+0x82 (resched_task+0x48)
: #func -1194 0.130 _raw_spin_unlock+0x3 (try_to_wake_up+0x16f)
: #func -1194 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1194 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1194 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1194 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1194 0.115 __ipipe_spin_unlock_debug+0x3 (try_to_wake_up+0x177)
: #func -1193 0.124 _raw_spin_unlock_irqrestore+0x4 (try_to_wake_up+0x182)
: #func -1193 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1193 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1193 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1193 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1193 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1193 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1193 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1192 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1192 0.117 _raw_spin_lock+0x4 (__run_hrtimer+0xc5)
: #func -1192 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1192 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1192 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1192 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1192 0.124 _raw_spin_unlock+0x3 (hrtimer_interrupt+0x125)
: #func -1191 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1191 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1191 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1191 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1191 0.121 tick_program_event+0x4 (hrtimer_interrupt+0x13d)
: #func -1191 0.118 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -1191 0.151 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -1191 0.121 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -1190 0.163 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -1190 0.139 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -1190 0.195 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -1190 0.132 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1190 0.145 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -1190 0.130 irq_exit+0x3 (smp_apic_timer_interrupt+0x5d)
: #func -1190 0.115 sub_preempt_count+0x4 (irq_exit+0x17)
: #func -1189 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1189 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1189 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1189 0.142 debug_smp_processor_id+0x36 (irq_exit+0x4b)
:| #end 0x80000001 -1189 0.127 debug_smp_processor_id+0x82 (irq_exit+0x4b)
: #func -1189 0.144 idle_cpu+0x3 (irq_exit+0x50)
: #func -1189 0.146 rcu_irq_exit+0x5 (irq_exit+0x7e)
:| #begin 0x80000001 -1188 0.142 debug_smp_processor_id+0x36 (rcu_irq_exit+0x1b)
:| #end 0x80000001 -1188 0.130 debug_smp_processor_id+0x82 (rcu_irq_exit+0x1b)
: #func -1188 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_irq_exit+0x68)
:| #begin 0x80000001 -1188 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1188 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1188 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1188 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1187 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1187 0.314 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +func -1187 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1187 0.224 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1187 0.144 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1186 0.172 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1186 0.145 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1186 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1186 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1186 0.135 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1186 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1186 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1185 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1185 0.123 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1185 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1185 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1185 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1185 0.130 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1185 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1185 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1184 0.139 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
: +func -1184 0.121 tick_nohz_idle_exit+0x7 (cpu_startup_entry+0x148)
:| +begin 0x80000001 -1184 0.142 debug_smp_processor_id+0x36 (tick_nohz_idle_exit+0x16)
:| +end 0x80000001 -1184 0.144 debug_smp_processor_id+0x82 (tick_nohz_idle_exit+0x16)
: #func -1184 0.148 ktime_get+0x9 (tick_nohz_idle_exit+0x62)
: #func -1184 0.115 tick_do_update_jiffies64+0x9 (tick_nohz_idle_exit+0x8b)
: #func -1184 0.123 update_cpu_load_nohz+0x5 (tick_nohz_idle_exit+0x90)
:| #begin 0x80000001 -1183 0.142 debug_smp_processor_id+0x36 (update_cpu_load_nohz+0x14)
:| #end 0x80000001 -1183 0.141 debug_smp_processor_id+0x82 (update_cpu_load_nohz+0x14)
: #func -1183 0.124 _raw_spin_lock+0x4 (update_cpu_load_nohz+0x2d)
: #func -1183 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1183 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1183 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1183 0.146 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1182 0.136 __update_cpu_load+0x9 (update_cpu_load_nohz+0x40)
: #func -1182 0.121 sched_avg_update+0x9 (__update_cpu_load+0xa9)
: #func -1182 0.129 _raw_spin_unlock+0x3 (update_cpu_load_nohz+0x47)
: #func -1182 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1182 0.120 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1182 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1182 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1182 0.121 calc_load_exit_idle+0x4 (tick_nohz_idle_exit+0x95)
:| #begin 0x80000001 -1181 0.141 debug_smp_processor_id+0x36 (calc_load_exit_idle+0x13)
:| #end 0x80000001 -1181 0.142 debug_smp_processor_id+0x82 (calc_load_exit_idle+0x13)
: #func -1181 0.111 hrtimer_cancel+0x4 (tick_nohz_idle_exit+0xa9)
: #func -1181 0.114 hrtimer_try_to_cancel+0x7 (hrtimer_cancel+0x12)
: #func -1181 0.115 lock_hrtimer_base.isra.17+0x6 (hrtimer_try_to_cancel+0x1e)
: #func -1181 0.142 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1181 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1181 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1180 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1180 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1180 0.151 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -1180 0.132 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -1180 0.167 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -1180 0.114 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1180 0.120 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1179 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1179 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1179 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1179 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1179 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1179 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1179 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1178 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1178 0.149 hrtimer_forward+0xc (tick_nohz_idle_exit+0xd2)
: #func -1178 0.123 ktime_add_safe+0x6 (hrtimer_forward+0x103)
: #func -1178 0.120 ktime_add_safe+0x6 (hrtimer_forward+0x11d)
: #func -1178 0.115 hrtimer_start_range_ns+0x3 (hrtimer_start_expires.constprop.9+0x1d)
: #func -1178 0.114 __hrtimer_start_range_ns+0x9 (hrtimer_start_range_ns+0x15)
: #func -1178 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -1178 0.161 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1177 0.118 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1177 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1177 0.181 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1177 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1177 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1177 0.146 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1177 0.117 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1176 0.176 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
:| #begin 0x80000001 -1176 0.144 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x16d)
:| #end 0x80000001 -1176 0.135 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x16d)
:| #begin 0x80000001 -1176 0.152 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x192)
:| #end 0x80000001 -1176 0.133 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x192)
: #func -1176 0.114 tick_program_event+0x4 (__hrtimer_start_range_ns+0x20a)
: #func -1176 0.120 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -1175 0.149 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -1175 0.118 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -1175 0.154 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -1175 0.126 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -1175 0.181 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -1175 0.132 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1175 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: #func -1175 0.117 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1174 0.126 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1174 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1174 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1174 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1174 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1174 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1174 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1173 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1173 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1173 0.115 account_idle_ticks+0x3 (tick_nohz_idle_exit+0x122)
: #func -1173 0.127 account_idle_time+0x6 (account_idle_ticks+0xd)
:| #begin 0x80000001 -1173 0.142 debug_smp_processor_id+0x36 (account_idle_time+0x1c)
:| #end 0x80000001 -1173 0.135 debug_smp_processor_id+0x82 (account_idle_time+0x1c)
:| #begin 0x80000001 -1173 0.152 debug_smp_processor_id+0x36 (account_idle_time+0x28)
:| #end 0x80000001 -1172 0.130 debug_smp_processor_id+0x82 (account_idle_time+0x28)
: #func -1172 0.118 ipipe_unstall_root+0x3 (tick_nohz_idle_exit+0x127)
:| #begin 0x80000000 -1172 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1172 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1172 0.123 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1172 0.118 schedule_preempt_disabled+0x3 (cpu_startup_entry+0x14d)
: +func -1172 0.114 sub_preempt_count+0x4 (schedule_preempt_disabled+0x12)
: +func -1172 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1171 0.172 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1171 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1171 0.114 schedule+0x3 (schedule_preempt_disabled+0x17)
: +func -1171 0.115 __schedule+0x9 (schedule+0x18)
: +func -1171 0.115 add_preempt_count+0x4 (__schedule+0x23)
: +func -1171 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1171 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1171 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1170 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1170 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1170 0.115 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1170 0.144 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1170 0.124 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -1170 0.111 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1170 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1169 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1169 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1169 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1169 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1169 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1169 0.118 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1169 0.118 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1169 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1168 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1168 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1168 0.114 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1168 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1168 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1168 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1168 0.138 pre_schedule_idle+0x3 (__schedule+0x242)
: #func -1167 0.117 put_prev_task_idle+0x3 (__schedule+0x275)
: #func -1167 0.117 pick_next_task_fair+0x7 (__schedule+0x28c)
: #func -1167 0.114 clear_buddies+0x5 (pick_next_task_fair+0x8c)
: #func -1167 0.135 set_next_entity+0x6 (pick_next_task_fair+0x98)
: #func -1167 0.120 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1167 0.145 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1167 0.118 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1167 0.124 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1166 0.173 rt_task_set_mode+0x7 (taskSwitchLinux+0x8d [sigmatek_lrt])
:| #begin 0x80000000 -1166 0.194 __schedule+0x3a3 (schedule+0x18)
:| #func -1166 0.227 __switch_to+0x9 (__schedule+0x50a)
:| #func -1166 0.149 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1166 0.124 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1166 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1165 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1165 0.121 finish_task_switch+0x7 (__schedule+0x533)
: #func -1165 0.124 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1165 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1165 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1165 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1165 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1164 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1164 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1164 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1164 0.182 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1164 0.142 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1164 0.127 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1164 0.115 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1163 0.124 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1163 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1163 0.179 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1163 0.126 hrtimer_cancel+0x4 (schedule_hrtimeout_range_clock+0x9e)
: +func -1163 0.117 hrtimer_try_to_cancel+0x7 (hrtimer_cancel+0x12)
: +func -1163 0.121 lock_hrtimer_base.isra.17+0x6 (hrtimer_try_to_cancel+0x1e)
: +func -1163 0.148 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1163 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1162 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1162 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1162 0.146 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1162 0.124 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1162 0.123 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1162 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1162 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1161 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1161 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1161 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1161 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1161 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1161 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1161 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1161 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1160 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1160 0.325 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1160 0.138 fget_light+0x6 (do_select+0x2aa)
: +func -1160 0.155 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1160 0.139 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1159 0.161 sock_poll+0x5 (do_select+0x30f)
: +func -1159 0.144 tcp_poll+0x6 (sock_poll+0x17)
: +func -1159 0.138 fput+0x5 (do_select+0x327)
: +func -1159 0.115 fget_light+0x6 (do_select+0x2aa)
: +func -1159 0.146 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1159 0.123 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1159 0.161 sock_poll+0x5 (do_select+0x30f)
: +func -1158 0.114 udp_poll+0x6 (sock_poll+0x17)
: +func -1158 0.145 datagram_poll+0x6 (udp_poll+0x14)
: +func -1158 0.135 fput+0x5 (do_select+0x327)
: +func -1158 0.120 _cond_resched+0x3 (do_select+0x3f1)
: +func -1158 0.118 poll_freewait+0x9 (do_select+0x4ab)
: +func -1158 0.111 remove_wait_queue+0x6 (poll_freewait+0x30)
: +func -1158 0.144 _raw_spin_lock_irqsave+0x5 (remove_wait_queue+0x14)
: #func -1158 0.123 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1157 0.121 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1157 0.157 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1157 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1157 0.121 __ipipe_spin_unlock_debug+0x3 (remove_wait_queue+0x36)
: #func -1157 0.121 _raw_spin_unlock_irqrestore+0x4 (remove_wait_queue+0x3f)
: #func -1157 0.120 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1157 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1157 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1156 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1156 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1156 0.132 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1156 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1156 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1156 0.121 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1156 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1155 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1155 0.136 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1155 0.133 fput+0x5 (poll_freewait+0x38)
: +func -1155 0.114 remove_wait_queue+0x6 (poll_freewait+0x30)
: +func -1155 0.141 _raw_spin_lock_irqsave+0x5 (remove_wait_queue+0x14)
: #func -1155 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1155 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1155 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1154 0.148 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1154 0.121 __ipipe_spin_unlock_debug+0x3 (remove_wait_queue+0x36)
: #func -1154 0.120 _raw_spin_unlock_irqrestore+0x4 (remove_wait_queue+0x3f)
: #func -1154 0.120 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1154 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1154 0.167 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1154 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1153 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1153 0.132 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1153 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1153 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1153 0.120 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1153 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1153 0.166 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1152 0.135 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1152 0.151 fput+0x5 (poll_freewait+0x38)
: +func -1152 0.121 ipipe_root_only+0x5 (set_fd_set+0x17)
:| +begin 0x80000001 -1152 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1152 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1152 0.169 __copy_to_user_ll+0x6 (set_fd_set+0x2d)
: +func -1152 0.126 poll_select_copy_remaining+0x9 (SyS_select+0x90)
: +func -1151 0.154 ktime_get_ts+0x9 (poll_select_copy_remaining+0x46)
: +func -1151 0.135 set_normalized_timespec+0x4 (poll_select_copy_remaining+0x60)
: +func -1151 0.120 copy_to_user+0x9 (poll_select_copy_remaining+0xb0)
: +func -1151 0.118 ipipe_root_only+0x5 (copy_to_user+0x35)
:| +begin 0x80000001 -1151 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1151 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1151 0.711 __copy_to_user_ll+0x6 (copy_to_user+0x44)
: +func -1150 0.130 __ipipe_syscall_root+0x5 (system_call+0x30)
: +func -1150 0.130 SyS_select+0x9 (syscall_call+0x7)
: +func -1150 0.120 _copy_from_user+0x7 (SyS_select+0x39)
: +func -1150 0.118 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -1149 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1149 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1149 0.157 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -1149 0.120 poll_select_set_timeout+0x6 (SyS_select+0x60)
: +func -1149 0.157 ktime_get_ts+0x9 (poll_select_set_timeout+0x3a)
: +func -1149 0.115 timespec_add_safe+0x9 (poll_select_set_timeout+0x46)
: +func -1149 0.129 set_normalized_timespec+0x4 (timespec_add_safe+0x2f)
: +func -1148 0.121 core_sys_select+0xc (SyS_select+0x80)
: +func -1148 0.114 __rcu_read_lock+0x3 (core_sys_select+0x31)
: +func -1148 0.129 __rcu_read_unlock+0x4 (core_sys_select+0x47)
: +func -1148 0.114 _copy_from_user+0x7 (get_fd_set+0x1b)
: +func -1148 0.129 ipipe_root_only+0x5 (_copy_from_user+0x30)
:| +begin 0x80000001 -1148 0.173 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1148 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1148 0.218 __copy_from_user_ll+0x6 (_copy_from_user+0x3c)
: +func -1147 0.123 do_select+0xc (core_sys_select+0x12b)
: +func -1147 0.188 __rcu_read_lock+0x3 (do_select+0x23)
: +func -1147 0.123 __rcu_read_unlock+0x4 (do_select+0xe5)
: +func -1147 0.121 poll_initwait+0x3 (do_select+0xfd)
: +func -1147 0.118 select_estimate_accuracy+0x9 (do_select+0x160)
: +func -1147 0.151 ktime_get_ts+0x9 (select_estimate_accuracy+0x9f)
: +func -1146 0.117 set_normalized_timespec+0x4 (select_estimate_accuracy+0xb9)
: +func -1146 0.194 task_nice+0x3 (select_estimate_accuracy+0x2d)
: +func -1146 0.136 fget_light+0x6 (do_select+0x2aa)
: +func -1146 0.133 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1146 0.123 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1146 0.129 sock_poll+0x5 (do_select+0x30f)
: +func -1146 0.117 tcp_poll+0x6 (sock_poll+0x17)
: +func -1146 0.127 __pollwait+0x7 (tcp_poll+0x3a)
: +func -1145 0.120 add_wait_queue+0x6 (__pollwait+0xa2)
: +func -1145 0.145 _raw_spin_lock_irqsave+0x5 (add_wait_queue+0x17)
: #func -1145 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1145 0.123 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1145 0.157 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1145 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1145 0.124 __ipipe_spin_unlock_debug+0x3 (add_wait_queue+0x35)
: #func -1144 0.124 _raw_spin_unlock_irqrestore+0x4 (add_wait_queue+0x3e)
: #func -1144 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1144 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1144 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1144 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1144 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1144 0.130 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1144 0.190 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1143 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1143 0.120 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1143 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1143 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1143 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1143 0.130 fput+0x5 (do_select+0x327)
: +func -1143 0.118 fget_light+0x6 (do_select+0x2aa)
: +func -1142 0.135 __rcu_read_lock+0x3 (fget_light+0x4a)
: +func -1142 0.123 __rcu_read_unlock+0x4 (fget_light+0x7b)
: +func -1142 0.135 sock_poll+0x5 (do_select+0x30f)
: +func -1142 0.118 udp_poll+0x6 (sock_poll+0x17)
: +func -1142 0.115 datagram_poll+0x6 (udp_poll+0x14)
: +func -1142 0.126 __pollwait+0x7 (datagram_poll+0x3a)
: +func -1142 0.118 add_wait_queue+0x6 (__pollwait+0xa2)
: +func -1142 0.166 _raw_spin_lock_irqsave+0x5 (add_wait_queue+0x17)
: #func -1141 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1141 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1141 0.158 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1141 0.145 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1141 0.123 __ipipe_spin_unlock_debug+0x3 (add_wait_queue+0x35)
: #func -1141 0.124 _raw_spin_unlock_irqrestore+0x4 (add_wait_queue+0x3e)
: #func -1141 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1140 0.121 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1140 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1140 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1140 0.123 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1140 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1140 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1140 0.120 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1139 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1139 0.123 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1139 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1139 0.160 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1139 0.129 fput+0x5 (do_select+0x327)
: +func -1139 0.126 _cond_resched+0x3 (do_select+0x3f1)
: +func -1139 0.123 poll_schedule_timeout+0x6 (do_select+0x48b)
: +func -1139 0.118 schedule_hrtimeout_range+0x3 (poll_schedule_timeout+0x3b)
: +func -1138 0.121 schedule_hrtimeout_range_clock+0x9 (schedule_hrtimeout_range+0xf)
: +func -1138 0.141 hrtimer_init+0x9 (schedule_hrtimeout_range_clock+0x55)
: +func -1138 0.270 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
:| +begin 0x000000ef -1138 0.146 apic_timer_interrupt+0x3a (__ipipe_trace+0x40b)
:| +func -1138 0.157 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1138 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1137 0.120 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1137 0.129 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1137 0.146 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1137 0.170 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1137 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1137 0.173 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1137 0.136 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1136 0.123 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1136 0.135 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1136 0.194 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1136 0.141 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-1038 -1136 0.114 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1136 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1136 0.279 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1135 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -1135 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1135 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1135 0.308 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1135 0.325 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1134 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1134 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1134 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1134 0.242 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1134 0.164 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1133 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1133 0.221 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1133 0.166 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1133 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1133 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1133 0.372 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1132 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1132 0.121 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1132 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1132 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1132 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1132 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1131 0.136 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1131 0.132 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1131 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1131 0.120 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1131 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1131 0.227 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1131 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1130 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1130 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1130 0.292 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1130 0.271 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 582] TCP-Lis -1 -1130 0.207 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1129 0.118 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1129 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1129 0.144 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1129 0.277 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1129 0.187 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1129 0.169 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1128 0.280 apic_timer_interrupt+0x52 (__ipipe_trace+0x40b)
: +func -1128 0.120 hrtimer_start_range_ns+0x3 (hrtimer_start_expires+0x1c)
: +func -1128 0.117 __hrtimer_start_range_ns+0x9 (hrtimer_start_range_ns+0x15)
: +func -1128 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: +func -1128 0.142 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1128 0.117 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1127 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1127 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1127 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1127 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1127 0.127 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1127 0.132 idle_cpu+0x3 (__hrtimer_start_range_ns+0x5f)
: #func -1127 0.117 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1127 0.138 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
: #func -1126 0.130 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1126 0.124 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1126 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1126 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1126 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1126 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1126 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1125 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1125 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1125 0.117 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1125 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1125 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1125 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1125 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1124 0.120 schedule+0x3 (schedule_hrtimeout_range_clock+0x96)
: +func -1124 0.114 __schedule+0x9 (schedule+0x18)
: +func -1124 0.115 add_preempt_count+0x4 (__schedule+0x23)
: +func -1124 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1124 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1124 0.139 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1124 0.142 debug_smp_processor_id+0x36 (__schedule+0x28)
:| +end 0x80000001 -1124 0.129 debug_smp_processor_id+0x82 (__schedule+0x28)
: +func -1123 0.115 rcu_note_context_switch+0x9 (__schedule+0x3a)
: +func -1123 0.145 rcu_sched_qs+0x3 (rcu_note_context_switch+0x2a)
: #func -1123 0.123 rcu_preempt_qs+0x3 (rcu_note_context_switch+0x147)
: #func -1123 0.111 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1123 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1123 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1123 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1123 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1122 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1122 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1122 0.118 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1122 0.118 ipipe_root_only+0x5 (__schedule+0x73)
:| +begin 0x80000001 -1122 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1122 0.127 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1122 0.129 _raw_spin_lock_irq+0x4 (__schedule+0xba)
: #func -1121 0.115 add_preempt_count+0x4 (_raw_spin_lock_irq+0x1a)
: #func -1121 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1121 0.175 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1121 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1121 0.118 deactivate_task+0x4 (__schedule+0x10e)
: #func -1121 0.114 dequeue_task+0x8 (deactivate_task+0x21)
: #func -1121 0.133 update_rq_clock+0x6 (dequeue_task+0x1a)
:| #begin 0x80000001 -1121 0.154 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1120 0.160 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
: #func -1120 0.121 dequeue_task_fair+0x9 (dequeue_task+0x28)
: #func -1120 0.132 update_curr+0x9 (dequeue_task_fair+0x2f)
: #func -1120 0.142 update_min_vruntime+0x9 (update_curr+0x6e)
: #func -1120 0.129 clear_buddies+0x5 (dequeue_task_fair+0x38)
: #func -1120 0.121 update_min_vruntime+0x9 (dequeue_task_fair+0xa9)
: #func -1120 0.130 hrtick_update+0x5 (dequeue_task_fair+0xb7)
: #func -1119 0.114 idle_balance+0x9 (__schedule+0x252)
: #func -1119 0.133 _raw_spin_unlock+0x3 (idle_balance+0x5f)
: #func -1119 0.126 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1119 0.141 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1119 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1119 0.130 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1119 0.124 __rcu_read_lock+0x3 (idle_balance+0x64)
: #func -1118 0.118 load_balance+0x9 (idle_balance+0xa5)
:| #begin 0x80000001 -1118 0.151 debug_smp_processor_id+0x36 (load_balance+0x25)
:| #end 0x80000001 -1118 0.166 debug_smp_processor_id+0x82 (load_balance+0x25)
: #func -1118 0.154 find_busiest_group+0x9 (load_balance+0x8f)
: #func -1118 0.164 group_balance_cpu+0x3 (update_sd_lb_stats+0x8b)
: #func -1118 0.121 idle_cpu+0x3 (update_sd_lb_stats+0xd7)
: #func -1118 0.115 target_load+0x4 (update_sd_lb_stats+0x102)
: #func -1118 0.236 idle_cpu+0x3 (update_sd_lb_stats+0x14e)
: #func -1117 0.121 source_load+0x4 (update_sd_lb_stats+0x10e)
: #func -1117 0.178 idle_cpu+0x3 (update_sd_lb_stats+0x14e)
: #func -1117 0.115 msecs_to_jiffies+0x3 (idle_balance+0xb4)
: #func -1117 0.115 __rcu_read_unlock+0x4 (idle_balance+0xee)
: #func -1117 0.127 _raw_spin_lock+0x4 (idle_balance+0xf5)
: #func -1117 0.114 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1116 0.124 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1116 0.152 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1116 0.141 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1116 0.130 put_prev_task_fair+0x6 (__schedule+0x275)
: #func -1116 0.115 pick_next_task_fair+0x7 (__schedule+0x28c)
: #func -1116 0.135 pick_next_task_stop+0x4 (__schedule+0x299)
: #func -1116 0.126 pick_next_task_rt+0x5 (__schedule+0x299)
: #func -1116 0.130 pick_next_task_fair+0x7 (__schedule+0x299)
: #func -1115 0.129 pick_next_task_idle+0x3 (__schedule+0x299)
: #func -1115 0.118 taskSwitchLinux+0x6 [sigmatek_lrt] (__schedule+0x2dc)
: #func -1115 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchLinux+0x32 [sigmatek_lrt])
: #func -1115 0.120 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
: #func -1115 0.142 rt_task_set_mode+0x7 (taskSwitchLinux+0x4f [sigmatek_lrt])
: #func -1115 0.127 rt_task_set_mode+0x7 (taskSwitchLinux+0x8d [sigmatek_lrt])
:| #begin 0x80000000 -1115 0.160 __schedule+0x3a3 (schedule+0x18)
:| #func -1115 0.163 __switch_to+0x9 (__schedule+0x50a)
:| #func -1114 0.145 __ipipe_switch_tail+0x4 (__schedule+0x514)
:| #end 0x80000000 -1114 0.126 hard_local_irq_enable+0x16 (__ipipe_switch_tail+0x2e)
:| #begin 0x80000001 -1114 0.142 debug_smp_processor_id+0x36 (__schedule+0x51d)
:| #end 0x80000001 -1114 0.130 debug_smp_processor_id+0x82 (__schedule+0x51d)
: #func -1114 0.127 finish_task_switch+0x7 (__schedule+0x533)
: #func -1114 0.123 _raw_spin_unlock_irq+0x3 (finish_task_switch+0x4a)
: #func -1114 0.123 ipipe_unstall_root+0x3 (_raw_spin_unlock_irq+0x12)
:| #begin 0x80000000 -1113 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1113 0.185 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1113 0.115 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1113 0.117 sub_preempt_count+0x4 (_raw_spin_unlock_irq+0x1c)
: +func -1113 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1113 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1113 0.149 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| +begin 0x80000001 -1112 0.144 debug_smp_processor_id+0x36 (__schedule+0x538)
:| +end 0x80000001 -1112 0.136 debug_smp_processor_id+0x82 (__schedule+0x538)
: +func -1112 0.141 _raw_spin_lock_irqsave+0x5 (post_schedule.part.48+0xc)
: #func -1112 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1112 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1112 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1112 0.157 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1111 0.114 post_schedule_idle+0x3 (post_schedule.part.48+0x22)
: #func -1111 0.120 __ipipe_spin_unlock_debug+0x3 (post_schedule.part.48+0x29)
: #func -1111 0.120 _raw_spin_unlock_irqrestore+0x4 (post_schedule.part.48+0x32)
: #func -1111 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1111 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1111 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1111 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1111 0.120 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1110 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1110 0.187 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1110 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1110 0.118 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: +func -1110 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1110 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1110 0.133 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1109 0.120 sub_preempt_count+0x4 (__schedule+0x56c)
: +func -1109 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| +begin 0x80000001 -1109 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1109 0.154 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1109 0.117 add_preempt_count+0x4 (schedule_preempt_disabled+0x21)
: +func -1109 0.120 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| +begin 0x80000001 -1109 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| +end 0x80000001 -1109 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: +func -1108 0.129 tick_nohz_idle_enter+0x4 (cpu_startup_entry+0x24)
: +func -1108 0.127 set_cpu_sd_state_idle+0x4 (tick_nohz_idle_enter+0x37)
:| +begin 0x80000001 -1108 0.144 debug_smp_processor_id+0x36 (set_cpu_sd_state_idle+0xe)
:| +end 0x80000001 -1108 0.126 debug_smp_processor_id+0x82 (set_cpu_sd_state_idle+0xe)
: +func -1108 0.118 __rcu_read_lock+0x3 (set_cpu_sd_state_idle+0x15)
: +func -1108 0.136 __rcu_read_unlock+0x4 (set_cpu_sd_state_idle+0x4d)
:| #begin 0x80000001 -1108 0.142 debug_smp_processor_id+0x36 (tick_nohz_idle_enter+0x46)
:| #end 0x80000001 -1107 0.129 debug_smp_processor_id+0x82 (tick_nohz_idle_enter+0x46)
: #func -1107 0.120 __tick_nohz_idle_enter+0x9 (tick_nohz_idle_enter+0x5b)
:| #begin 0x80000001 -1107 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x16)
:| #end 0x80000001 -1107 0.129 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x16)
: #func -1107 0.158 ktime_get+0x9 (__tick_nohz_idle_enter+0x1e)
:| #begin 0x80000001 -1107 0.152 debug_smp_processor_id+0x36 (sched_clock_idle_sleep_event+0x8)
:| #end 0x80000001 -1107 0.144 debug_smp_processor_id+0x82 (sched_clock_idle_sleep_event+0x8)
:| #begin 0x80000001 -1106 0.158 debug_smp_processor_id+0x36 (__cycles_2_ns+0x14)
:| #end 0x80000001 -1106 0.149 debug_smp_processor_id+0x82 (__cycles_2_ns+0x14)
:| #begin 0x80000001 -1106 0.142 debug_smp_processor_id+0x36 (__tick_nohz_idle_enter+0x3b5)
:| #end 0x80000001 -1106 0.138 debug_smp_processor_id+0x82 (__tick_nohz_idle_enter+0x3b5)
: #func -1106 0.135 timekeeping_max_deferment+0x5 (__tick_nohz_idle_enter+0x108)
: #func -1106 0.139 rcu_needs_cpu+0x5 (__tick_nohz_idle_enter+0x124)
:| #begin 0x80000001 -1106 0.142 debug_smp_processor_id+0x36 (irq_work_needs_cpu+0xe)
:| #end 0x80000001 -1105 0.129 debug_smp_processor_id+0x82 (irq_work_needs_cpu+0xe)
: #func -1105 0.120 get_next_timer_interrupt+0x9 (__tick_nohz_idle_enter+0x147)
:| #begin 0x80000001 -1105 0.142 debug_smp_processor_id+0x36 (get_next_timer_interrupt+0x25)
:| #end 0x80000001 -1105 0.127 debug_smp_processor_id+0x82 (get_next_timer_interrupt+0x25)
: #func -1105 0.114 _raw_spin_lock+0x4 (get_next_timer_interrupt+0x3f)
: #func -1105 0.115 add_preempt_count+0x4 (_raw_spin_lock+0x15)
: #func -1105 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1105 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1104 0.144 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1104 0.127 _raw_spin_unlock+0x3 (get_next_timer_interrupt+0x158)
: #func -1104 0.117 sub_preempt_count+0x4 (_raw_spin_unlock+0x17)
: #func -1104 0.117 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1104 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1104 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1104 0.124 hrtimer_get_next_event+0x9 (get_next_timer_interrupt+0x162)
:| #begin 0x80000001 -1104 0.142 debug_smp_processor_id+0x36 (hrtimer_get_next_event+0x18)
:| #end 0x80000001 -1103 0.127 debug_smp_processor_id+0x82 (hrtimer_get_next_event+0x18)
: #func -1103 0.149 _raw_spin_lock_irqsave+0x5 (hrtimer_get_next_event+0x26)
: #func -1103 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1103 0.118 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1103 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1103 0.142 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1103 0.117 __ipipe_spin_unlock_debug+0x3 (hrtimer_get_next_event+0x97)
: #func -1102 0.123 _raw_spin_unlock_irqrestore+0x4 (hrtimer_get_next_event+0xa0)
: #func -1102 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1102 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1102 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1102 0.151 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1102 0.115 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1102 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1102 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1101 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1101 0.124 nohz_balance_enter_idle+0x3 (__tick_nohz_idle_enter+0x24a)
: #func -1101 0.124 calc_load_enter_idle+0x4 (__tick_nohz_idle_enter+0x24f)
:| #begin 0x80000001 -1101 0.152 debug_smp_processor_id+0x36 (calc_load_enter_idle+0x13)
:| #end 0x80000001 -1101 0.132 debug_smp_processor_id+0x82 (calc_load_enter_idle+0x13)
: #func -1101 0.115 hrtimer_start+0x3 (__tick_nohz_idle_enter+0x2e1)
: #func -1101 0.114 __hrtimer_start_range_ns+0x9 (hrtimer_start+0x14)
: #func -1100 0.114 lock_hrtimer_base.isra.17+0x6 (__hrtimer_start_range_ns+0x23)
: #func -1100 0.149 _raw_spin_lock_irqsave+0x5 (lock_hrtimer_base.isra.17+0x1c)
: #func -1100 0.115 add_preempt_count+0x4 (_raw_spin_lock_irqsave+0x1d)
: #func -1100 0.117 ipipe_root_only+0x5 (add_preempt_count+0x10)
:| #begin 0x80000001 -1100 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1100 0.163 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
:| #begin 0x80000001 -1100 0.151 debug_smp_processor_id+0x36 (remove_hrtimer.part.15+0x1f)
:| #end 0x80000001 -1099 0.130 debug_smp_processor_id+0x82 (remove_hrtimer.part.15+0x1f)
: #func -1099 0.154 __remove_hrtimer+0x9 (remove_hrtimer.part.15+0x3e)
: #func -1099 0.136 hrtimer_force_reprogram+0x9 (__remove_hrtimer+0x64)
: #func -1099 0.111 tick_program_event+0x4 (hrtimer_force_reprogram+0xc6)
: #func -1099 0.121 clockevents_program_event+0x9 (tick_program_event+0x24)
: #func -1099 0.149 ktime_get+0x9 (clockevents_program_event+0x7a)
: #func -1099 0.118 xnarch_next_htick_shot+0x5 (clockevents_program_event+0xeb)
:| #begin 0x80000000 -1099 0.161 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| *#func -1098 0.124 xntimer_start_aperiodic+0x9 (xnarch_next_htick_shot+0x33)
:| *#func -1098 0.203 xnarch_ns_to_tsc+0x6 (xntimer_start_aperiodic+0x67)
:| *#func -1098 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| #end 0x80000000 -1098 0.136 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| #begin 0x80000001 -1098 0.142 debug_smp_processor_id+0x36 (__hrtimer_start_range_ns+0x46)
:| #end 0x80000001 -1098 0.136 debug_smp_processor_id+0x82 (__hrtimer_start_range_ns+0x46)
: #func -1097 0.115 ktime_add_safe+0x6 (hrtimer_set_expires_range_ns+0x21)
: #func -1097 0.149 enqueue_hrtimer+0x6 (__hrtimer_start_range_ns+0x155)
: #func -1097 0.117 __ipipe_spin_unlock_debug+0x3 (unlock_hrtimer_base.isra.18+0x10)
: #func -1097 0.126 _raw_spin_unlock_irqrestore+0x4 (unlock_hrtimer_base.isra.18+0x1b)
: #func -1097 0.117 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1097 0.120 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1097 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1097 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1096 0.114 sub_preempt_count+0x4 (_raw_spin_unlock_irqrestore+0x21)
: #func -1096 0.118 ipipe_root_only+0x5 (sub_preempt_count+0x10)
:| #begin 0x80000001 -1096 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1096 0.167 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1096 0.121 ipipe_unstall_root+0x3 (tick_nohz_idle_enter+0x60)
:| #begin 0x80000000 -1096 0.146 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1096 0.190 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1095 0.146 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
:| +begin 0x80000001 -1095 0.142 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1095 0.144 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1095 0.115 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1095 0.114 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1095 0.120 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1095 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1094 0.132 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1094 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1094 0.142 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1094 0.130 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1094 0.118 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1094 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1094 0.141 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1094 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1093 0.117 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1093 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1093 0.158 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1093 0.115 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1093 0.120 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1093 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1093 0.129 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1092 0.121 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1092 0.144 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1092! 54.458 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x000000ef -1038 0.129 apic_timer_interrupt+0x3a (__ipipe_halt_root+0x43)
:| +func -1038 0.149 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| +func -1037 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -1037 0.118 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -1037 0.127 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| +func -1037 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| +func -1037 0.169 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -1037 0.130 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -1037 0.170 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -1036 0.133 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -1036 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -1036 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -1036 0.197 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -1036 0.141 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-938 -1036 0.114 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -1036 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -1036 0.258 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -1035 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 582] TCP-Lis -1 -1035 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1035 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1035 0.311 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -1035 0.322 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -1034 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1034 0.124 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1034 0.120 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1034 0.242 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1034 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -1033 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -1033 0.215 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -1033 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -1033 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -1033 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -1033 0.317 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -1032 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -1032 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -1032 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -1032 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -1032 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -1032 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -1031 0.136 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -1031 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1031 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -1031 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -1031 0.120 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -1031 0.167 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -1031 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -1030 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -1030 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -1030 0.289 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -1030 0.221 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 0] -<?>- -1 -1030 0.198 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -1029 0.117 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -1029 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -1029 0.129 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -1029 0.279 xnarch_tsc_to_ns+0x6 (measureIdleTime+0xae [sigmatek_lrt])
:| +func -1029 0.188 __ipipe_do_sync_pipeline+0x4 (__ipipe_dispatch_irq+0x1c1)
:| +func -1029 0.167 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| +end 0x000000ef -1028 0.224 apic_timer_interrupt+0x52 (__ipipe_halt_root+0x43)
:| +begin 0x80000001 -1028 0.142 debug_smp_processor_id+0x36 (default_idle+0x2d)
:| +end 0x80000001 -1028 0.160 debug_smp_processor_id+0x82 (default_idle+0x2d)
: +func -1028 0.149 rcu_idle_exit+0x6 (cpu_startup_entry+0x11f)
:| #begin 0x80000001 -1028 0.142 debug_smp_processor_id+0x36 (rcu_idle_exit+0x1c)
:| #end 0x80000001 -1028 0.130 debug_smp_processor_id+0x82 (rcu_idle_exit+0x1c)
: #func -1028 0.136 rcu_eqs_exit_common.isra.47+0x5 (rcu_idle_exit+0x73)
:| #begin 0x80000001 -1027 0.142 debug_smp_processor_id+0x36 (rcu_eqs_exit_common.isra.47+0x41)
:| #end 0x80000001 -1027 0.130 debug_smp_processor_id+0x82 (rcu_eqs_exit_common.isra.47+0x41)
: #func -1027 0.115 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1027 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1027 0.164 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1027 0.129 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1027 0.121 ipipe_unstall_root+0x3 (ipipe_restore_root+0x20)
:| #begin 0x80000000 -1026 0.127 hard_local_irq_disable+0x17 (ipipe_unstall_root+0xd)
:| #func -1026 0.188 ipipe_root_only+0x5 (ipipe_unstall_root+0x12)
:| +end 0x80000000 -1026 0.114 hard_local_irq_enable+0x16 (ipipe_unstall_root+0x2b)
: +func -1026 0.133 arch_cpu_idle_exit+0x3 (cpu_startup_entry+0x136)
:| +begin 0x80000001 -1026 0.144 debug_smp_processor_id+0x36 (cpu_startup_entry+0x33)
:| +end 0x80000001 -1026 0.145 debug_smp_processor_id+0x82 (cpu_startup_entry+0x33)
: #func -1026 0.115 arch_cpu_idle_enter+0x3 (cpu_startup_entry+0x59)
: #func -1025 0.115 local_touch_nmi+0x3 (arch_cpu_idle_enter+0xd)
: #func -1025 0.121 tick_check_broadcast_expired+0x3 (cpu_startup_entry+0x67)
:| #begin 0x80000001 -1025 0.142 debug_smp_processor_id+0x36 (tick_check_broadcast_expired+0xd)
:| #end 0x80000001 -1025 0.135 debug_smp_processor_id+0x82 (tick_check_broadcast_expired+0xd)
: #func -1025 0.145 rcu_idle_enter+0x5 (cpu_startup_entry+0xd3)
:| #begin 0x80000001 -1025 0.141 debug_smp_processor_id+0x36 (rcu_idle_enter+0x1b)
:| #end 0x80000001 -1025 0.132 debug_smp_processor_id+0x82 (rcu_idle_enter+0x1b)
: #func -1025 0.120 rcu_eqs_enter_common.isra.49+0x6 (rcu_idle_enter+0x76)
:| #begin 0x80000001 -1024 0.142 debug_smp_processor_id+0x36 (rcu_eqs_enter_common.isra.49+0x7d)
:| #end 0x80000001 -1024 0.139 debug_smp_processor_id+0x82 (rcu_eqs_enter_common.isra.49+0x7d)
: #func -1024 0.112 ipipe_restore_root+0x4 (arch_local_irq_restore+0x11)
: #func -1024 0.118 ipipe_root_only+0x5 (ipipe_restore_root+0x10)
:| #begin 0x80000001 -1024 0.163 hard_local_irq_save+0x16 (ipipe_root_only+0xf)
:| #end 0x80000001 -1024 0.152 hard_local_irq_restore+0x15 (ipipe_root_only+0x3f)
: #func -1024 0.114 arch_cpu_idle+0x3 (cpu_startup_entry+0xd8)
: #func -1023 0.120 default_idle+0x3 (arch_cpu_idle+0xe)
:| #begin 0x80000001 -1023 0.142 debug_smp_processor_id+0x36 (default_idle+0xd)
:| #end 0x80000001 -1023 0.129 debug_smp_processor_id+0x82 (default_idle+0xd)
: #func -1023 0.120 __ipipe_halt_root+0x3 (default_idle+0x28)
:| #begin 0x80000000 -1023 0.142 hard_local_irq_disable+0x17 (__ipipe_halt_root+0xd)
:| +end 0x8000000e -1023! 44.987 __ipipe_halt_root+0x41 (default_idle+0x28)
:| +begin 0x00000051 -978 0.142 common_interrupt+0x39 (__ipipe_halt_root+0x43)
:| +func -978 0.145 __ipipe_handle_irq+0x7 (common_interrupt+0x40)
:| +func -978 0.145 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| +func -977 0.142 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| +func -977 0.132 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x26)
:| +func -977 0.132 irq_to_desc+0x3 (__ipipe_dispatch_irq+0x216)
:| +func -977 0.124 __ipipe_ack_fasteoi_irq+0x3 (__ipipe_dispatch_irq+0xa9)
:| +func -977 0.127 hold_ioapic_irq+0x4 (__ipipe_ack_fasteoi_irq+0x10)
:| +func -977 0.127 io_apic_modify_irq.isra.9+0x7 (hold_ioapic_irq+0x41)
:| +func -977 0.112 native_io_apic_read+0x4 (io_apic_read+0x9)
:| +func -977+ 1.369 io_apic_base+0x3 (native_io_apic_read+0x10)
:| +func -975 0.112 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| +func -975 0.130 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| +func -975 0.142 io_apic_sync+0x3 (io_apic_modify_irq.isra.9+0x41)
:| +func -975+ 1.162 io_apic_base+0x3 (io_apic_sync+0xf)
:| +func -974 0.187 ack_apic_level+0x5 (hold_ioapic_irq+0x50)
:| +func -973 0.187 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -973 0.126 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -973 0.144 xnintr_irq_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -973 0.141 irqhandler_fpga+0x9 [sigmatek_lrt] (xnintr_irq_handler+0x78)
:| # func -973+ 6.102 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -967 0.148 xnarch_get_cpu_time+0x4 (irqhandler_fpga+0x132 [sigmatek_lrt])
:| # func -967 0.121 xnarch_tsc_to_ns+0x6 (xnarch_get_cpu_time+0x13)
:| # func -966 0.120 xnarch_tsc_to_ns+0x6 (irqhandler_fpga+0x14c [sigmatek_lrt])
:| # func -966 0.130 period_update+0x4 [sigmatek_lrt] (irqhandler_fpga+0x163 [sigmatek_lrt])
:| # func -966 0.154 lrtdrv_monitoring_start_timed+0x4 [sigmatek_lrt] (period_update+0x1f [sigmatek_lrt])
:| # func -966 0.207 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -966 0.227 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -966 0.144 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -965 0.239 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -965 0.149 rtdm_event_signal+0x5 (period_update+0xd9 [sigmatek_lrt])
:| # func -965 0.161 xnsynch_flush+0x9 (rtdm_event_signal+0x25)
:| # func -965 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 607] realtim 95 -965 0.239 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -965 0.145 __rtdm_synch_flush+0x6 (period_update+0xe3 [sigmatek_lrt])
:| # func -964 0.167 xnsynch_flush+0x9 (__rtdm_synch_flush+0x2f)
:| # func -964 0.209 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 512] VARAN_T 30 -964 0.222 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -964 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 605] cyclic: 30 -964 0.163 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -963 0.138 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 603] Loader: 29 -963 0.152 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -963 0.206 xnpod_resume_thread+0x9 (xnsynch_flush+0x47)
:| # [ 609] backgrou 0 -963 0.234 xnpod_resume_thread+0x39 (xnsynch_flush+0x47)
:| # func -963 0.114 ipipe_raise_irq+0x8 (period_update+0x10f [sigmatek_lrt])
:| # func -963 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (ipipe_raise_irq+0x36)
:| # func -963 0.139 __ipipe_handle_irq+0x7 (ipipe_raise_irq+0x3e)
:| # func -962 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| # func -962 0.141 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| # func -962 0.112 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| # func -962 0.187 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| # func -962 0.176 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| # func -962+ 1.655 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -960+ 1.676 ___xnpod_lock_sched+0x4 (__xnpod_lock_sched+0x19 [sigmatek_lrt])
:| # func -958 0.178 ___xnpod_unlock_sched+0x3 (__xnpod_unlock_sched+0x19 [sigmatek_lrt])
:| # func -958 0.114 rthal_irq_end+0x5 (xnintr_irq_handler+0x14d)
:| # func -958 0.132 irq_to_desc+0x3 (rthal_irq_end+0x1d)
:| # func -958 0.120 irq_to_desc+0x3 (rthal_irq_end+0x28)
:| # func -958 0.120 irq_to_desc+0x3 (rthal_irq_end+0x32)
:| # func -958 0.115 __ipipe_end_fasteoi_irq+0x3 (rthal_irq_end+0x38)
:| # func -958 0.115 release_ioapic_irq+0x5 (__ipipe_end_fasteoi_irq+0x16)
:| # func -957 0.151 __ipipe_spin_lock_irqsave+0x6 (release_ioapic_irq+0x17)
:| # func -957 0.114 io_apic_modify_irq.isra.9+0x7 (release_ioapic_irq+0x29)
:| # func -957 0.114 native_io_apic_read+0x4 (io_apic_read+0x9)
:| # func -957+ 1.349 io_apic_base+0x3 (native_io_apic_read+0x10)
:| # func -956 0.112 native_io_apic_modify+0x5 (io_apic_modify_irq.isra.9+0x36)
:| # func -956 0.132 io_apic_base+0x3 (native_io_apic_modify+0x13)
:| # func -955 0.204 __ipipe_spin_unlock_irqrestore+0x4 (release_ioapic_irq+0x35)
:| # func -955 0.175 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 0] -<?>- -1 -955 0.145 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -955 0.151 ipipe_send_ipi+0x6 (__xnpod_schedule+0x7d)
:| # func -955 0.197 default_send_IPI_mask_logical+0x6 (ipipe_send_ipi+0x40)
:| # func -955 0.127 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -954 0.316 __ipipe_notify_vm_preemption+0x4 (__xnpod_schedule+0x141)
:| # func -954 0.311 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 607] realtim 95 -954 0.212 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -954 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -953 0.118 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -953 0.345 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -953 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -953 0.184 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -953 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -953 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -952 0.124 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -952 0.302 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -952 0.181 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -952 0.152 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -952 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -952! 13.796 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
:| + begin 0x000000ef -938 0.155 apic_timer_interrupt+0x3a (<080ac8fb>)
:| + func -938 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -937 0.130 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -937 0.124 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -937 0.129 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -937 0.144 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -937 0.161 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -937 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -937 0.219 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -936 0.141 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -936 0.123 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -936 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -936 0.206 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -936 0.127 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-838 -936 0.114 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -936 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -935 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -935 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 607] realtim 95 -935 0.135 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -935 0.384 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -934 0.408 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -934 0.207 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -934 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -934 0.145 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -934 0.250 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -933 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -933 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -933 0.209 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -933 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -933 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -933 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -932 0.317 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -932 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -932 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -932 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -932 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -931 0.118 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -931 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -931 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -931 0.132 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -931 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -931 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -931 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -931 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -930 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -930 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -930 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -930 0.304 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -930 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 607] realtim 95 -929 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -929 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -929 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -929 0.276 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -929 0.144 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + end 0x000000ef -929+ 5.305 apic_timer_interrupt+0x52 (<080ac8fb>)
: + func -923 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -923 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -923 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -923 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -923 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -923 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -922 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -922 0.172 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -922 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -922 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -922 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -922 0.273 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -921 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -921 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -921 0.142 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -921 0.144 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -921 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -921 0.158 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -921 0.179 pci_locker_signal_stop+0x3 [sigmatek_lrt] (lrt_ioctl_work+0x768 [sigmatek_lrt])
:| + begin 0x80000001 -920 0.170 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -920 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -920 0.161 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -920 0.429 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -920 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -919 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -919 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -919 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -919 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -919 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -919 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -919 0.142 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -918 0.120 sys_rtdm_ioctl+0x3 (hisyscall_event+0x129)
: + func -918 0.115 __rt_dev_ioctl+0x9 (sys_rtdm_ioctl+0x1d)
: + func -918 0.121 rtdm_context_get+0x6 (__rt_dev_ioctl+0x24)
:| + begin 0x80000000 -918 0.204 hard_local_irq_disable+0x15 (__xnlock_get_irqsave+0xc)
:| # func -918 0.136 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -918 0.124 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -918 0.144 rtdm_in_rt_context+0x1d (__rt_dev_ioctl+0x36)
:| + end 0x80000001 -918 0.126 rtdm_in_rt_context+0x67 (__rt_dev_ioctl+0x36)
: + func -917 0.118 lrt_ioctl+0x3 [sigmatek_lrt] (__rt_dev_ioctl+0x46)
: + func -917 0.120 lrt_ioctl_work+0xc [sigmatek_lrt] (lrt_ioctl+0x10 [sigmatek_lrt])
: + func -917 0.167 __copy_from_user_ll_nozero+0x6 (rtdm_safe_copy_from_user.isra.4+0x23 [sigmatek_lrt])
: + func -917 0.117 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4a2 [sigmatek_lrt])
: + func -917 0.117 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -917 0.121 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -917 0.123 lrtdrv_timing_wait+0x9 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + begin 0x80000001 -917 0.144 lrtdrv_timing_wait+0x39 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
:| + end 0x80000001 -916 0.129 lrtdrv_timing_wait+0x85 [sigmatek_lrt] (lrt_ioctl_work+0x4c9 [sigmatek_lrt])
: + func -916 0.118 rtdm_event_wait+0x3 (lrtdrv_timing_wait+0xdc [sigmatek_lrt])
: + func -916 0.121 rtdm_event_timedwait+0x9 (rtdm_event_wait+0x13)
:| + begin 0x80000000 -916 0.209 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.14+0x9)
:| # func -916 0.161 xnsynch_sleep_on+0x9 (rtdm_event_timedwait+0x8e)
:| # func -916 0.179 xnpod_suspend_thread+0x9 (xnsynch_sleep_on+0x8b)
:| # func -915 0.191 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 607] realtim 95 -915 0.115 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -915 0.314 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -915 0.406 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 605] cyclic: 30 -914 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -914 0.121 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -914 0.141 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -914 0.323 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -914 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -913 0.158 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
: + func -913 0.118 lrtdrv_timing_watchdog+0x6 [sigmatek_lrt] (lrt_ioctl_work+0x4ea [sigmatek_lrt])
: + func -913 0.118 setup_watchdog+0x6 [sigmatek_lrt] (lrtdrv_timing_watchdog+0x46 [sigmatek_lrt])
: + func -913 0.124 rt_task_self+0x3 (setup_watchdog+0x42 [sigmatek_lrt])
: + func -913 0.231 __copy_to_user_ll+0x6 (lrt_ioctl_work+0x52e [sigmatek_lrt])
:| + begin 0x80000001 -913 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -913 0.146 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -912 0.169 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -912! 74.275 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
:| + begin 0x000000ef -838 0.132 apic_timer_interrupt+0x3a (<102d024a>)
:| + func -838 0.154 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -838 0.118 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -838 0.117 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -837 0.142 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -837 0.138 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -837 0.155 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -837 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -837 0.250 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -837 0.138 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -837 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -836 0.138 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -836 0.204 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -836 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-738 -836 0.112 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -836 0.133 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -836 0.258 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -835 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 605] cyclic: 30 -835 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -835 0.378 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -835 0.403 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -834 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -834 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -834 0.139 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -834 0.249 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -834 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -833 0.133 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -833 0.190 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -833 0.169 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -833 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -833 0.163 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -833 0.308 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -832 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -832 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -832 0.187 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -832 0.129 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -832 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -832 0.145 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -832 0.152 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -831 0.136 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -831 0.112 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -831 0.118 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -831 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -831 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -831 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -831 0.167 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -830 0.118 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -830 0.302 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -830 0.305 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 605] cyclic: 30 -830 0.195 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -830 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -829 0.123 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -829 0.259 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -829 0.146 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + end 0x000000ef -829! 90.866 apic_timer_interrupt+0x52 (<102d024a>)
:| + begin 0x000000ef -738 0.130 apic_timer_interrupt+0x3a (<102d0254>)
:| + func -738 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -738 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -738 0.118 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -737 0.132 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -737 0.141 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -737 0.154 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -737 0.115 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -737 0.218 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -737 0.136 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -737 0.112 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -736 0.136 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -736 0.201 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -736 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@-638 -736 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -736 0.135 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -736 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -735 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 605] cyclic: 30 -735 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -735 0.325 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -735 0.397 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -734 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -734 0.123 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -734 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -734 0.212 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -734 0.161 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -734 0.135 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -734 0.185 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -733 0.167 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -733 0.144 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -733 0.160 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -733 0.305 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -733 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -732 0.120 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -732 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -732 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -732 0.120 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -732 0.146 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -732 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -732 0.127 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -731 0.114 __rt_task_wait_period+0x5 (hisyscall_event+0x129)
: + func -731 0.117 rt_task_wait_period+0x4 (__rt_task_wait_period+0x14)
: + func -731 0.121 xnpod_wait_thread_period+0x9 (rt_task_wait_period+0x44)
:| + begin 0x80000000 -731 0.166 hard_local_irq_disable+0x15 (__xnlock_get_irqsave.constprop.43+0x9)
:| # func -731 0.145 xnpod_suspend_thread+0x9 (xnpod_wait_thread_period+0x6c)
:| # func -731 0.169 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 624] -<?>- 99 -731 0.117 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -731 0.301 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -730 0.304 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 605] cyclic: 30 -730 0.194 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -730 0.120 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -730 0.117 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -729 0.259 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| + func -729 0.145 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + end 0x000000ef -729! 65.687 apic_timer_interrupt+0x52 (<102d0254>)
:| + begin 0x000000fd -663 0.155 reschedule_interrupt+0x3a (<1013cb63>)
:| + func -663 0.167 __ipipe_handle_irq+0x7 (reschedule_interrupt+0x41)
:| + func -663 0.124 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -663 0.170 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -663 0.154 __ipipe_ack_apic+0x3 (__ipipe_dispatch_irq+0xa9)
:| + func -663 0.114 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x1db)
:| + func -663 0.212 __ipipe_set_irq_pending+0x6 (__ipipe_dispatch_irq+0x1e4)
:| + func -662 0.142 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0x10e)
:| + end 0x000000fd -662! 653.327 reschedule_interrupt+0x52 (<1013cb63>)
:| + begin 0x000000ef -9 0.164 apic_timer_interrupt+0x3a (<080ac951>)
:| + func -9 0.151 __ipipe_handle_irq+0x7 (apic_timer_interrupt+0x41)
:| + func -9 0.117 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_handle_irq+0xc4)
:| + func -8 0.118 __ipipe_dispatch_irq+0x9 (__ipipe_handle_irq+0xce)
:| + func -8 0.136 __ipipe_ack_hrtimer_irq+0x6 (__ipipe_dispatch_irq+0xa9)
:| + func -8 0.138 lapic_itimer_ack+0x3 (__ipipe_ack_hrtimer_irq+0x59)
:| + func -8 0.152 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x118)
:| # func -8 0.127 irq_hook_handler+0x3 [sigmatek_lrt] (__ipipe_dispatch_irq+0x172)
:| # func -8 0.219 xnintr_clock_handler+0x9 (__ipipe_dispatch_irq+0x182)
:| # func -8 0.148 xntimer_tick_aperiodic+0x9 (xnintr_clock_handler+0xb9)
:| # func -7 0.127 xnthread_periodic_handler+0x3 (xntimer_tick_aperiodic+0xa0)
:| # func -7 0.142 xnpod_resume_thread+0x9 (xnthread_periodic_handler+0x20)
:| # [ 624] -<?>- 99 -7 0.288 xnpod_resume_thread+0x39 (xnthread_periodic_handler+0x20)
:| # func -7 0.126 xntimer_next_local_shot+0x9 (xntimer_tick_aperiodic+0x177)
:| # event tick@60 -7 0.111 xntimer_next_local_shot+0xca (xntimer_tick_aperiodic+0x177)
:| # func -7 0.135 ipipe_timer_set+0x5 (xntimer_next_local_shot+0xd1)
:| # func -6 0.253 lapic_next_deadline+0x5 (ipipe_timer_set+0x67)
:| # func -6 0.176 __xnpod_schedule+0x9 (xnpod_schedule+0x32)
:| # [ 605] cyclic: 30 -6 0.120 __xnpod_schedule+0x58 (xnpod_schedule+0x32)
:| # func -6 0.323 xnsched_pick_next+0x5 (__xnpod_schedule+0xa5)
:| # func -6 0.399 __switch_to+0x9 (__xnpod_schedule+0x3b8)
:| # [ 624] -<?>- 99 -5 0.197 __xnpod_schedule+0x431 (xnpod_schedule+0x32)
:| # func -5 0.126 xnpod_fire_callouts+0x7 (__xnpod_schedule+0x4b4)
:| # func -5 0.142 taskSwitchXenomai+0x9 [sigmatek_lrt] (xnpod_fire_callouts+0x4c)
:| # func -5 0.230 measureIdleTime+0x9 [sigmatek_lrt] (taskSwitchXenomai+0x9a [sigmatek_lrt])
:| # func -4 0.244 xntimer_get_overruns+0x8 (xnpod_wait_thread_period+0x93)
:| # func -4 0.142 __ipipe_restore_head+0x4 (ipipe_restore_pipeline_head+0x42)
:| + end 0x80000000 -4 0.206 hard_local_irq_enable+0x16 (__ipipe_restore_head+0x66)
:| + begin 0x80000001 -4 0.191 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
:| + end 0x80000001 -4 0.142 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
:| + begin 0x80000000 -4 0.172 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
:| + end 0x80000000 -3 0.824 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
: + func -3 0.141 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
: + func -2 0.129 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
:| + begin 0x80000001 -2 0.188 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
:| + end 0x80000001 -2 0.126 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
: + func -2 0.121 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
:| + begin 0x80000001 -2 0.164 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
:| + end 0x80000001 -2 0.126 hard_local_irq_restore+0x15 (ipipe_syscall_hook+0x5b)
: + func -2 0.279 hisyscall_event+0x9 (ipipe_syscall_hook+0x6b)
: + func -1 0.342 xnshadow_sys_trace+0x6 (hisyscall_event+0x129)
: + func -1 0.212 ipipe_trace_frozen_reset+0x5 (xnshadow_sys_trace+0x5e)
: + func -1 0.130 __ipipe_global_path_lock+0x7 (ipipe_trace_frozen_reset+0x14)
: + func -1 0.135 __ipipe_spin_lock_irqsave+0x6 (__ipipe_global_path_lock+0x16)
:| + begin 0x80000001 0 0.634 hard_local_irq_save+0x16 (__ipipe_spin_lock_irqsave+0x12)
:| # func 0 0.148 __ipipe_spin_unlock_irqcomplete+0x4 (__ipipe_global_path_unlock+0x5f)
:| + end 0x80000001 0 0.154 hard_local_irq_restore+0x15 (__ipipe_spin_unlock_irqcomplete+0x25)
< + freeze 0x0009a3b7 0 0.172 xnshadow_sys_trace+0x67 (hisyscall_event+0x129)
| + begin 0x80000001 0 0.190 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x6b)
| + end 0x80000001 0 0.141 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0xad)
| + begin 0x80000000 0 0.181 hard_local_irq_disable+0x17 (__ipipe_syscall_root+0x38)
| + end 0x80000000 0 0.200 __ipipe_syscall_root+0x70 (sysenter_past_esp+0x5b)
+ func 0 0.118 __ipipe_syscall_root+0x5 (sysenter_past_esp+0x5b)
+ func 1 0.132 __ipipe_notify_syscall+0x9 (__ipipe_syscall_root+0x31)
| + begin 0x80000001 1 0.176 hard_local_irq_save+0x16 (__ipipe_notify_syscall+0x18)
| + end 0x80000001 1 0.127 hard_local_irq_restore+0x15 (__ipipe_notify_syscall+0x59)
+ func 1 0.126 ipipe_syscall_hook+0x7 (__ipipe_notify_syscall+0x64)
| + begin 0x80000001 1 0.000 hard_local_irq_save+0x16 (ipipe_syscall_hook+0x1b)
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 20:54 ` Gilles Chanteperdrix
2015-07-13 21:30 ` Johann Obermayr
@ 2015-07-13 21:58 ` Johann Obermayr
2015-07-13 22:39 ` Gilles Chanteperdrix
1 sibling, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 21:58 UTC (permalink / raw)
To: xenomai
Am 13.07.2015 um 22:54 schrieb Gilles Chanteperdrix:
> On Mon, Jul 13, 2015 at 10:45:17PM +0200, Johann Obermayr wrote:
>> Am 13.07.2015 um 22:31 schrieb Gilles Chanteperdrix:
>>> On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
>>>> Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
>>>>> On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
>>>>>> Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
>>>>>>> Johann Obermayr wrote:
>>>>>>>> Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
>>>>>>>>> Johann Obermayr wrote:
>>>>>>>>>> without your application, there are no large latencies.
>>>>>>>>>> with your application see frozen.txt (with latency -f)
>>>>>>>>> I am confused. You mean "our application", not "your application",
>>>>>>>>> right?
>>>>>>>>> lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
>>>>>>>>> project.
>>>>>>>>>
>>>>>>>>>> We see the problem only if one task (background) is accessing the SRAM
>>>>>>>>>> on your PCI-Card. if we stop this task, all is ok.
>>>>>>>>> Again: the Xenomai project does not make PCI-card. So, you probably mean
>>>>>>>>> "our PCI-Card"?
>>>>>>>> yes, our PCI-Card. (sorry for my bad english)
>>>>>>>>>> So we have a higher prior task (pci-locker), that interrupt the
>>>>>>>>>> background task, so that the pci bus get free.
>>>>>>>>> I am not sure I understand your explanations. But the trace is pretty
>>>>>>>>> clear:
>>>>>>>>>
>>>>>>>>> At time -658 the timer is programmed to tick at -561.
>>>>>>>>>
>>>>>>>>>> :| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
>>>>>>>>>> :| + func -651 0.145 lrtdrv_monitoring_irq+0x4
>>>>>>>>>> [sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
>>>>>>>>>> :| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
>>>>>>>>>> (<102d0254>)
>>>>>>>>> But at that point the tick is delayed for 600us. And according to the
>>>>>>>>> trace, the last traced function called before that delay is the function
>>>>>>>>>
>>>>>>>>> ltdrv_monitoring_irq.
>>>>>>>>>
>>>>>>>>> So, I do not know what this irq is doing, but I would suggest having a
>>>>>>>>> close look at it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> hello,
>>>>>>>>
>>>>>>>> i have disable our lrtdrv_monitoring_irq.
>>>>>>>> Only have this callback
>>>>>>>> static void irq_hook_handler(unsigned int irq, unsigned int state)
>>>>>>>> {
>>>>>>>> if (fpga_interrupt == irq && state == 0x01)
>>>>>>>> {
>>>>>>>> time_fpga_irq = rt_timer_tsc();
>>>>>>>> }
>>>>>>>> }
>>>>>>>> same latency
>>>>>>> Maybe, but your trace does not contain enough points to see it. The trace
>>>>>>> should at least contain the "tick@" event which gets missed, so that we
>>>>>>> can see how much the interrupt is delayed, and what was happening at the
>>>>>>> time.
>>>>>>>
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Sorry, here with more points.
>>>>> Ok, what is irq_hook_handler ?
>>>>>
>>>> Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
>>>> PC. But internal in the FPGA there
>>>> are different IRQ sources. One of them is our Tick.
>>>> So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
>>>> handler.
>>>> In our handler we can check, if it our Tick and than we can calc the correct
>>>> time to start our pci_locker task 50us before next Tick-irq.
>>>>
>>>> It's a callback function from some irq function ipipe_raise_irq,
>>>> __ipipe_do_IRQ, __ipipe_handle_irq
>>>> for our own tracing and it save the fpga irq time.
>>>> Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
>>>> of the function.
>>> I see two weird things in your traces:
>>> - irq_hook_handler which is taking a lot of time
>>> - or some APIC related functions taking a lot of time.
>>>
>>> Are you sure your system is not one of those which disable the APIC
>>> during idle period. Is your system UP or SMP?
>>>
>> It's a SMP (Dual core Celeron)
> Real dual core, or hyperthreaded ?
>
>> Kernel cmdline
>> nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1 console=ttyS0,115200
>> BOOT_IMAGE=/bzImage FirstUsbDrive=E console=/dev/null noconsole
>> root=/dev/sda2 rw
> Do you have the same problem without these options ?
> nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1
>
Hello Gilles,
the trouble are only if our SRAM test on our pci-card is running (on core1).
we have see, that the the core0 must wait, if he will access the fpga.
can it be, that the apic has the same trouble ?
Regards
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 21:58 ` Johann Obermayr
@ 2015-07-13 22:39 ` Gilles Chanteperdrix
2015-07-13 23:30 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-13 22:39 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Mon, Jul 13, 2015 at 11:58:03PM +0200, Johann Obermayr wrote:
> Am 13.07.2015 um 22:54 schrieb Gilles Chanteperdrix:
> >On Mon, Jul 13, 2015 at 10:45:17PM +0200, Johann Obermayr wrote:
> >>Am 13.07.2015 um 22:31 schrieb Gilles Chanteperdrix:
> >>>On Mon, Jul 13, 2015 at 10:23:02PM +0200, Johann Obermayr wrote:
> >>>>Am 13.07.2015 um 21:58 schrieb Gilles Chanteperdrix:
> >>>>>On Mon, Jul 13, 2015 at 07:26:11PM +0200, Johann Obermayr wrote:
> >>>>>>Am 13.07.2015 um 19:21 schrieb Gilles Chanteperdrix:
> >>>>>>>Johann Obermayr wrote:
> >>>>>>>>Am 13.07.2015 um 17:24 schrieb Gilles Chanteperdrix:
> >>>>>>>>>Johann Obermayr wrote:
> >>>>>>>>>>without your application, there are no large latencies.
> >>>>>>>>>>with your application see frozen.txt (with latency -f)
> >>>>>>>>>I am confused. You mean "our application", not "your application",
> >>>>>>>>>right?
> >>>>>>>>>lrtdrv_monitoring_irq is not part of the code delivered by the Xenomai
> >>>>>>>>>project.
> >>>>>>>>>
> >>>>>>>>>>We see the problem only if one task (background) is accessing the SRAM
> >>>>>>>>>>on your PCI-Card. if we stop this task, all is ok.
> >>>>>>>>>Again: the Xenomai project does not make PCI-card. So, you probably mean
> >>>>>>>>>"our PCI-Card"?
> >>>>>>>>yes, our PCI-Card. (sorry for my bad english)
> >>>>>>>>>>So we have a higher prior task (pci-locker), that interrupt the
> >>>>>>>>>>background task, so that the pci bus get free.
> >>>>>>>>>I am not sure I understand your explanations. But the trace is pretty
> >>>>>>>>>clear:
> >>>>>>>>>
> >>>>>>>>>At time -658 the timer is programmed to tick at -561.
> >>>>>>>>>
> >>>>>>>>>>:| # event tick@-561 -658 0.112 xntimer_next_local_shot+0xca
> >>>>>>>>>>:| + func -651 0.145 lrtdrv_monitoring_irq+0x4
> >>>>>>>>>>[sigmatek_lrt] (irq_hook_handler+0x32 [sigmatek_lrt])
> >>>>>>>>>>:| + end 0x000000ef -651! 641.640 apic_timer_interrupt+0x52
> >>>>>>>>>>(<102d0254>)
> >>>>>>>>>But at that point the tick is delayed for 600us. And according to the
> >>>>>>>>>trace, the last traced function called before that delay is the function
> >>>>>>>>>
> >>>>>>>>>ltdrv_monitoring_irq.
> >>>>>>>>>
> >>>>>>>>>So, I do not know what this irq is doing, but I would suggest having a
> >>>>>>>>>close look at it.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>hello,
> >>>>>>>>
> >>>>>>>>i have disable our lrtdrv_monitoring_irq.
> >>>>>>>>Only have this callback
> >>>>>>>>static void irq_hook_handler(unsigned int irq, unsigned int state)
> >>>>>>>>{
> >>>>>>>> if (fpga_interrupt == irq && state == 0x01)
> >>>>>>>> {
> >>>>>>>> time_fpga_irq = rt_timer_tsc();
> >>>>>>>> }
> >>>>>>>>}
> >>>>>>>>same latency
> >>>>>>>Maybe, but your trace does not contain enough points to see it. The trace
> >>>>>>>should at least contain the "tick@" event which gets missed, so that we
> >>>>>>>can see how much the interrupt is delayed, and what was happening at the
> >>>>>>>time.
> >>>>>>>
> >>>>>>>
> >>>>>>Hi,
> >>>>>>
> >>>>>>Sorry, here with more points.
> >>>>>Ok, what is irq_hook_handler ?
> >>>>>
> >>>>Ok. on out PCI Card there is a FPGA. This FPGA generate am Interrupt to the
> >>>>PC. But internal in the FPGA there
> >>>>are different IRQ sources. One of them is our Tick.
> >>>>So we measure the time from __ipipe_handle_irq to the our rtdm_irq_request
> >>>>handler.
> >>>>In our handler we can check, if it our Tick and than we can calc the correct
> >>>>time to start our pci_locker task 50us before next Tick-irq.
> >>>>
> >>>>It's a callback function from some irq function ipipe_raise_irq,
> >>>>__ipipe_do_IRQ, __ipipe_handle_irq
> >>>>for our own tracing and it save the fpga irq time.
> >>>>Only __ipipe_handle_irq have state 0x01 (begin irq) & state 0x02 at the end
> >>>>of the function.
> >>>I see two weird things in your traces:
> >>>- irq_hook_handler which is taking a lot of time
> >>>- or some APIC related functions taking a lot of time.
> >>>
> >>>Are you sure your system is not one of those which disable the APIC
> >>>during idle period. Is your system UP or SMP?
> >>>
> >>It's a SMP (Dual core Celeron)
> >Real dual core, or hyperthreaded ?
> >
> >>Kernel cmdline
> >>nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1 console=ttyS0,115200
> >>BOOT_IMAGE=/bzImage FirstUsbDrive=E console=/dev/null noconsole
> >>root=/dev/sda2 rw
> >Do you have the same problem without these options ?
> >nohlt idle=poll xeno_hal.smi=1 isolcpus=0 irqaffinity=1
> >
> Hello Gilles,
>
> the trouble are only if our SRAM test on our pci-card is running (on core1).
> we have see, that the the core0 must wait, if he will access the fpga.
> can it be, that the apic has the same trouble ?
You mean there is an issue with bus contention? Are there many
things occupying the bus when the problem happens?
>
> Regards
>
> _______________________________________________
> Xenomai mailing list
> Xenomai@xenomai.org
> http://xenomai.org/mailman/listinfo/xenomai
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 22:39 ` Gilles Chanteperdrix
@ 2015-07-13 23:30 ` Johann Obermayr
2015-07-14 0:02 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-13 23:30 UTC (permalink / raw)
To: xenomai
Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
> You mean there is an issue with bus contention? Are there many
> things occupying the bus when the problem happens?
Hi,
both cpu core can access our pci-card.
at this time we only have one pci-card with an fpga(with dpram) and a sram.
the fpga generate our system tick. And have a company internal bus system.
this bus system can configured with the dpram & fpga-config register.
we also must read witch irq come from the fpga, because internal he can have
more than one irq sources. and we must quit this irq. (with reading a
fpga register)
we see that if one core accessing the pci-bus, the other core have a
high latency.
core0 = importan & high prior accessing fpga and sram
core1 = linux & visu & some other low prior
if a low prior copy a memblock to/from the sram, the core0 have high
latency in
our irq handler at reading some data from fpga.
so we add a own pci-blocker task on core1. this task ist started about
50us before
next tick-irq is coming. now our irq handle can access the fpga without
waiting.
the old mainboard have pci on the southbridge.
but now we have a new mainboard. the new chipset have no pci bus.
on the mainboard there is now a pci-express to pci bridge chip.
we see that __ipipe_handle_irq is always called correct. but from this
function to our irq-handler, we have
a high latency and we see, that pci-locker task is not started 50us
before irq. sometime he started 100us or more
after the hardware irq.
i have orderd more documents about the bridge chip and wiring diagram
about bridge-chip and apic.
tomorrow i will measure the apic_eoi function time.
regards
Johann
PS: think we need a PCIexpress card, not a pci-card.
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-13 23:30 ` Johann Obermayr
@ 2015-07-14 0:02 ` Gilles Chanteperdrix
2015-07-15 21:00 ` Johann Obermayr
0 siblings, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-14 0:02 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Tue, Jul 14, 2015 at 01:30:59AM +0200, Johann Obermayr wrote:
> Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
> >You mean there is an issue with bus contention? Are there many
> >things occupying the bus when the problem happens?
> Hi,
>
> both cpu core can access our pci-card.
>
> at this time we only have one pci-card with an fpga(with dpram) and a sram.
> the fpga generate our system tick. And have a company internal bus system.
> this bus system can configured with the dpram & fpga-config register.
> we also must read witch irq come from the fpga, because internal he can have
> more than one irq sources. and we must quit this irq. (with reading a fpga
> register)
>
> we see that if one core accessing the pci-bus, the other core have a high
> latency.
> core0 = importan & high prior accessing fpga and sram
> core1 = linux & visu & some other low prior
>
> if a low prior copy a memblock to/from the sram, the core0 have high latency
> in
> our irq handler at reading some data from fpga.
> so we add a own pci-blocker task on core1. this task ist started about 50us
> before
> next tick-irq is coming. now our irq handle can access the fpga without
> waiting.
Well, normally, a PCI bus controller has parameters controlling the
duration of the longest burst, adequately named "PCI latency". You
should also look at whether caching is enabled. A PCI bridge will
normally prefetch from a PCI bar (provided that the FPGA indicates
in the configuration bytes that the memory is prefetchable) to avoid
CPU wait states. Using what was once called MTRR, but has a new name
in new processor, you can cause the procesdor to buffer data before
sending large bursts on the PCI bus. I think what you need to have a
look at is the documentation of the PCIe to PCI bridge, to see if
you can not improve the situation by better configuring it. Also,
reading or writing to FPGA is RAM looks pretty strange, since I
would guess an FPGA could be master on the PCI bus and do DMA
itself, thereby relieving the CPU.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-14 0:02 ` Gilles Chanteperdrix
@ 2015-07-15 21:00 ` Johann Obermayr
2015-07-15 21:21 ` [Xenomai] ipipe trace Johann Obermayr
2015-07-16 6:30 ` [Xenomai] usage of rtdm_task_sleep_abs Gilles Chanteperdrix
0 siblings, 2 replies; 31+ messages in thread
From: Johann Obermayr @ 2015-07-15 21:00 UTC (permalink / raw)
To: xenomai
Am 14.07.2015 um 02:02 schrieb Gilles Chanteperdrix:
> On Tue, Jul 14, 2015 at 01:30:59AM +0200, Johann Obermayr wrote:
>> Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
>>> You mean there is an issue with bus contention? Are there many
>>> things occupying the bus when the problem happens?
>> Hi,
>>
>> both cpu core can access our pci-card.
>>
>> at this time we only have one pci-card with an fpga(with dpram) and a sram.
>> the fpga generate our system tick. And have a company internal bus system.
>> this bus system can configured with the dpram & fpga-config register.
>> we also must read witch irq come from the fpga, because internal he can have
>> more than one irq sources. and we must quit this irq. (with reading a fpga
>> register)
>>
>> we see that if one core accessing the pci-bus, the other core have a high
>> latency.
>> core0 = importan & high prior accessing fpga and sram
>> core1 = linux & visu & some other low prior
>>
>> if a low prior copy a memblock to/from the sram, the core0 have high latency
>> in
>> our irq handler at reading some data from fpga.
>> so we add a own pci-blocker task on core1. this task ist started about 50us
>> before
>> next tick-irq is coming. now our irq handle can access the fpga without
>> waiting.
> Well, normally, a PCI bus controller has parameters controlling the
> duration of the longest burst, adequately named "PCI latency". You
> should also look at whether caching is enabled. A PCI bridge will
> normally prefetch from a PCI bar (provided that the FPGA indicates
> in the configuration bytes that the memory is prefetchable) to avoid
> CPU wait states. Using what was once called MTRR, but has a new name
> in new processor, you can cause the procesdor to buffer data before
> sending large bursts on the PCI bus. I think what you need to have a
> look at is the documentation of the PCIe to PCI bridge, to see if
> you can not improve the situation by better configuring it. Also,
> reading or writing to FPGA is RAM looks pretty strange, since I
> would guess an FPGA could be master on the PCI bus and do DMA
> itself, thereby relieving the CPU.
>
Hello,
how can i check, if the TSC on both core are run syncron ?
Dual Core Celeron, no Hyperthreading.
Thanks
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* [Xenomai] ipipe trace
2015-07-15 21:00 ` Johann Obermayr
@ 2015-07-15 21:21 ` Johann Obermayr
2015-07-16 6:30 ` [Xenomai] usage of rtdm_task_sleep_abs Gilles Chanteperdrix
1 sibling, 0 replies; 31+ messages in thread
From: Johann Obermayr @ 2015-07-15 21:21 UTC (permalink / raw)
To: xenomai
Hello,
in the ipipe trace, i see only one cpu.
Is there a way to show all cpu's ?
Regards
Johann
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-15 21:00 ` Johann Obermayr
2015-07-15 21:21 ` [Xenomai] ipipe trace Johann Obermayr
@ 2015-07-16 6:30 ` Gilles Chanteperdrix
2015-07-16 11:46 ` Johann Obermayr
1 sibling, 1 reply; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-16 6:30 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Wed, Jul 15, 2015 at 11:00:55PM +0200, Johann Obermayr wrote:
> Am 14.07.2015 um 02:02 schrieb Gilles Chanteperdrix:
> >On Tue, Jul 14, 2015 at 01:30:59AM +0200, Johann Obermayr wrote:
> >>Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
> >>>You mean there is an issue with bus contention? Are there many
> >>>things occupying the bus when the problem happens?
> >>Hi,
> >>
> >>both cpu core can access our pci-card.
> >>
> >>at this time we only have one pci-card with an fpga(with dpram) and a sram.
> >>the fpga generate our system tick. And have a company internal bus system.
> >>this bus system can configured with the dpram & fpga-config register.
> >>we also must read witch irq come from the fpga, because internal he can have
> >>more than one irq sources. and we must quit this irq. (with reading a fpga
> >>register)
> >>
> >>we see that if one core accessing the pci-bus, the other core have a high
> >>latency.
> >>core0 = importan & high prior accessing fpga and sram
> >>core1 = linux & visu & some other low prior
> >>
> >>if a low prior copy a memblock to/from the sram, the core0 have high latency
> >>in
> >>our irq handler at reading some data from fpga.
> >>so we add a own pci-blocker task on core1. this task ist started about 50us
> >>before
> >>next tick-irq is coming. now our irq handle can access the fpga without
> >>waiting.
> >Well, normally, a PCI bus controller has parameters controlling the
> >duration of the longest burst, adequately named "PCI latency". You
> >should also look at whether caching is enabled. A PCI bridge will
> >normally prefetch from a PCI bar (provided that the FPGA indicates
> >in the configuration bytes that the memory is prefetchable) to avoid
> >CPU wait states. Using what was once called MTRR, but has a new name
> >in new processor, you can cause the procesdor to buffer data before
> >sending large bursts on the PCI bus. I think what you need to have a
> >look at is the documentation of the PCIe to PCI bridge, to see if
> >you can not improve the situation by better configuring it. Also,
> >reading or writing to FPGA is RAM looks pretty strange, since I
> >would guess an FPGA could be master on the PCI bus and do DMA
> >itself, thereby relieving the CPU.
> >
> Hello,
>
> how can i check, if the TSC on both core are run syncron ?
> Dual Core Celeron, no Hyperthreading.
Quite frankly, I think it is pretty clear what the problem is now:
you are accessing your SRAM in uncached mode, which causes large
wait states in your processor and cause it to misbehave somehow. Try
to fix that, enable pre-fetching on the PCI bridge as well, I
believe everything else are red herrings.
Regards.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-16 6:30 ` [Xenomai] usage of rtdm_task_sleep_abs Gilles Chanteperdrix
@ 2015-07-16 11:46 ` Johann Obermayr
2015-07-17 5:25 ` Gilles Chanteperdrix
0 siblings, 1 reply; 31+ messages in thread
From: Johann Obermayr @ 2015-07-16 11:46 UTC (permalink / raw)
To: xenomai
Am 16.07.2015 um 08:30 schrieb Gilles Chanteperdrix:
> On Wed, Jul 15, 2015 at 11:00:55PM +0200, Johann Obermayr wrote:
>> Am 14.07.2015 um 02:02 schrieb Gilles Chanteperdrix:
>>> On Tue, Jul 14, 2015 at 01:30:59AM +0200, Johann Obermayr wrote:
>>>> Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
>>>>> You mean there is an issue with bus contention? Are there many
>>>>> things occupying the bus when the problem happens?
>>>> Hi,
>>>>
>>>> both cpu core can access our pci-card.
>>>>
>>>> at this time we only have one pci-card with an fpga(with dpram) and a sram.
>>>> the fpga generate our system tick. And have a company internal bus system.
>>>> this bus system can configured with the dpram & fpga-config register.
>>>> we also must read witch irq come from the fpga, because internal he can have
>>>> more than one irq sources. and we must quit this irq. (with reading a fpga
>>>> register)
>>>>
>>>> we see that if one core accessing the pci-bus, the other core have a high
>>>> latency.
>>>> core0 = importan & high prior accessing fpga and sram
>>>> core1 = linux & visu & some other low prior
>>>>
>>>> if a low prior copy a memblock to/from the sram, the core0 have high latency
>>>> in
>>>> our irq handler at reading some data from fpga.
>>>> so we add a own pci-blocker task on core1. this task ist started about 50us
>>>> before
>>>> next tick-irq is coming. now our irq handle can access the fpga without
>>>> waiting.
>>> Well, normally, a PCI bus controller has parameters controlling the
>>> duration of the longest burst, adequately named "PCI latency". You
>>> should also look at whether caching is enabled. A PCI bridge will
>>> normally prefetch from a PCI bar (provided that the FPGA indicates
>>> in the configuration bytes that the memory is prefetchable) to avoid
>>> CPU wait states. Using what was once called MTRR, but has a new name
>>> in new processor, you can cause the procesdor to buffer data before
>>> sending large bursts on the PCI bus. I think what you need to have a
>>> look at is the documentation of the PCIe to PCI bridge, to see if
>>> you can not improve the situation by better configuring it. Also,
>>> reading or writing to FPGA is RAM looks pretty strange, since I
>>> would guess an FPGA could be master on the PCI bus and do DMA
>>> itself, thereby relieving the CPU.
>>>
>> Hello,
>>
>> how can i check, if the TSC on both core are run syncron ?
>> Dual Core Celeron, no Hyperthreading.
> Quite frankly, I think it is pretty clear what the problem is now:
> you are accessing your SRAM in uncached mode, which causes large
> wait states in your processor and cause it to misbehave somehow. Try
> to fix that, enable pre-fetching on the PCI bridge as well, I
> believe everything else are red herrings.
>
> Regards.
>
Hello,
this is our PCI-card
02:04.0 ACCESS Bus: Device 5112:2200
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 32
Interrupt: pin A routed to IRQ 11
Region 0: Memory at f6800000 (32-bit, non-prefetchable) FPGA & SRAM
Region 1: Memory at f7000000 (32-bit, non-prefetchable) DPRAM
Kernel driver in use: Sigmatek PCI
SRAM is a part of the FPGA space. for FPGA we must use non-prefetchable.
i will talk , if there is a way to add a own region for SRAM, that is
prefetchable.
for the bridge chip, i does not found about prefetchable.
Why can sometime Core1 SRAM access not interrupted by the timer/schedule ?
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: [Xenomai] usage of rtdm_task_sleep_abs
2015-07-16 11:46 ` Johann Obermayr
@ 2015-07-17 5:25 ` Gilles Chanteperdrix
0 siblings, 0 replies; 31+ messages in thread
From: Gilles Chanteperdrix @ 2015-07-17 5:25 UTC (permalink / raw)
To: Johann Obermayr; +Cc: xenomai
On Thu, Jul 16, 2015 at 01:46:08PM +0200, Johann Obermayr wrote:
> Am 16.07.2015 um 08:30 schrieb Gilles Chanteperdrix:
> >On Wed, Jul 15, 2015 at 11:00:55PM +0200, Johann Obermayr wrote:
> >>Am 14.07.2015 um 02:02 schrieb Gilles Chanteperdrix:
> >>>On Tue, Jul 14, 2015 at 01:30:59AM +0200, Johann Obermayr wrote:
> >>>>Am 14.07.2015 um 00:39 schrieb Gilles Chanteperdrix:
> >>>>>You mean there is an issue with bus contention? Are there many
> >>>>>things occupying the bus when the problem happens?
> >>>>Hi,
> >>>>
> >>>>both cpu core can access our pci-card.
> >>>>
> >>>>at this time we only have one pci-card with an fpga(with dpram) and a sram.
> >>>>the fpga generate our system tick. And have a company internal bus system.
> >>>>this bus system can configured with the dpram & fpga-config register.
> >>>>we also must read witch irq come from the fpga, because internal he can have
> >>>>more than one irq sources. and we must quit this irq. (with reading a fpga
> >>>>register)
> >>>>
> >>>>we see that if one core accessing the pci-bus, the other core have a high
> >>>>latency.
> >>>>core0 = importan & high prior accessing fpga and sram
> >>>>core1 = linux & visu & some other low prior
> >>>>
> >>>>if a low prior copy a memblock to/from the sram, the core0 have high latency
> >>>>in
> >>>>our irq handler at reading some data from fpga.
> >>>>so we add a own pci-blocker task on core1. this task ist started about 50us
> >>>>before
> >>>>next tick-irq is coming. now our irq handle can access the fpga without
> >>>>waiting.
> >>>Well, normally, a PCI bus controller has parameters controlling the
> >>>duration of the longest burst, adequately named "PCI latency". You
> >>>should also look at whether caching is enabled. A PCI bridge will
> >>>normally prefetch from a PCI bar (provided that the FPGA indicates
> >>>in the configuration bytes that the memory is prefetchable) to avoid
> >>>CPU wait states. Using what was once called MTRR, but has a new name
> >>>in new processor, you can cause the procesdor to buffer data before
> >>>sending large bursts on the PCI bus. I think what you need to have a
> >>>look at is the documentation of the PCIe to PCI bridge, to see if
> >>>you can not improve the situation by better configuring it. Also,
> >>>reading or writing to FPGA is RAM looks pretty strange, since I
> >>>would guess an FPGA could be master on the PCI bus and do DMA
> >>>itself, thereby relieving the CPU.
> >>>
> >>Hello,
> >>
> >>how can i check, if the TSC on both core are run syncron ?
> >>Dual Core Celeron, no Hyperthreading.
> >Quite frankly, I think it is pretty clear what the problem is now:
> >you are accessing your SRAM in uncached mode, which causes large
> >wait states in your processor and cause it to misbehave somehow. Try
> >to fix that, enable pre-fetching on the PCI bridge as well, I
> >believe everything else are red herrings.
> >
> >Regards.
> >
> Hello,
>
> this is our PCI-card
>
> 02:04.0 ACCESS Bus: Device 5112:2200
> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx-
> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 32
> Interrupt: pin A routed to IRQ 11
> Region 0: Memory at f6800000 (32-bit, non-prefetchable) FPGA & SRAM
> Region 1: Memory at f7000000 (32-bit, non-prefetchable) DPRAM
> Kernel driver in use: Sigmatek PCI
>
> SRAM is a part of the FPGA space. for FPGA we must use non-prefetchable.
> i will talk , if there is a way to add a own region for SRAM, that is
> prefetchable.
> for the bridge chip, i does not found about prefetchable.
The DPRAM should probably be made prefetchable as well. Everything
which is memory and not registers should be prefetchable.
>
> Why can sometime Core1 SRAM access not interrupted by the
> timer/schedule ?
It depends on the processor architecture, and I am not an x86
specialist, but maybe when the processor is accessing the SRAM
space, when it is waiting for the FPGA to answer, the interrupt
handling is postponed until the wait is finished. And maybe when it
is in that state, the other core will block for any operation trying
to access the bus.
--
Gilles.
https://click-hack.org
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2015-07-17 5:25 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-09 12:15 [Xenomai] usage of rtdm_task_sleep_abs Johann Obermayr
2015-07-09 14:19 ` Gilles Chanteperdrix
2015-07-09 15:43 ` Johann Obermayr
2015-07-09 15:51 ` Gilles Chanteperdrix
2015-07-10 9:34 ` Johann Obermayr
2015-07-10 9:46 ` Gilles Chanteperdrix
2015-07-10 10:43 ` Johann Obermayr
2015-07-10 10:49 ` Gilles Chanteperdrix
2015-07-13 13:41 ` Johann Obermayr
2015-07-13 14:02 ` Gilles Chanteperdrix
2015-07-13 14:56 ` Johann Obermayr
2015-07-13 15:24 ` Gilles Chanteperdrix
2015-07-13 16:58 ` Johann Obermayr
2015-07-13 17:21 ` Gilles Chanteperdrix
2015-07-13 17:26 ` Johann Obermayr
2015-07-13 19:58 ` Gilles Chanteperdrix
2015-07-13 20:23 ` Johann Obermayr
2015-07-13 20:31 ` Gilles Chanteperdrix
2015-07-13 20:45 ` Johann Obermayr
2015-07-13 20:54 ` Gilles Chanteperdrix
2015-07-13 21:30 ` Johann Obermayr
2015-07-13 21:58 ` Johann Obermayr
2015-07-13 22:39 ` Gilles Chanteperdrix
2015-07-13 23:30 ` Johann Obermayr
2015-07-14 0:02 ` Gilles Chanteperdrix
2015-07-15 21:00 ` Johann Obermayr
2015-07-15 21:21 ` [Xenomai] ipipe trace Johann Obermayr
2015-07-16 6:30 ` [Xenomai] usage of rtdm_task_sleep_abs Gilles Chanteperdrix
2015-07-16 11:46 ` Johann Obermayr
2015-07-17 5:25 ` Gilles Chanteperdrix
2015-07-13 20:35 ` Gilles Chanteperdrix
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.