[Xenomai] xenomai scheduling

[Xenomai] xenomai scheduling

[Sagi Maimon]

Hi all,
Going over the xenomai documentation and code there is something I can't quite understand.

As far as I understand :
One of the xenomai/nucleus tasks is INIT task, this task triggers the Linux kernel scheduler when no other xenomai task needs the CPU time.
In the RPI system when a xenomai task migrate to secondary domain (Linux), the INIT task gets the migrating task priority.
That way xenomai preserve the priority scheme consistent across both schedulers.

But what about Linux timer interrupt?
In case of Linux timer interrupt, ADOS/xenomai will pass interrupt to Linux handling which during scheduler_tick() will invoke the Linux scheduler.
In this case a Linux process can get precedence over a xenomai task, how can it be?

I guess I am missing something, can someone explain?

Thanks,

Sagi Maimon
Manager R&D Software Oscilloquartz SA.
ADVA Optical Networking
Phone: +972 9 775 0124
Mobile: +972 54 5662632
e-mail: SMaimon@advaoptical.com
www.oscilloquartz.com
www.advaoptical.com

Re: [Xenomai] xenomai scheduling

[Jan Kiszka]

On 2018-06-04 10:24, Sagi Maimon wrote:
> Hi all,
> Going over the xenomai documentation and code there is something I can't quite understand.
> 
> As far as I understand :
> One of the xenomai/nucleus tasks is INIT task, this task triggers the Linux kernel scheduler when no other xenomai task needs the CPU time.

I suppose that refers to what is called the ROOT task: A Xenomai task
which represents the whole Linux kernel on a particular core to the
Xenomai scheduler. It runs with lowest priority, as Xenomai idle task.

> In the RPI system when a xenomai task migrate to secondary domain (Linux), the INIT task gets the migrating task priority.
> That way xenomai preserve the priority scheme consistent across both schedulers.

This likely refers to the priority inheritance feature for that ROOT
task in case a Xenomai task migrated to Linux mode. This features was
obsoleted and removed, ignore any related descriptions in older documents.

> 
> But what about Linux timer interrupt?
> In case of Linux timer interrupt, ADOS/xenomai will pass interrupt to Linux handling which during scheduler_tick() will invoke the Linux scheduler.
> In this case a Linux process can get precedence over a xenomai task, how can it be?

Let's look at the simple case first: The timer interrupt is exclusively
used by Linux. In that case, I-pipe will recognize the hardware
interrupt when it arrives, but that interrupt will only be replayed in a
virtualized way for Linux when we switch to that ROOT task, i.e. when
there is no Xenomai RT task runnable at this point. There is no explicit
kicking of the Linux scheduler here, that is done by Linux itself after
running that delayed timer interrupt handler.

It's a bit more complex when the timer is shared by Linux and Xenomai.
In that case, the Linux timer event is abstracted by Xenomai software
timer that uses the shared hardware timer. If that software timer
expired, Xenomai will pend a virtual timer interrupt for Linux and let
that one handle when switching to Linux the next time - just like in the
above case.

Hope that helps to clarify the picture.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [Xenomai] xenomai scheduling

[Sagi Maimon]

Hi Jan,
Thanks for your answer.
Can you please explain something else in regard to your explanation?

The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.

Can you please explain?

Thanks,

Sagi

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: יום ב 04 יוני 2018 13:48
To: Sagi Maimon <SMaimon@advaoptical.com>
Subject: Re: xenomai scheduling

Off-list? Maybe better resend with the list re-added.

On 2018-06-04 12:46, Sagi Maimon wrote:
> Hi Jan,
> Thanks again.
> Can you please explain something else in regard to your explanation?
> 
> The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
> But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.
> 
> Can you please explain?
> 
> Thanks,
> 
> Sagi 
> 
> 
> 
> 
> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com> 
> Sent: יום ב 04 יוני 2018 13:16
> To: Sagi Maimon <SMaimon@advaoptical.com>; Xenomai@xenomai.org
> Subject: Re: xenomai scheduling
> 
> On 2018-06-04 10:24, Sagi Maimon wrote:
>> Hi all,
>> Going over the xenomai documentation and code there is something I can't quite understand.
>>
>> As far as I understand :
>> One of the xenomai/nucleus tasks is INIT task, this task triggers the Linux kernel scheduler when no other xenomai task needs the CPU time.
> 
> I suppose that refers to what is called the ROOT task: A Xenomai task which represents the whole Linux kernel on a particular core to the Xenomai scheduler. It runs with lowest priority, as Xenomai idle task.
> 
>> In the RPI system when a xenomai task migrate to secondary domain (Linux), the INIT task gets the migrating task priority.
>> That way xenomai preserve the priority scheme consistent across both schedulers.
> 
> This likely refers to the priority inheritance feature for that ROOT task in case a Xenomai task migrated to Linux mode. This features was obsoleted and removed, ignore any related descriptions in older documents.
> 
>>
>> But what about Linux timer interrupt?
>> In case of Linux timer interrupt, ADOS/xenomai will pass interrupt to Linux handling which during scheduler_tick() will invoke the Linux scheduler.
>> In this case a Linux process can get precedence over a xenomai task, how can it be?
> 
> Let's look at the simple case first: The timer interrupt is exclusively used by Linux. In that case, I-pipe will recognize the hardware interrupt when it arrives, but that interrupt will only be replayed in a virtualized way for Linux when we switch to that ROOT task, i.e. when there is no Xenomai RT task runnable at this point. There is no explicit kicking of the Linux scheduler here, that is done by Linux itself after running that delayed timer interrupt handler.
> 
> It's a bit more complex when the timer is shared by Linux and Xenomai.
> In that case, the Linux timer event is abstracted by Xenomai software timer that uses the shared hardware timer. If that software timer expired, Xenomai will pend a virtual timer interrupt for Linux and let that one handle when switching to Linux the next time - just like in the above case.
> 
> Hope that helps to clarify the picture.
> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
> 

Re: [Xenomai] xenomai scheduling

[Jan Kiszka]

On 2018-06-04 12:50, Sagi Maimon wrote:
> Hi Jan,
> Thanks for your answer.
> Can you please explain something else in regard to your explanation?
> 
> The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
> But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.
> 
> Can you please explain?

Already on real hardware, timer ticks may come delayed. That does not
get better in virtual environments when a hypervisor time-shares
physical CPUs between multiple guests. So Linux is already prepared for
delayed ticks (for those cases where ticks are still needed -
timekeeping is typically tickless these days).

But, you are right, if a Xenomai RT application starves Linux for too
long (hundreds of milliseconds, if not seconds), Linux will run into
troubles. That's also why there is a watchdog in Xenomai that monitors
run-away RT tasks and is able to kick them out of hard RT. You should
design your application to prevent that safety measure having to fire in
the first place, though.

Jan

> 
> Thanks,
> 
> Sagi
> 
> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com> 
> Sent: יום ב 04 יוני 2018 13:48
> To: Sagi Maimon <SMaimon@advaoptical.com>
> Subject: Re: xenomai scheduling
> 
> Off-list? Maybe better resend with the list re-added.
> 
> On 2018-06-04 12:46, Sagi Maimon wrote:
>> Hi Jan,
>> Thanks again.
>> Can you please explain something else in regard to your explanation?
>>
>> The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
>> But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.
>>
>> Can you please explain?
>>
>> Thanks,
>>
>> Sagi 
>>
>>
>>
>>
>> -----Original Message-----
>> From: Jan Kiszka <jan.kiszka@siemens.com> 
>> Sent: יום ב 04 יוני 2018 13:16
>> To: Sagi Maimon <SMaimon@advaoptical.com>; Xenomai@xenomai.org
>> Subject: Re: xenomai scheduling
>>
>> On 2018-06-04 10:24, Sagi Maimon wrote:
>>> Hi all,
>>> Going over the xenomai documentation and code there is something I can't quite understand.
>>>
>>> As far as I understand :
>>> One of the xenomai/nucleus tasks is INIT task, this task triggers the Linux kernel scheduler when no other xenomai task needs the CPU time.
>>
>> I suppose that refers to what is called the ROOT task: A Xenomai task which represents the whole Linux kernel on a particular core to the Xenomai scheduler. It runs with lowest priority, as Xenomai idle task.
>>
>>> In the RPI system when a xenomai task migrate to secondary domain (Linux), the INIT task gets the migrating task priority.
>>> That way xenomai preserve the priority scheme consistent across both schedulers.
>>
>> This likely refers to the priority inheritance feature for that ROOT task in case a Xenomai task migrated to Linux mode. This features was obsoleted and removed, ignore any related descriptions in older documents.
>>
>>>
>>> But what about Linux timer interrupt?
>>> In case of Linux timer interrupt, ADOS/xenomai will pass interrupt to Linux handling which during scheduler_tick() will invoke the Linux scheduler.
>>> In this case a Linux process can get precedence over a xenomai task, how can it be?
>>
>> Let's look at the simple case first: The timer interrupt is exclusively used by Linux. In that case, I-pipe will recognize the hardware interrupt when it arrives, but that interrupt will only be replayed in a virtualized way for Linux when we switch to that ROOT task, i.e. when there is no Xenomai RT task runnable at this point. There is no explicit kicking of the Linux scheduler here, that is done by Linux itself after running that delayed timer interrupt handler.
>>
>> It's a bit more complex when the timer is shared by Linux and Xenomai.
>> In that case, the Linux timer event is abstracted by Xenomai software timer that uses the shared hardware timer. If that software timer expired, Xenomai will pend a virtual timer interrupt for Linux and let that one handle when switching to Linux the next time - just like in the above case.
>>
>> Hope that helps to clarify the picture.
>>
>> Jan
>>
>> --
>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
>>

Re: [Xenomai] xenomai scheduling

[Sagi Maimon]

Thanks

-----Original Message-----
From: Jan Kiszka <jan.kiszka@siemens.com> 
Sent: יום ב 04 יוני 2018 13:59
To: Sagi Maimon <SMaimon@advaoptical.com>
Cc: Xenomai@xenomai.org
Subject: Re: xenomai scheduling

On 2018-06-04 12:50, Sagi Maimon wrote:
> Hi Jan,
> Thanks for your answer.
> Can you please explain something else in regard to your explanation?
> 
> The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
> But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.
> 
> Can you please explain?

Already on real hardware, timer ticks may come delayed. That does not get better in virtual environments when a hypervisor time-shares physical CPUs between multiple guests. So Linux is already prepared for delayed ticks (for those cases where ticks are still needed - timekeeping is typically tickless these days).

But, you are right, if a Xenomai RT application starves Linux for too long (hundreds of milliseconds, if not seconds), Linux will run into troubles. That's also why there is a watchdog in Xenomai that monitors run-away RT tasks and is able to kick them out of hard RT. You should design your application to prevent that safety measure having to fire in the first place, though.

Jan

> 
> Thanks,
> 
> Sagi
> 
> -----Original Message-----
> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: יום ב 04 יוני 2018 13:48
> To: Sagi Maimon <SMaimon@advaoptical.com>
> Subject: Re: xenomai scheduling
> 
> Off-list? Maybe better resend with the list re-added.
> 
> On 2018-06-04 12:46, Sagi Maimon wrote:
>> Hi Jan,
>> Thanks again.
>> Can you please explain something else in regard to your explanation?
>>
>> The Linux timer ISR does many thing among them the Jiffies update (which effects the Linux time handle).
>> But since the timer interrupt is now delayed , it can affect the Linux time handling mechanism.
>>
>> Can you please explain?
>>
>> Thanks,
>>
>> Sagi
>>
>>
>>
>>
>> -----Original Message-----
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>> Sent: יום ב 04 יוני 2018 13:16
>> To: Sagi Maimon <SMaimon@advaoptical.com>; Xenomai@xenomai.org
>> Subject: Re: xenomai scheduling
>>
>> On 2018-06-04 10:24, Sagi Maimon wrote:
>>> Hi all,
>>> Going over the xenomai documentation and code there is something I can't quite understand.
>>>
>>> As far as I understand :
>>> One of the xenomai/nucleus tasks is INIT task, this task triggers the Linux kernel scheduler when no other xenomai task needs the CPU time.
>>
>> I suppose that refers to what is called the ROOT task: A Xenomai task which represents the whole Linux kernel on a particular core to the Xenomai scheduler. It runs with lowest priority, as Xenomai idle task.
>>
>>> In the RPI system when a xenomai task migrate to secondary domain (Linux), the INIT task gets the migrating task priority.
>>> That way xenomai preserve the priority scheme consistent across both schedulers.
>>
>> This likely refers to the priority inheritance feature for that ROOT task in case a Xenomai task migrated to Linux mode. This features was obsoleted and removed, ignore any related descriptions in older documents.
>>
>>>
>>> But what about Linux timer interrupt?
>>> In case of Linux timer interrupt, ADOS/xenomai will pass interrupt to Linux handling which during scheduler_tick() will invoke the Linux scheduler.
>>> In this case a Linux process can get precedence over a xenomai task, how can it be?
>>
>> Let's look at the simple case first: The timer interrupt is exclusively used by Linux. In that case, I-pipe will recognize the hardware interrupt when it arrives, but that interrupt will only be replayed in a virtualized way for Linux when we switch to that ROOT task, i.e. when there is no Xenomai RT task runnable at this point. There is no explicit kicking of the Linux scheduler here, that is done by Linux itself after running that delayed timer interrupt handler.
>>
>> It's a bit more complex when the timer is shared by Linux and Xenomai.
>> In that case, the Linux timer event is abstracted by Xenomai software timer that uses the shared hardware timer. If that software timer expired, Xenomai will pend a virtual timer interrupt for Linux and let that one handle when switching to Linux the next time - just like in the above case.
>>
>> Hope that helps to clarify the picture.
>>
>> Jan
>>
>> --
>> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate 
>> Competence Center Embedded Linux
>>