[Xenomai] i.MX Timer Tick stall on iMX6Solo

[Xenomai] i.MX Timer Tick stall on iMX6Solo

[Henri Roosen]

Hi all,

I'm running on a iMX6Solo (singlecore) chip. The problem we are seeing is
that a couple of times during the day, the i.MX Timer Tick irq doesn't
trigger and after some time starts running again. So the reprogramming of
the timer is suspect ;-)

I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.

Can anyone help me to make the timer reprogramming robust? Is there a
robust way to do that (instead of adding a factor to the mxc_min_delay like
commit-e1c0a1461512f does)? All help or pointers are highly appreciated!

Thanks,
Henri.

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Gilles Chanteperdrix]

On 07/15/2013 05:48 PM, Henri Roosen wrote:

> Hi all,
>
> I'm running on a iMX6Solo (singlecore) chip. The problem we are seeing is
> that a couple of times during the day, the i.MX Timer Tick irq doesn't
> trigger and after some time starts running again. So the reprogramming of
> the timer is suspect ;-)


That is a documented issue with timers of the type "free-running + match 
register", it means yes, that the timer was programmed for a too short 
delay, and that you had wait for it to wrap to finally tick.

>
> I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>
> Can anyone help me to make the timer reprogramming robust? Is there a
> robust way to do that (instead of adding a factor to the mxc_min_delay like
> commit-e1c0a1461512f does)? All help or pointers are highly appreciated!


The robust solution is the solution used by the commit you cite: use the 
"next_event" callback return value to manually trigger a timer irq.
Unfortunately, this increases the number of timer registers access, and 
so the timer programming latency. But I agree that it is a better solution.

Regards.

-- 
                                                                 Gilles.

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Henri Roosen]

On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix <
gilles.chanteperdrix@xenomai.org> wrote:

> On 07/15/2013 05:48 PM, Henri Roosen wrote:
>
>  Hi all,
>>
>> I'm running on a iMX6Solo (singlecore) chip. The problem we are seeing is
>> that a couple of times during the day, the i.MX Timer Tick irq doesn't
>> trigger and after some time starts running again. So the reprogramming of
>> the timer is suspect ;-)
>>
>
>
> That is a documented issue with timers of the type "free-running + match
> register", it means yes, that the timer was programmed for a too short
> delay, and that you had wait for it to wrap to finally tick.
>
>
>
>> I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>>
>> Can anyone help me to make the timer reprogramming robust? Is there a
>> robust way to do that (instead of adding a factor to the mxc_min_delay
>> like
>> commit-e1c0a1461512f does)? All help or pointers are highly appreciated!
>>
>
>
> The robust solution is the solution used by the commit you cite: use the
> "next_event" callback return value to manually trigger a timer irq.
> Unfortunately, this increases the number of timer registers access, and so
> the timer programming latency. But I agree that it is a better solution.
>
>
Gilles, thanks for your reply! I think there is a misunderstanding about my
referral to commit-e1c0a1461512f: this commit is already in the code I use
(ipipe-3.0-imx6q). So the robust way should be implemented already, however
it still fails to program the timer in time every once in a while.

Actually I was referring to the commit because it added a factor of 2 to
the mxc_min_delay. I don't know why this factor of 2, Was it added 'just to
be sure'?

I'm trying to find a solution that solves the reprogramming of the timer so
that it always is programmed in time. Do you have a suggestion for
calculating the correct mxc_min_delay?

Or would you know a way around this mxc_min_delay?

Thanks!



> Regards.
>
> --
>                                                                 Gilles.
>

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Henri Roosen]

On Tue, Jul 16, 2013 at 9:01 AM, Henri Roosen <henriroosen@gmail.com> wrote:

>
>
>
> On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix <
> gilles.chanteperdrix@xenomai.org> wrote:
>
>> On 07/15/2013 05:48 PM, Henri Roosen wrote:
>>
>>  Hi all,
>>>
>>> I'm running on a iMX6Solo (singlecore) chip. The problem we are seeing is
>>> that a couple of times during the day, the i.MX Timer Tick irq doesn't
>>> trigger and after some time starts running again. So the reprogramming of
>>> the timer is suspect ;-)
>>>
>>
>>
>> That is a documented issue with timers of the type "free-running + match
>> register", it means yes, that the timer was programmed for a too short
>> delay, and that you had wait for it to wrap to finally tick.
>>
>>
>>
>>> I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>>>
>>> Can anyone help me to make the timer reprogramming robust? Is there a
>>> robust way to do that (instead of adding a factor to the mxc_min_delay
>>> like
>>> commit-e1c0a1461512f does)? All help or pointers are highly appreciated!
>>>
>>
>>
>> The robust solution is the solution used by the commit you cite: use the
>> "next_event" callback return value to manually trigger a timer irq.
>> Unfortunately, this increases the number of timer registers access, and
>> so the timer programming latency. But I agree that it is a better solution.
>>
>>
> Gilles, thanks for your reply! I think there is a misunderstanding about
> my referral to commit-e1c0a1461512f: this commit is already in the code I
> use (ipipe-3.0-imx6q). So the robust way should be implemented already,
> however it still fails to program the timer in time every once in a while.
>
> Actually I was referring to the commit because it added a factor of 2 to
> the mxc_min_delay. I don't know why this factor of 2, Was it added 'just to
> be sure'?
>
> Sorry, forget about the factor 2; debugging learned the  __ipipe_cpu_freq
is 0 anyway, so a delay of 1 is used.

Now looking into the reprogramming of the timer again: I noticed
that v2_set_next_event() is missing the re-reading of the TCN like the
function mx1_2_set_next_event() does .
I will add this and do the duration tests. Will keep you posted about the
outcome.

I'm trying to find a solution that solves the reprogramming of the timer so
> that it always is programmed in time. Do you have a suggestion for
> calculating the correct mxc_min_delay?
>
> Or would you know a way around this mxc_min_delay?
>
> Thanks!
>
>
>
>> Regards.
>>
>> --
>>                                                                 Gilles.
>>
>
>

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Henri Roosen]

On Tue, Jul 16, 2013 at 10:04 AM, Henri Roosen <henriroosen@gmail.com>wrote:

>
>
>
> On Tue, Jul 16, 2013 at 9:01 AM, Henri Roosen <henriroosen@gmail.com>wrote:
>
>>
>>
>>
>> On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix <
>> gilles.chanteperdrix@xenomai.org> wrote:
>>
>>> On 07/15/2013 05:48 PM, Henri Roosen wrote:
>>>
>>>  Hi all,
>>>>
>>>> I'm running on a iMX6Solo (singlecore) chip. The problem we are seeing
>>>> is
>>>> that a couple of times during the day, the i.MX Timer Tick irq doesn't
>>>> trigger and after some time starts running again. So the reprogramming
>>>> of
>>>> the timer is suspect ;-)
>>>>
>>>
>>>
>>> That is a documented issue with timers of the type "free-running + match
>>> register", it means yes, that the timer was programmed for a too short
>>> delay, and that you had wait for it to wrap to finally tick.
>>>
>>>
>>>
>>>> I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>>>>
>>>> Can anyone help me to make the timer reprogramming robust? Is there a
>>>> robust way to do that (instead of adding a factor to the mxc_min_delay
>>>> like
>>>> commit-e1c0a1461512f does)? All help or pointers are highly appreciated!
>>>>
>>>
>>>
>>> The robust solution is the solution used by the commit you cite: use the
>>> "next_event" callback return value to manually trigger a timer irq.
>>> Unfortunately, this increases the number of timer registers access, and
>>> so the timer programming latency. But I agree that it is a better solution.
>>>
>>>
>> Gilles, thanks for your reply! I think there is a misunderstanding about
>> my referral to commit-e1c0a1461512f: this commit is already in the code I
>> use (ipipe-3.0-imx6q). So the robust way should be implemented already,
>> however it still fails to program the timer in time every once in a while.
>>
>> Actually I was referring to the commit because it added a factor of 2 to
>> the mxc_min_delay. I don't know why this factor of 2, Was it added 'just to
>> be sure'?
>>
>> Sorry, forget about the factor 2; debugging learned the  __ipipe_cpu_freq
> is 0 anyway, so a delay of 1 is used.
>
> Now looking into the reprogramming of the timer again: I noticed
> that v2_set_next_event() is missing the re-reading of the TCN like the
> function mx1_2_set_next_event() does .
> I will add this and do the duration tests. Will keep you posted about the
> outcome.
>
> This seems to have fixed that timer stall issue we were seeing.
Thanks Gilles for pointing us in the right direction!


>  I'm trying to find a solution that solves the reprogramming of the timer
>> so that it always is programmed in time. Do you have a suggestion for
>> calculating the correct mxc_min_delay?
>>
>> Or would you know a way around this mxc_min_delay?
>>
>> Thanks!
>>
>>
>>
>>> Regards.
>>>
>>> --
>>>                                                                 Gilles.
>>>
>>
>>
>

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Gilles Chanteperdrix]

On 07/18/2013 12:46 PM, Henri Roosen wrote:

>
>
>
> On Tue, Jul 16, 2013 at 10:04 AM, Henri Roosen <henriroosen@gmail.com
> <mailto:henriroosen@gmail.com>> wrote:
>
>
>
>
>     On Tue, Jul 16, 2013 at 9:01 AM, Henri Roosen <henriroosen@gmail.com
>     <mailto:henriroosen@gmail.com>> wrote:
>
>
>
>
>         On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix
>         <gilles.chanteperdrix@xenomai.org
>         <mailto:gilles.chanteperdrix@xenomai.org>> wrote:
>
>             On 07/15/2013 05:48 PM, Henri Roosen wrote:
>
>                 Hi all,
>
>                 I'm running on a iMX6Solo (singlecore) chip. The problem
>                 we are seeing is
>                 that a couple of times during the day, the i.MX Timer
>                 Tick irq doesn't
>                 trigger and after some time starts running again. So the
>                 reprogramming of
>                 the timer is suspect ;-)
>
>
>
>             That is a documented issue with timers of the type
>             "free-running + match register", it means yes, that the
>             timer was programmed for a too short delay, and that you had
>             wait for it to wrap to finally tick.
>
>
>
>                 I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>
>                 Can anyone help me to make the timer reprogramming
>                 robust? Is there a
>                 robust way to do that (instead of adding a factor to the
>                 mxc_min_delay like
>                 commit-e1c0a1461512f does)? All help or pointers are
>                 highly appreciated!
>
>
>
>             The robust solution is the solution used by the commit you
>             cite: use the "next_event" callback return value to manually
>             trigger a timer irq.
>             Unfortunately, this increases the number of timer registers
>             access, and so the timer programming latency. But I agree
>             that it is a better solution.
>
>
>         Gilles, thanks for your reply! I think there is a
>         misunderstanding about my referral to commit-e1c0a1461512f: this
>         commit is already in the code I use (ipipe-3.0-imx6q). So the
>         robust way should be implemented already, however it still fails
>         to program the timer in time every once in a while.
>
>         Actually I was referring to the commit because it added a factor
>         of 2 to the mxc_min_delay. I don't know why this factor of 2,
>         Was it added 'just to be sure'?
>
>     Sorry, forget about the factor 2; debugging learned the
>       __ipipe_cpu_freq is 0 anyway, so a delay of 1 is used.
>
>     Now looking into the reprogramming of the timer again: I noticed
>     that v2_set_next_event() is missing the re-reading of the TCN like
>     the function mx1_2_set_next_event() does .
>     I will add this and do the duration tests. Will keep you posted
>     about the outcome.
>
> This seems to have fixed that timer stall issue we were seeing.
> Thanks Gilles for pointing us in the right direction!


You are welcome, if you send the pâtch, it could be merged in the I-pipe 
tree.

-- 
                                                                 Gilles.

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Henri Roosen]

On Thu, Jul 18, 2013 at 1:17 PM, Gilles Chanteperdrix <
gilles.chanteperdrix@xenomai.org> wrote:

> On 07/18/2013 12:46 PM, Henri Roosen wrote:
>
>
>>
>>
>> On Tue, Jul 16, 2013 at 10:04 AM, Henri Roosen <henriroosen@gmail.com
>> <mailto:henriroosen@gmail.com>**> wrote:
>>
>>
>>
>>
>>     On Tue, Jul 16, 2013 at 9:01 AM, Henri Roosen <henriroosen@gmail.com
>>     <mailto:henriroosen@gmail.com>**> wrote:
>>
>>
>>
>>
>>         On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix
>>         <gilles.chanteperdrix@xenomai.**org<gilles.chanteperdrix@xenomai.org>
>>         <mailto:gilles.chanteperdrix@**xenomai.org<gilles.chanteperdrix@xenomai.org>>>
>> wrote:
>>
>>             On 07/15/2013 05:48 PM, Henri Roosen wrote:
>>
>>                 Hi all,
>>
>>                 I'm running on a iMX6Solo (singlecore) chip. The problem
>>                 we are seeing is
>>                 that a couple of times during the day, the i.MX Timer
>>                 Tick irq doesn't
>>                 trigger and after some time starts running again. So the
>>                 reprogramming of
>>                 the timer is suspect ;-)
>>
>>
>>
>>             That is a documented issue with timers of the type
>>             "free-running + match register", it means yes, that the
>>             timer was programmed for a too short delay, and that you had
>>             wait for it to wrap to finally tick.
>>
>>
>>
>>                 I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>>
>>                 Can anyone help me to make the timer reprogramming
>>                 robust? Is there a
>>                 robust way to do that (instead of adding a factor to the
>>                 mxc_min_delay like
>>                 commit-e1c0a1461512f does)? All help or pointers are
>>                 highly appreciated!
>>
>>
>>
>>             The robust solution is the solution used by the commit you
>>             cite: use the "next_event" callback return value to manually
>>             trigger a timer irq.
>>             Unfortunately, this increases the number of timer registers
>>             access, and so the timer programming latency. But I agree
>>             that it is a better solution.
>>
>>
>>         Gilles, thanks for your reply! I think there is a
>>         misunderstanding about my referral to commit-e1c0a1461512f: this
>>         commit is already in the code I use (ipipe-3.0-imx6q). So the
>>         robust way should be implemented already, however it still fails
>>         to program the timer in time every once in a while.
>>
>>         Actually I was referring to the commit because it added a factor
>>         of 2 to the mxc_min_delay. I don't know why this factor of 2,
>>         Was it added 'just to be sure'?
>>
>>     Sorry, forget about the factor 2; debugging learned the
>>       __ipipe_cpu_freq is 0 anyway, so a delay of 1 is used.
>>
>>     Now looking into the reprogramming of the timer again: I noticed
>>     that v2_set_next_event() is missing the re-reading of the TCN like
>>     the function mx1_2_set_next_event() does .
>>     I will add this and do the duration tests. Will keep you posted
>>     about the outcome.
>>
>> This seems to have fixed that timer stall issue we were seeing.
>> Thanks Gilles for pointing us in the right direction!
>>
>
>
> You are welcome, if you send the pâtch, it could be merged in the I-pipe
> tree.


Note that I'm using the ipipe-3.0-imx6q (freescale tree) from your ipipe
git repository.
It is actually commit-b3743135efb7fffffb864e468a922afa537222d9 that removed
the essential code from arch/arm/plat-mxc/time.c. Now, because I am not
using CONFIG_LOCAL_TIMERS for the iMX6Solo kernel, I can easily revert that
commit, but I don't know what the impact is for someone using
CONFIG_LOCAL_TIMERS...
Maybe you could review that commit?


>
> --
>                                                                 Gilles.
>

Re: [Xenomai] i.MX Timer Tick stall on iMX6Solo

[Gilles Chanteperdrix]

On 07/18/2013 03:43 PM, Henri Roosen wrote:

> On Thu, Jul 18, 2013 at 1:17 PM, Gilles Chanteperdrix<
> gilles.chanteperdrix@xenomai.org>  wrote:
>
>> On 07/18/2013 12:46 PM, Henri Roosen wrote:
>>
>>
>>>
>>>
>>> On Tue, Jul 16, 2013 at 10:04 AM, Henri Roosen<henriroosen@gmail.com
>>> <mailto:henriroosen@gmail.com>**>  wrote:
>>>
>>>
>>>
>>>
>>>      On Tue, Jul 16, 2013 at 9:01 AM, Henri Roosen<henriroosen@gmail.com
>>>      <mailto:henriroosen@gmail.com>**>  wrote:
>>>
>>>
>>>
>>>
>>>          On Mon, Jul 15, 2013 at 8:21 PM, Gilles Chanteperdrix
>>>          <gilles.chanteperdrix@xenomai.**org<gilles.chanteperdrix@xenomai.org>
>>>          <mailto:gilles.chanteperdrix@**xenomai.org<gilles.chanteperdrix@xenomai.org>>>
>>> wrote:
>>>
>>>              On 07/15/2013 05:48 PM, Henri Roosen wrote:
>>>
>>>                  Hi all,
>>>
>>>                  I'm running on a iMX6Solo (singlecore) chip. The problem
>>>                  we are seeing is
>>>                  that a couple of times during the day, the i.MX Timer
>>>                  Tick irq doesn't
>>>                  trigger and after some time starts running again. So the
>>>                  reprogramming of
>>>                  the timer is suspect ;-)
>>>
>>>
>>>
>>>              That is a documented issue with timers of the type
>>>              "free-running + match register", it means yes, that the
>>>              timer was programmed for a too short delay, and that you had
>>>              wait for it to wrap to finally tick.
>>>
>>>
>>>
>>>                  I'm using ipipe-3.0-imx6q and Xenomai 2.6.2.1.
>>>
>>>                  Can anyone help me to make the timer reprogramming
>>>                  robust? Is there a
>>>                  robust way to do that (instead of adding a factor to the
>>>                  mxc_min_delay like
>>>                  commit-e1c0a1461512f does)? All help or pointers are
>>>                  highly appreciated!
>>>
>>>
>>>
>>>              The robust solution is the solution used by the commit you
>>>              cite: use the "next_event" callback return value to manually
>>>              trigger a timer irq.
>>>              Unfortunately, this increases the number of timer registers
>>>              access, and so the timer programming latency. But I agree
>>>              that it is a better solution.
>>>
>>>
>>>          Gilles, thanks for your reply! I think there is a
>>>          misunderstanding about my referral to commit-e1c0a1461512f: this
>>>          commit is already in the code I use (ipipe-3.0-imx6q). So the
>>>          robust way should be implemented already, however it still fails
>>>          to program the timer in time every once in a while.
>>>
>>>          Actually I was referring to the commit because it added a factor
>>>          of 2 to the mxc_min_delay. I don't know why this factor of 2,
>>>          Was it added 'just to be sure'?
>>>
>>>      Sorry, forget about the factor 2; debugging learned the
>>>        __ipipe_cpu_freq is 0 anyway, so a delay of 1 is used.
>>>
>>>      Now looking into the reprogramming of the timer again: I noticed
>>>      that v2_set_next_event() is missing the re-reading of the TCN like
>>>      the function mx1_2_set_next_event() does .
>>>      I will add this and do the duration tests. Will keep you posted
>>>      about the outcome.
>>>
>>> This seems to have fixed that timer stall issue we were seeing.
>>> Thanks Gilles for pointing us in the right direction!
>>>
>>
>>
>> You are welcome, if you send the pâtch, it could be merged in the I-pipe
>> tree.
>
>
> Note that I'm using the ipipe-3.0-imx6q (freescale tree) from your ipipe
> git repository.
> It is actually commit-b3743135efb7fffffb864e468a922afa537222d9 that removed
> the essential code from arch/arm/plat-mxc/time.c. Now, because I am not
> using CONFIG_LOCAL_TIMERS for the iMX6Solo kernel, I can easily revert that
> commit, but I don't know what the impact is for someone using
> CONFIG_LOCAL_TIMERS...
> Maybe you could review that commit?


Well, enable_wait_mode is set to "false" with CONFIG_IPIPE and 
CONFIG_SMP, so, we do not really care of the broadcast mode, and can put 
back the re-reading of the mxc register, maybe compiled conditionnaly if 
CONFIG_IPIPE is enabled.

-- 
                                                                 Gilles.