[Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a 
mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just 
application) of the patch that Gilles referenced in the "IRQ latency" 
thread last week:

http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c

Of perhaps critical note is that I'm building Linux with CONFIG_PREEMPT. 
  The possible problem that I'm about to describe appears specific to 
CONFIG_PREEMPT being defined.  To date, I have not been able to trigger 
the problem if I rebuild the kernel with CONFIG_NONE.

Almost exactly as with last week's "IRQ latency" thread, I have a GPIO 
(164 in my case) configured as an input, and configured to trigger a 
real-time interrupt upon a falling edge of the signal.  The virtual 
interrupt number assigned by Xenomai for that GPIO is 324 (0x144 hex).

What I have found via I-Pipe tracing is that sometimes many, many 
milliseconds elapse within the call to __ipipe_handle_irq() from the 
inlined function ipipe_handle_chained_irq() in gpio_demux_inner(), e.g.:

> :|   +begin   0x00000144 -33547    0.925  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
> :|   +end     0x00000144  -196+   1.370  gpio_demux_inner+0x88 (gpio_irq_handler+0x178)

The present I-Pipe for 2.6.38.8 for ARM patch set patches the 
plat-omap-specific implementation of gpio_irq_handler() to 
unconditionally disable unmasking of the GPIO bank interrupt until just 
before that function returns.  So, if we somehow get "stuck" in 
gpio_demux_inner() for an extended period, the bank interrupt remains 
masked, and new GPIO line state transitions will go unhandled during 
that period.

I find that the GPIO interrupt is processed as would be expected, and 
any RT tasks that have work to do are scheduled to do that work.

What appears to be going awry is that when it becomes time to return to 
Linux domain, the I-Pipe machinery finds that there is an interrupt 
pending for that domain.  Rather than gpio_demux_inner() running to 
completion, and winding back to gpio_handle_irq(), the new interrupt is 
processed, and it can be many, many milliseconds before 
gpio_demux_inner() gets a chance to run and finish its work.

As it happens, my code is the source of that new (virtual) interrupt, 
and it's possible that I'm exercising a use case that hasn't been 
considered.  On the other hand, I could easily imagine other cases where 
interrupts happen to appear at arbitrary points along the timeline.

The sequence of events is as follows:

1. The interrupt fires while the system is running in the Linux domain, 
and the real-time interrupt handler for GPIO 164 calls rt_sem_v() to 
wake up a native Xenomai kernel task that is blocked on rt_sem_p().
I can see that from Xenomai's perspective, processing of the interrupt 
finishes just after that point, e.g. around -33527 in this sample:

> :|   +func               -33558    0.814  __ipipe_grab_irq+0x10 (__irq_svc+0x3c)
> :|   +begin   0xffffffff -33557    0.851  __ipipe_grab_irq+0x60 (__irq_svc+0x3c)
> :|   +func               -33556+   1.481  __ipipe_handle_irq+0x14 (__ipipe_grab_irq+0x6c)
> :|   +func               -33555+   1.037  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
> :|   +func               -33554    0.925  gpio_irq_handler+0x10 (__ipipe_handle_irq+0x150)
> :|   +func               -33553+   1.333  omap_mask_ack_irq+0x10 (gpio_irq_handler+0x2c)
> :|   +func               -33552    1.000  irq_get_irq_data+0x10 (gpio_irq_handler+0x34)
> :|   +(0x01)  0xfb058018 -33551+   1.777  gpio_irq_handler+0x1b0 (__ipipe_handle_irq+0x150) /* custom trace point */
> :|   +(0x02)  0x00000010 -33549    0.629  gpio_irq_handler+0x15c (__ipipe_handle_irq+0x150) /* custom trace point */
> :|   +func               -33548    0.740  gpio_demux_inner+0x10 (gpio_irq_handler+0x178)
> :|   +begin   0x00000144 -33547    0.925  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
> :|   +func               -33547    0.740  __ipipe_handle_irq+0x14 (gpio_demux_inner+0x80)
> :|   +func               -33546    0.925  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
> :|   +func               -33545    0.925  __ipipe_ack_edge_irq+0x10 (__ipipe_handle_irq+0x150)
> :|   +func               -33544    0.962  gpio_ack_irq+0x10 (__ipipe_ack_edge_irq+0x24)
> :|   +func               -33543+   1.444  __ipipe_walk_pipeline+0x10 (__ipipe_handle_irq+0x1a0)
> :|  + func               -33542    0.962  ipipe_suspend_domain+0x10 (__ipipe_walk_pipeline+0x68)
> :|  + func               -33541+   1.037  __ipipe_sync_stage+0x14 (ipipe_suspend_domain+0x9c)
> :|  # func               -33540+   1.592  xnintr_irq_handler+0x14 (__ipipe_sync_stage+0x124)
> :|  # func               -33538    1.000  my_isr_rt+0x10 [phy_srv] (xnintr_irq_handler+0x60)
/* do a bit of work */
> :|  # func               -33533    0.703  rt_sem_v+0x10 (physrv_htm_isr_rt+0x24 [phy_srv])
> :|  # func               -33532    1.000  xnsynch_wakeup_one_sleeper+0x10 (rt_sem_v+0x64)
> :|  # func               -33531    0.925  xnpod_resume_thread+0x10 (xnsynch_wakeup_one_sleeper+0xac)
> :|  # [   -1] -<?>-   60 -33530+   1.851  xnpod_resume_thread+0x64 (xnsynch_wakeup_one_sleeper+0xac)
> :|  # func               -33528+   1.222  rthal_irq_end+0x10 (xnintr_irq_handler+0x138)
> :|  # func               -33527+   1.333  __ipipe_end_edge_irq+0x10 (rthal_irq_end+0x3c)
> :|  # func               -33526+   1.037  __xnpod_schedule+0x14 (xnintr_irq_handler+0x1cc)
> :|  # [ 1633] -<?>-   -1 -33525    0.777  __xnpod_schedule+0x9c (xnintr_irq_handler+0x1cc)
> :|  # func               -33524+   1.555  xnsched_pick_next+0x10 (__xnpod_schedule+0xd4)
> :|  # func               -33523    0.888  ipipe_mute_pic+0x10 (__xnpod_schedule+0x17c)
> :|  # func               -33522+   2.111  omap3_intc_mute+0x10 (ipipe_mute_pic+0x1c)
> :|  # func               -33520    0.703  ipipe_unstall_pipeline_head+0x10 (__xnpod_schedule+0x2ac)
> :|  + end     0x80000000 -33519+   1.851  ipipe_unstall_pipeline_head+0x88 (__xnpod_schedule+0x2ac)


2.  That task that had been waiting on rt_sem_p() awakens and collects 
the data signaled by the GPIO transition, and in turn wakes up another 
native Xenomai kernel task via rt_sem_v().

3.  The second task invokes a write entry point on an RTDM driver.

4.  The RTDM driver may decide to place a copy of the data in a known 
location in kernel memory, and call rtdm_nrtsig_pend() to invoke a 
non-realtime handler in Linux kernel space.  This results in a virtual 
interrupt being generated for the Linux domain, e.g.:

> :   + func               -33238+   1.037  ipipe_trigger_irq+0x10 (my_write+0x84 [my_rtdm_driver])
> :|  + begin   0x80000001 -33237    0.925  ipipe_trigger_irq+0x74 (my_write+0x84 [my_rtdm_driver])
> :|  + func               -33236+   1.481  __ipipe_handle_irq+0x14 (ipipe_trigger_irq+0x80)
> :|  + func               -33235+   1.370  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
> :|  + end     0x80000001 -33233+   1.555  ipipe_trigger_irq+0x90 (my_write+0x84 [my_rtdm_driver])

5.  The non-realtime handler schedules a softint via tasklet_schedule()

6.  The tasklet retrieves the data posted by the real-time task.

My hope would be that steps 5 and 6 would only run after 
gpio_demux_inner() had a chance to unwind back to gpio_handle_irq(). 
Instead, upon transition back to the Linux domain, the new interrupt is 
processed instead, followed by the soft IRQ, and then Linux does various 
housekeeping tasks, here's an example snippet:

> :   + func               -33119    1.000  xnsched_finish_unlocked_switch+0x10 (__xnpod_schedule+0x4c0)
> :|  + begin   0x80000000 -33118+   1.037  xnsched_finish_unlocked_switch+0x30 (__xnpod_schedule+0x4c0)
> :|  # [ 1633] -<?>-   -1 -33116+   1.481  __xnpod_schedule+0x4e4 (xnintr_irq_handler+0x1cc)
> :|   +func               -33115+   1.222  __ipipe_sync_stage+0x14 (ipipe_suspend_domain+0x9c)
> :|   #end     0x80000000 -33114    0.851  __ipipe_sync_stage+0xf8 (ipipe_suspend_domain+0x9c)
> :    #func               -33113    0.666  __ipipe_do_IRQ+0x10 (__ipipe_sync_stage+0x1bc)
> :    #func               -33112+   1.111  asm_do_IRQ+0x10 (__ipipe_do_IRQ+0x1c)
> :    #func               -33111    0.851  irq_enter+0x10 (asm_do_IRQ+0x28)
> :    #func               -33110    0.925  rcu_exit_nohz+0x10 (irq_enter+0x18)
> :    #func               -33109+   1.407  idle_cpu+0x10 (irq_enter+0x20)
> :    #func               -33108    0.814  ipipe_check_context+0x10 (irq_enter+0x60)
> :    #func               -33107+   1.074  __ipipe_noack_irq+0x10 (asm_do_IRQ+0x84)
> :    #func               -33106    0.888  irq_exit+0x10 (asm_do_IRQ+0x88)
> :    #func               -33105+   1.111  ipipe_check_context+0x10 (irq_exit+0x1c)
> :    #func               -33104    1.000  rcu_enter_nohz+0x10 (irq_exit+0x60)
> :    #func               -33103    0.703  idle_cpu+0x10 (irq_exit+0x68)
> :    #func               -33102    0.740  ipipe_check_context+0x10 (irq_exit+0xa8)
> :|   #begin   0x80000000 -33102    0.888  __ipipe_sync_stage+0x1d4 (ipipe_suspend_domain+0x9c)
> :|   #end     0x80000000 -33101    0.592  __ipipe_sync_stage+0xf8 (ipipe_suspend_domain+0x9c)
> :    #func               -33100    0.888  irq_enter+0x10 (__ipipe_sync_stage+0x14c)
> :    #func               -33099    0.666  rcu_exit_nohz+0x10 (irq_enter+0x18)
> :    #func               -33099    0.629  idle_cpu+0x10 (irq_enter+0x20)
> :    #func               -33098+   1.148  ipipe_check_context+0x10 (irq_enter+0x60)
/* __tasklet_schedule */
...
> :    #func               -33091    0.592  __ipipe_restore_root+0x10 (__tasklet_schedule+0x12c)
> :    #func               -33090    0.962  ipipe_check_context+0x10 (__ipipe_restore_root+0x20)
> :|   #begin   0x80000001 -33089    0.666  __ipipe_restore_root+0x40 (__tasklet_schedule+0x12c)
> :|   #end     0x80000001 -33088    0.666  __ipipe_restore_root+0x60 (__tasklet_schedule+0x12c)
> :    #func               -33088    0.851  irq_exit+0x10 (__ipipe_sync_stage+0x168)
> :    #func               -33087    0.740  ipipe_check_context+0x10 (irq_exit+0x1c)
> :    #func               -33086+   1.111  __do_softirq+0x14 (irq_exit+0x5c)
...

We don't seem to get around to finishing gpio_demux_inner() until time -196.

> :|   +begin   0x80000000  -219+   1.481  schedule+0x5ac (schedule_timeout+0x2ec)
> :|   +func                -218    1.000  __ipipe_switch_to_notifier_call_chain+0x10 (__switch_to+0x2c)
> :|   #func                -217    0.666  atomic_notifier_call_chain+0x14 (__ipipe_switch_to_notifier_call_chain+0x6c)
> :|   #func                -216    0.629  __atomic_notifier_call_chain+0x14 (atomic_notifier_call_chain+0x28)
> :|   #func                -216    0.962  ipipe_check_context+0x10 (__atomic_notifier_call_chain+0x3c)
> :|   #func                -215    0.703  notifier_call_chain+0x10 (__atomic_notifier_call_chain+0x64)
> :|   #func                -214    0.851  vfp_notifier+0x10 (notifier_call_chain+0x3c)
> :|   #func                -213+   1.111  thumbee_notifier+0x10 (notifier_call_chain+0x3c)
> :|   #func                -212+   1.666  ipipe_check_context+0x10 (__atomic_notifier_call_chain+0x70
> :|   +end     0x80000000  -210+   1.111  schedule+0x5d4 (preempt_schedule_irq+0x64)
> :    +func                -209    0.666  finish_task_switch+0x10 (schedule+0x60c)
> :|   +begin   0x80000001  -209    0.703  finish_task_switch+0x40 (schedule+0x60c)
> :|   #end     0x80000001  -208    0.925  finish_task_switch+0x60 (schedule+0x60c)
> :    #func                -207    0.888  __ipipe_unstall_root+0x10 (finish_task_switch+0x8c)
> :|   #begin   0x80000000  -206    0.666  __ipipe_unstall_root+0x2c (finish_task_switch+0x8c)
> :|   #func                -205    0.925  ipipe_check_context+0x10 (__ipipe_unstall_root+0x34)
> :|   +end     0x80000000  -205    0.740  __ipipe_unstall_root+0x68 (finish_task_switch+0x8c)
> :    +func                -204+   1.629  ipipe_check_context+0x10 (schedule+0x664)
> :|   +begin   0x80000001  -202    0.814  preempt_schedule_irq+0x7c (__ipipe_preempt_schedule_irq+0x84)
> :|   #end     0x80000001  -201    0.592  preempt_schedule_irq+0x9c (__ipipe_preempt_schedule_irq+0x84)
> :    #func                -201+   1.148  ipipe_check_context+0x10 (preempt_schedule_irq+0xa8)
> :|   #begin   0x80000000  -200+   3.185  __ipipe_preempt_schedule_irq+0x9c (__ipipe_sync_stage+0x188)
> :|   +end     0x00000144  -196+   1.370  gpio_demux_inner+0x88 (gpio_irq_handler+0x178)
> :|   +func                -195+   2.962  gpio_demux_inner+0x10 (gpio_irq_handler+0x18c)
> :|   +(0x02)  0x00000010  -192    0.629  gpio_irq_handler+0x15c (__ipipe_handle_irq+0x150) /* a custom trace point */
> :|   +func                -191    0.925  gpio_demux_inner+0x10 (gpio_irq_handler+0x178)
> :|   +begin   0x00000144  -191+   1.111  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
> :|   +func                -189+   1.518  __ipipe_handle_irq+0x14 (gpio_demux_inner+0x80)
> :|   +func                -188+   2.148  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
> :|   +func                -186+   1.074  __ipipe_ack_edge_irq+0x10 (__ipipe_handle_irq+0x150)
> :|   +func                -185+   1.370  gpio_ack_irq+0x10 (__ipipe_ack_edge_irq+0x24)
> :|   +func                -183+   1.333  __ipipe_walk_pipeline+0x10 (__ipipe_handle_irq+0x1a0)
> :|  + func                -182    0.740  ipipe_suspend_domain+0x10 (__ipipe_walk_pipeline+0x68)
> :|  + func                -181+   1.407  __ipipe_sync_stage+0x14 (ipipe_suspend_domain+0x9c)
> :|  # func                -180+   1.888  xnintr_irq_handler+0x14 (__ipipe_sync_stage+0x124)
/* process the new GPIO interrupt */

I don't presently understand why we get back to running 
gpio_demux_inner() at this particular point, rather than sometime during 
the prior 33 milliseconds.  Note that once gpio_demux_inner() gets a 
chance to unwind, we detect the next (generated long ago) GPIO 
transition almost immediately thereafter.

My intuition is that the overall situation may have something to do with 
the fact that when CONFIG_PREEMPT enabled, __ipipe_sync_stage() calls 
the CONFIG_PREEMPT-specific, non-NULL, implementation 
of__ipipe_preempt_schedule_irq(), but that's speculation.

I'm hoping that the Xenomai community can shed some further light on 
what's going awry, and perhaps suggest a fix.  If possible, I would 
prefer to retain the option to build the kernel with CONFIG_PREEMPT, but 
without encountering the extended latencies documented above.

Thanks much and Best Regards,
Mitch

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 02:31 AM, Mitchell Tasman wrote:
> Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a 
> mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just 
> application) of the patch that Gilles referenced in the "IRQ latency" 
> thread last week:
> 
> http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c
> 
> Of perhaps critical note is that I'm building Linux with CONFIG_PREEMPT. 
>   The possible problem that I'm about to describe appears specific to 
> CONFIG_PREEMPT being defined.  To date, I have not been able to trigger 
> the problem if I rebuild the kernel with CONFIG_NONE.
> 
> Almost exactly as with last week's "IRQ latency" thread, I have a GPIO 
> (164 in my case) configured as an input, and configured to trigger a 
> real-time interrupt upon a falling edge of the signal.  The virtual 
> interrupt number assigned by Xenomai for that GPIO is 324 (0x144 hex).
> 
> What I have found via I-Pipe tracing is that sometimes many, many 
> milliseconds elapse within the call to __ipipe_handle_irq() from the 
> inlined function ipipe_handle_chained_irq() in gpio_demux_inner(), e.g.:
> 
>> :|   +begin   0x00000144 -33547    0.925  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
>> :|   +end     0x00000144  -196+   1.370  gpio_demux_inner+0x88 (gpio_irq_handler+0x178)
> 
> The present I-Pipe for 2.6.38.8 for ARM patch set patches the 
> plat-omap-specific implementation of gpio_irq_handler() to 
> unconditionally disable unmasking of the GPIO bank interrupt until just 
> before that function returns.  So, if we somehow get "stuck" in 
> gpio_demux_inner() for an extended period, the bank interrupt remains 
> masked, and new GPIO line state transitions will go unhandled during 
> that period.
> 
> I find that the GPIO interrupt is processed as would be expected, and 
> any RT tasks that have work to do are scheduled to do that work.
> 
> What appears to be going awry is that when it becomes time to return to 
> Linux domain, the I-Pipe machinery finds that there is an interrupt 
> pending for that domain.  Rather than gpio_demux_inner() running to 
> completion, and winding back to gpio_handle_irq(), the new interrupt is 
> processed, and it can be many, many milliseconds before 
> gpio_demux_inner() gets a chance to run and finish its work.
> 
> As it happens, my code is the source of that new (virtual) interrupt, 
> and it's possible that I'm exercising a use case that hasn't been 
> considered.  On the other hand, I could easily imagine other cases where 
> interrupts happen to appear at arbitrary points along the timeline.
> 
> The sequence of events is as follows:
> 
> 1. The interrupt fires while the system is running in the Linux domain, 
> and the real-time interrupt handler for GPIO 164 calls rt_sem_v() to 
> wake up a native Xenomai kernel task that is blocked on rt_sem_p().
> I can see that from Xenomai's perspective, processing of the interrupt 
> finishes just after that point, e.g. around -33527 in this sample:
> 
>> :|   +func               -33558    0.814  __ipipe_grab_irq+0x10 (__irq_svc+0x3c)
>> :|   +begin   0xffffffff -33557    0.851  __ipipe_grab_irq+0x60 (__irq_svc+0x3c)
>> :|   +func               -33556+   1.481  __ipipe_handle_irq+0x14 (__ipipe_grab_irq+0x6c)
>> :|   +func               -33555+   1.037  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
>> :|   +func               -33554    0.925  gpio_irq_handler+0x10 (__ipipe_handle_irq+0x150)
>> :|   +func               -33553+   1.333  omap_mask_ack_irq+0x10 (gpio_irq_handler+0x2c)
>> :|   +func               -33552    1.000  irq_get_irq_data+0x10 (gpio_irq_handler+0x34)
>> :|   +(0x01)  0xfb058018 -33551+   1.777  gpio_irq_handler+0x1b0 (__ipipe_handle_irq+0x150) /* custom trace point */
>> :|   +(0x02)  0x00000010 -33549    0.629  gpio_irq_handler+0x15c (__ipipe_handle_irq+0x150) /* custom trace point */
>> :|   +func               -33548    0.740  gpio_demux_inner+0x10 (gpio_irq_handler+0x178)
>> :|   +begin   0x00000144 -33547    0.925  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
>> :|   +func               -33547    0.740  __ipipe_handle_irq+0x14 (gpio_demux_inner+0x80)
>> :|   +func               -33546    0.925  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
>> :|   +func               -33545    0.925  __ipipe_ack_edge_irq+0x10 (__ipipe_handle_irq+0x150)
>> :|   +func               -33544    0.962  gpio_ack_irq+0x10 (__ipipe_ack_edge_irq+0x24)
>> :|   +func               -33543+   1.444  __ipipe_walk_pipeline+0x10 (__ipipe_handle_irq+0x1a0)
>> :|  + func               -33542    0.962  ipipe_suspend_domain+0x10 (__ipipe_walk_pipeline+0x68)
>> :|  + func               -33541+   1.037  __ipipe_sync_stage+0x14 (ipipe_suspend_domain+0x9c)
>> :|  # func               -33540+   1.592  xnintr_irq_handler+0x14 (__ipipe_sync_stage+0x124)
>> :|  # func               -33538    1.000  my_isr_rt+0x10 [phy_srv] (xnintr_irq_handler+0x60)
> /* do a bit of work */
>> :|  # func               -33533    0.703  rt_sem_v+0x10 (physrv_htm_isr_rt+0x24 [phy_srv])
>> :|  # func               -33532    1.000  xnsynch_wakeup_one_sleeper+0x10 (rt_sem_v+0x64)
>> :|  # func               -33531    0.925  xnpod_resume_thread+0x10 (xnsynch_wakeup_one_sleeper+0xac)
>> :|  # [   -1] -<?>-   60 -33530+   1.851  xnpod_resume_thread+0x64 (xnsynch_wakeup_one_sleeper+0xac)
>> :|  # func               -33528+   1.222  rthal_irq_end+0x10 (xnintr_irq_handler+0x138)
>> :|  # func               -33527+   1.333  __ipipe_end_edge_irq+0x10 (rthal_irq_end+0x3c)
>> :|  # func               -33526+   1.037  __xnpod_schedule+0x14 (xnintr_irq_handler+0x1cc)
>> :|  # [ 1633] -<?>-   -1 -33525    0.777  __xnpod_schedule+0x9c (xnintr_irq_handler+0x1cc)
>> :|  # func               -33524+   1.555  xnsched_pick_next+0x10 (__xnpod_schedule+0xd4)
>> :|  # func               -33523    0.888  ipipe_mute_pic+0x10 (__xnpod_schedule+0x17c)
>> :|  # func               -33522+   2.111  omap3_intc_mute+0x10 (ipipe_mute_pic+0x1c)
>> :|  # func               -33520    0.703  ipipe_unstall_pipeline_head+0x10 (__xnpod_schedule+0x2ac)
>> :|  + end     0x80000000 -33519+   1.851  ipipe_unstall_pipeline_head+0x88 (__xnpod_schedule+0x2ac)
> 
> 
> 2.  That task that had been waiting on rt_sem_p() awakens and collects 
> the data signaled by the GPIO transition, and in turn wakes up another 
> native Xenomai kernel task via rt_sem_v().
> 
> 3.  The second task invokes a write entry point on an RTDM driver.
> 
> 4.  The RTDM driver may decide to place a copy of the data in a known 
> location in kernel memory, and call rtdm_nrtsig_pend() to invoke a 
> non-realtime handler in Linux kernel space.  This results in a virtual 
> interrupt being generated for the Linux domain, e.g.:
> 
>> :   + func               -33238+   1.037  ipipe_trigger_irq+0x10 (my_write+0x84 [my_rtdm_driver])
>> :|  + begin   0x80000001 -33237    0.925  ipipe_trigger_irq+0x74 (my_write+0x84 [my_rtdm_driver])
>> :|  + func               -33236+   1.481  __ipipe_handle_irq+0x14 (ipipe_trigger_irq+0x80)
>> :|  + func               -33235+   1.370  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
>> :|  + end     0x80000001 -33233+   1.555  ipipe_trigger_irq+0x90 (my_write+0x84 [my_rtdm_driver])
> 
> 5.  The non-realtime handler schedules a softint via tasklet_schedule()
> 
> 6.  The tasklet retrieves the data posted by the real-time task.
> 
> My hope would be that steps 5 and 6 would only run after 
> gpio_demux_inner() had a chance to unwind back to gpio_handle_irq(). 
> Instead, upon transition back to the Linux domain, the new interrupt is 
> processed instead, followed by the soft IRQ, and then Linux does various 
> housekeeping tasks, here's an example snippet:
> 
>> :   + func               -33119    1.000  xnsched_finish_unlocked_switch+0x10 (__xnpod_schedule+0x4c0)
>> :|  + begin   0x80000000 -33118+   1.037  xnsched_finish_unlocked_switch+0x30 (__xnpod_schedule+0x4c0)
>> :|  # [ 1633] -<?>-   -1 -33116+   1.481  __xnpod_schedule+0x4e4 (xnintr_irq_handler+0x1cc)
>> :|   +func               -33115+   1.222  __ipipe_sync_stage+0x14 (ipipe_suspend_domain+0x9c)
>> :|   #end     0x80000000 -33114    0.851  __ipipe_sync_stage+0xf8 (ipipe_suspend_domain+0x9c)
>> :    #func               -33113    0.666  __ipipe_do_IRQ+0x10 (__ipipe_sync_stage+0x1bc)
>> :    #func               -33112+   1.111  asm_do_IRQ+0x10 (__ipipe_do_IRQ+0x1c)
>> :    #func               -33111    0.851  irq_enter+0x10 (asm_do_IRQ+0x28)
>> :    #func               -33110    0.925  rcu_exit_nohz+0x10 (irq_enter+0x18)
>> :    #func               -33109+   1.407  idle_cpu+0x10 (irq_enter+0x20)
>> :    #func               -33108    0.814  ipipe_check_context+0x10 (irq_enter+0x60)
>> :    #func               -33107+   1.074  __ipipe_noack_irq+0x10 (asm_do_IRQ+0x84)
>> :    #func               -33106    0.888  irq_exit+0x10 (asm_do_IRQ+0x88)
>> :    #func               -33105+   1.111  ipipe_check_context+0x10 (irq_exit+0x1c)
>> :    #func               -33104    1.000  rcu_enter_nohz+0x10 (irq_exit+0x60)
>> :    #func               -33103    0.703  idle_cpu+0x10 (irq_exit+0x68)
>> :    #func               -33102    0.740  ipipe_check_context+0x10 (irq_exit+0xa8)
>> :|   #begin   0x80000000 -33102    0.888  __ipipe_sync_stage+0x1d4 (ipipe_suspend_domain+0x9c)
>> :|   #end     0x80000000 -33101    0.592  __ipipe_sync_stage+0xf8 (ipipe_suspend_domain+0x9c)
>> :    #func               -33100    0.888  irq_enter+0x10 (__ipipe_sync_stage+0x14c)
>> :    #func               -33099    0.666  rcu_exit_nohz+0x10 (irq_enter+0x18)
>> :    #func               -33099    0.629  idle_cpu+0x10 (irq_enter+0x20)
>> :    #func               -33098+   1.148  ipipe_check_context+0x10 (irq_enter+0x60)
> /* __tasklet_schedule */
> ...
>> :    #func               -33091    0.592  __ipipe_restore_root+0x10 (__tasklet_schedule+0x12c)
>> :    #func               -33090    0.962  ipipe_check_context+0x10 (__ipipe_restore_root+0x20)
>> :|   #begin   0x80000001 -33089    0.666  __ipipe_restore_root+0x40 (__tasklet_schedule+0x12c)
>> :|   #end     0x80000001 -33088    0.666  __ipipe_restore_root+0x60 (__tasklet_schedule+0x12c)
>> :    #func               -33088    0.851  irq_exit+0x10 (__ipipe_sync_stage+0x168)
>> :    #func               -33087    0.740  ipipe_check_context+0x10 (irq_exit+0x1c)
>> :    #func               -33086+   1.111  __do_softirq+0x14 (irq_exit+0x5c)
> ...
> 
> We don't seem to get around to finishing gpio_demux_inner() until time -196.
> 
>> :|   +begin   0x80000000  -219+   1.481  schedule+0x5ac (schedule_timeout+0x2ec)
>> :|   +func                -218    1.000  __ipipe_switch_to_notifier_call_chain+0x10 (__switch_to+0x2c)
>> :|   #func                -217    0.666  atomic_notifier_call_chain+0x14 (__ipipe_switch_to_notifier_call_chain+0x6c)
>> :|   #func                -216    0.629  __atomic_notifier_call_chain+0x14 (atomic_notifier_call_chain+0x28)
>> :|   #func                -216    0.962  ipipe_check_context+0x10 (__atomic_notifier_call_chain+0x3c)
>> :|   #func                -215    0.703  notifier_call_chain+0x10 (__atomic_notifier_call_chain+0x64)
>> :|   #func                -214    0.851  vfp_notifier+0x10 (notifier_call_chain+0x3c)
>> :|   #func                -213+   1.111  thumbee_notifier+0x10 (notifier_call_chain+0x3c)
>> :|   #func                -212+   1.666  ipipe_check_context+0x10 (__atomic_notifier_call_chain+0x70
>> :|   +end     0x80000000  -210+   1.111  schedule+0x5d4 (preempt_schedule_irq+0x64)
>> :    +func                -209    0.666  finish_task_switch+0x10 (schedule+0x60c)
>> :|   +begin   0x80000001  -209    0.703  finish_task_switch+0x40 (schedule+0x60c)
>> :|   #end     0x80000001  -208    0.925  finish_task_switch+0x60 (schedule+0x60c)
>> :    #func                -207    0.888  __ipipe_unstall_root+0x10 (finish_task_switch+0x8c)
>> :|   #begin   0x80000000  -206    0.666  __ipipe_unstall_root+0x2c (finish_task_switch+0x8c)
>> :|   #func                -205    0.925  ipipe_check_context+0x10 (__ipipe_unstall_root+0x34)
>> :|   +end     0x80000000  -205    0.740  __ipipe_unstall_root+0x68 (finish_task_switch+0x8c)
>> :    +func                -204+   1.629  ipipe_check_context+0x10 (schedule+0x664)
>> :|   +begin   0x80000001  -202    0.814  preempt_schedule_irq+0x7c (__ipipe_preempt_schedule_irq+0x84)
>> :|   #end     0x80000001  -201    0.592  preempt_schedule_irq+0x9c (__ipipe_preempt_schedule_irq+0x84)
>> :    #func                -201+   1.148  ipipe_check_context+0x10 (preempt_schedule_irq+0xa8)
>> :|   #begin   0x80000000  -200+   3.185  __ipipe_preempt_schedule_irq+0x9c (__ipipe_sync_stage+0x188)
>> :|   +end     0x00000144  -196+   1.370  gpio_demux_inner+0x88 (gpio_irq_handler+0x178)
>> :|   +func                -195+   2.962  gpio_demux_inner+0x10 (gpio_irq_handler+0x18c)
>> :|   +(0x02)  0x00000010  -192    0.629  gpio_irq_handler+0x15c (__ipipe_handle_irq+0x150) /* a custom trace point */
>> :|   +func                -191    0.925  gpio_demux_inner+0x10 (gpio_irq_handler+0x178)
>> :|   +begin   0x00000144  -191+   1.111  gpio_demux_inner+0x74 (gpio_irq_handler+0x178)
>> :|   +func                -189+   1.518  __ipipe_handle_irq+0x14 (gpio_demux_inner+0x80)
>> :|   +func                -188+   2.148  __ipipe_set_irq_pending+0x10 (__ipipe_handle_irq+0x120)
>> :|   +func                -186+   1.074  __ipipe_ack_edge_irq+0x10 (__ipipe_handle_irq+0x150)
>> :|   +func                -185+   1.370  gpio_ack_irq+0x10 (__ipipe_ack_edge_irq+0x24)
>> :|   +func                -183+   1.333  __ipipe_walk_pipeline+0x10 (__ipipe_handle_irq+0x1a0)

The problem is here: __ipipe_walk_pipeline should not be called before
gpio_demux_inner or gpio_irq_handler are finished (and have unmasked the
parent irq). This is what the patch is supposed to avoid.


-- 
                                                                Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 02:31 AM, Mitchell Tasman wrote:
> Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a 
> mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just 
> application) of the patch that Gilles referenced in the "IRQ latency" 
> thread last week:
> 
> http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c

Please try the following patch instead:

diff --git a/arch/arm/include/asm/ipipe.h b/arch/arm/include/asm/ipipe.h
index 9b17c06..a510b09 100644
--- a/arch/arm/include/asm/ipipe.h
+++ b/arch/arm/include/asm/ipipe.h
@@ -285,14 +285,15 @@ void __ipipe_grab_irq(int irq, struct pt_regs *regs);
 
 void __ipipe_exit_irq(struct pt_regs *regs);
 
-void __ipipe_handle_irq(int irq, struct pt_regs *regs);
+#define IPIPE_IRQF_NOACK    0x1
+#define IPIPE_IRQF_NOSYNC   0x2
+
+void __ipipe_handle_irq(int irq, int flags);
 
 static inline void ipipe_handle_chained_irq(unsigned int irq)
 {
-	struct pt_regs regs;    /* dummy */
-
 	ipipe_trace_irq_entry(irq);
-	__ipipe_handle_irq(irq, &regs);
+	__ipipe_handle_irq(irq, IPIPE_IRQF_NOSYNC);
 	ipipe_trace_irq_exit(irq);
 }
 
diff --git a/arch/arm/include/asm/smp_twd.h b/arch/arm/include/asm/smp_twd.h
index b49c8ac..f655b27 100644
--- a/arch/arm/include/asm/smp_twd.h
+++ b/arch/arm/include/asm/smp_twd.h
@@ -55,7 +55,7 @@ extern void __iomem *twd_base;
 #define __ipipe_mach_relay_ipi(ipi, thiscpu)	\
 	({					\
 		(void)(thiscpu);		\
-		__ipipe_handle_irq(ipi, NULL);	\
+		__ipipe_handle_irq(ipi, IPIPE_IRQF_NOACK);	\
 	})
 
 struct irq_desc;
diff --git a/arch/arm/kernel/ipipe.c b/arch/arm/kernel/ipipe.c
index f2f618c..ff93fa9 100644
--- a/arch/arm/kernel/ipipe.c
+++ b/arch/arm/kernel/ipipe.c
@@ -254,7 +254,7 @@ int ipipe_trigger_irq(unsigned irq)
 		return -EINVAL;
 #endif
 	local_irq_save_hw(flags);
-	__ipipe_handle_irq(irq, NULL);
+	__ipipe_handle_irq(irq, IPIPE_IRQF_NOACK);
 	local_irq_restore_hw(flags);
 
 	return 1;
@@ -462,7 +462,7 @@ out:
  * interrupt protection log is maintained here for each domain. Hw
  * interrupts are off on entry.
  */
-void __ipipe_handle_irq(int irq, struct pt_regs *regs)
+void __ipipe_handle_irq(int irq, int flags)
 {
 	struct ipipe_domain *this_domain, *next_domain;
 	struct list_head *head, *pos;
@@ -470,7 +470,7 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
 	int m_ack;
 
 	/* Software-triggered IRQs do not need any ack. */
-	m_ack = (regs == NULL);
+	m_ack = (flags & IPIPE_IRQF_NOACK) != 0;
 
 #ifdef CONFIG_IPIPE_DEBUG
 	if (unlikely(irq >= IPIPE_NR_IRQS) ||
@@ -490,7 +490,11 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
 				desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
 				next_domain->irqs[irq].acknowledge(irq, desc);
 			}
-			__ipipe_dispatch_wired(next_domain, irq);
+			if ((flags & IPIPE_IRQF_NOSYNC) == 0)
+				__ipipe_dispatch_wired(next_domain, irq);
+			else
+				__ipipe_set_irq_pending(next_domain, irq);
+
 			return;
 		}
 	}
@@ -604,7 +608,7 @@ asmlinkage void __ipipe_grab_irq(int irq, struct pt_regs *regs)
 	ipipe_trace_begin(regs->ARM_ORIG_r0);
 #endif
 
-	__ipipe_handle_irq(irq, regs);
+	__ipipe_handle_irq(irq, 0);
 
 #ifdef CONFIG_IPIPE_TRACE_IRQSOFF
 	ipipe_trace_end(regs->ARM_ORIG_r0);
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 9ea207d..af02caa 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -527,7 +527,7 @@ __ipipe_grab_ipi(unsigned svc, struct pt_regs *regs) /* hw IRQs off */
 		__ipipe_do_vnmi(IPIPE_SERVICE_VNMI, NULL);
 	else if ((1 << svc) & IPI_IPIPE_ALL) {
 		virq = svc - IPI_IPIPE_CRITICAL + IPIPE_FIRST_IPI;
-		__ipipe_handle_irq(virq, NULL);
+		__ipipe_handle_irq(virq, IPIPE_IRQF_NOACK);
 	} else
 		__ipipe_mach_relay_ipi(svc, cpu);
 

-- 
                                                                Gilles.

[Xenomai] RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Jean-Pascal JULIEN]

Hi,

I have patch with the http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c. for avoid my IRQ latency.
Unfortunately, I have the same result. 

Between the new patch and the patch of the next week this piece of code is missing.
@@ -483,7 +483,8 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
        else {
                head = __ipipe_pipeline.next;
                next_domain = list_entry(head, struct ipipe_domain, p_link);
-               if (likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) {
+               if (!(flags & IPIPE_IRQF_NOSYNC)
+                   && likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) {
                        if (!m_ack && next_domain->irqs[irq].acknowledge) {
                                desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
                                next_domain->irqs[irq].acknowledge(irq, desc);

It is normally?

JP
________________________________________
De : xenomai-bounces@xenomai.org [xenomai-bounces@xenomai.org] de la part de Gilles Chanteperdrix [gilles.chanteperdrix@xenomai.org]
Date d'envoi : mardi 22 mai 2012 10:07
À : Mitchell Tasman
Cc : xenomai@xenomai.org
Objet : Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

On 05/22/2012 02:31 AM, Mitchell Tasman wrote:
> Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a
> mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just
> application) of the patch that Gilles referenced in the "IRQ latency"
> thread last week:
>
> http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c

Please try the following patch instead:

diff --git a/arch/arm/include/asm/ipipe.h b/arch/arm/include/asm/ipipe.h
index 9b17c06..a510b09 100644
--- a/arch/arm/include/asm/ipipe.h
+++ b/arch/arm/include/asm/ipipe.h
@@ -285,14 +285,15 @@ void __ipipe_grab_irq(int irq, struct pt_regs *regs);

 void __ipipe_exit_irq(struct pt_regs *regs);

-void __ipipe_handle_irq(int irq, struct pt_regs *regs);
+#define IPIPE_IRQF_NOACK    0x1
+#define IPIPE_IRQF_NOSYNC   0x2
+
+void __ipipe_handle_irq(int irq, int flags);

 static inline void ipipe_handle_chained_irq(unsigned int irq)
 {
-       struct pt_regs regs;    /* dummy */
-
        ipipe_trace_irq_entry(irq);
-       __ipipe_handle_irq(irq, &regs);
+       __ipipe_handle_irq(irq, IPIPE_IRQF_NOSYNC);
        ipipe_trace_irq_exit(irq);
 }

diff --git a/arch/arm/include/asm/smp_twd.h b/arch/arm/include/asm/smp_twd.h
index b49c8ac..f655b27 100644
--- a/arch/arm/include/asm/smp_twd.h
+++ b/arch/arm/include/asm/smp_twd.h
@@ -55,7 +55,7 @@ extern void __iomem *twd_base;
 #define __ipipe_mach_relay_ipi(ipi, thiscpu)   \
        ({                                      \
                (void)(thiscpu);                \
-               __ipipe_handle_irq(ipi, NULL);  \
+               __ipipe_handle_irq(ipi, IPIPE_IRQF_NOACK);      \
        })

 struct irq_desc;
diff --git a/arch/arm/kernel/ipipe.c b/arch/arm/kernel/ipipe.c
index f2f618c..ff93fa9 100644
--- a/arch/arm/kernel/ipipe.c
+++ b/arch/arm/kernel/ipipe.c
@@ -254,7 +254,7 @@ int ipipe_trigger_irq(unsigned irq)
                return -EINVAL;
 #endif
        local_irq_save_hw(flags);
-       __ipipe_handle_irq(irq, NULL);
+       __ipipe_handle_irq(irq, IPIPE_IRQF_NOACK);
        local_irq_restore_hw(flags);

        return 1;
@@ -462,7 +462,7 @@ out:
  * interrupt protection log is maintained here for each domain. Hw
  * interrupts are off on entry.
  */
-void __ipipe_handle_irq(int irq, struct pt_regs *regs)
+void __ipipe_handle_irq(int irq, int flags)
 {
        struct ipipe_domain *this_domain, *next_domain;
        struct list_head *head, *pos;
@@ -470,7 +470,7 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
        int m_ack;

        /* Software-triggered IRQs do not need any ack. */
-       m_ack = (regs == NULL);
+       m_ack = (flags & IPIPE_IRQF_NOACK) != 0;

 #ifdef CONFIG_IPIPE_DEBUG
        if (unlikely(irq >= IPIPE_NR_IRQS) ||
@@ -490,7 +490,11 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
                                desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
                                next_domain->irqs[irq].acknowledge(irq, desc);
                        }
-                       __ipipe_dispatch_wired(next_domain, irq);
+                       if ((flags & IPIPE_IRQF_NOSYNC) == 0)
+                               __ipipe_dispatch_wired(next_domain, irq);
+                       else
+                               __ipipe_set_irq_pending(next_domain, irq);
+
                        return;
                }
        }
@@ -604,7 +608,7 @@ asmlinkage void __ipipe_grab_irq(int irq, struct pt_regs *regs)
        ipipe_trace_begin(regs->ARM_ORIG_r0);
 #endif

-       __ipipe_handle_irq(irq, regs);
+       __ipipe_handle_irq(irq, 0);

 #ifdef CONFIG_IPIPE_TRACE_IRQSOFF
        ipipe_trace_end(regs->ARM_ORIG_r0);
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 9ea207d..af02caa 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -527,7 +527,7 @@ __ipipe_grab_ipi(unsigned svc, struct pt_regs *regs) /* hw IRQs off */
                __ipipe_do_vnmi(IPIPE_SERVICE_VNMI, NULL);
        else if ((1 << svc) & IPI_IPIPE_ALL) {
                virq = svc - IPI_IPIPE_CRITICAL + IPIPE_FIRST_IPI;
-               __ipipe_handle_irq(virq, NULL);
+               __ipipe_handle_irq(virq, IPIPE_IRQF_NOACK);
        } else
                __ipipe_mach_relay_ipi(svc, cpu);


--
                                                                Gilles.

_______________________________________________
Xenomai mailing list
Xenomai@xenomai.org
http://www.xenomai.org/mailman/listinfo/xenomai

Re: [Xenomai] RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 10:34 AM, Jean-Pascal JULIEN wrote:
> Hi,
> 
> I have patch with the http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c. for avoid my IRQ latency.
> Unfortunately, I have the same result. 
> 
> Between the new patch and the patch of the next week this piece of code is missing.
> @@ -483,7 +483,8 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
>         else {
>                 head = __ipipe_pipeline.next;
>                 next_domain = list_entry(head, struct ipipe_domain, p_link);
> -               if (likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) {
> +               if (!(flags & IPIPE_IRQF_NOSYNC)
> +                   && likely(test_bit(IPIPE_WIRED_FLAG, &next_domain->irqs[irq].control))) {
>                         if (!m_ack && next_domain->irqs[irq].acknowledge) {
>                                 desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
>                                 next_domain->irqs[irq].acknowledge(irq, desc);
> 
> It is normally?

Yes, as I explained in the two mails you are replying to, you should try
the patch I posted.


-- 
					    Gilles.

[Xenomai] RE : RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Jean-Pascal JULIEN]

Gilles,

I have tested the new patch but i have the same result.
i have a logic analyzer who monitored the irq and the answer to the irq.
Below two  acquisitions with response latency higher than 1ms :

       ******       ******       ******       ******       ******       ******
******       ******       ******       ******       ******       ******       ****** the irq input on GPIO (2KHz)

       *****************        ***********               ************                first acquisition
******                      ******              ************               *********** the toggle of the output GPIO on IRQ


       ***********************        ***********               ************        second acquisition
******                             ******              ************               ****** the toggle of the output GPIO on IRQ

My irq on the linux space is : (cat /proc/interrupts).
         CPU0
  7:       6005        INTC  TWL4030-PIH
 11:          0        INTC  prcm
 12:      84772        INTC  DMA
 25:          4        INTC  OMAP DSS
 38:     589982        INTC  gp timer
 56:     138661        INTC  omap_i2c
 61:       1015        INTC  omap_i2c
 72:          3        INTC  serial idle
 73:          0        INTC  serial idle
 74:        280        INTC  serial idle, OMAP UART2
 77:          0        INTC  ehci_hcd:usb2
 83:      31139        INTC  mmc0
 86:         26        INTC  mmc1
 92:          0        INTC  musb-hdrc
 93:          0        INTC  musb-hdrc
336:     291209        GPIO  eth0
371:          0     twl4030  twl4030_madc
378:          0     twl4030  twl4030_usb
379:       6005     twl4030  rtc0

My irq in the xenomai space are: (cat /proc/xenomai/irq).
IRQ         CPU0
 38:      603343         [timer]
307:     2956569         XENO_IRQ_RECEIVE
418:           0         [virtual]


What kind of test do you need, for help me to solve my problem?

Thanks.

JP

Re: [Xenomai] RE : RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 01:52 PM, Jean-Pascal JULIEN wrote:
> Gilles,
> 
> I have tested the new patch but i have the same result.
> i have a logic analyzer who monitored the irq and the answer to the irq.
> Below two  acquisitions with response latency higher than 1ms :
> 
>        ******       ******       ******       ******       ******       ******
> ******       ******       ******       ******       ******       ******       ****** the irq input on GPIO (2KHz)
> 
>        *****************        ***********               ************                first acquisition
> ******                      ******              ************               *********** the toggle of the output GPIO on IRQ
> 
> 
>        ***********************        ***********               ************        second acquisition
> ******                             ******              ************               ****** the toggle of the output GPIO on IRQ
> 
> My irq on the linux space is : (cat /proc/interrupts).
>          CPU0
>   7:       6005        INTC  TWL4030-PIH
>  11:          0        INTC  prcm
>  12:      84772        INTC  DMA
>  25:          4        INTC  OMAP DSS
>  38:     589982        INTC  gp timer
>  56:     138661        INTC  omap_i2c
>  61:       1015        INTC  omap_i2c
>  72:          3        INTC  serial idle
>  73:          0        INTC  serial idle
>  74:        280        INTC  serial idle, OMAP UART2
>  77:          0        INTC  ehci_hcd:usb2
>  83:      31139        INTC  mmc0
>  86:         26        INTC  mmc1
>  92:          0        INTC  musb-hdrc
>  93:          0        INTC  musb-hdrc
> 336:     291209        GPIO  eth0
> 371:          0     twl4030  twl4030_madc
> 378:          0     twl4030  twl4030_usb
> 379:       6005     twl4030  rtc0
> 
> My irq in the xenomai space are: (cat /proc/xenomai/irq).
> IRQ         CPU0
>  38:      603343         [timer]
> 307:     2956569         XENO_IRQ_RECEIVE
> 418:           0         [virtual]
> 
> 
> What kind of test do you need, for help me to solve my problem?

Please try the following (completely untested) patch. If it still
does not, try and do what Mitchell did: use the I-pipe tracer, and
show us the trace corresponding to the bug you observe.

diff --git a/arch/arm/plat-omap/gpio.c b/arch/arm/plat-omap/gpio.c
index 25c17a3..3e5a88d 100644
--- a/arch/arm/plat-omap/gpio.c
+++ b/arch/arm/plat-omap/gpio.c
@@ -1221,7 +1221,7 @@ static void omap_gpio_free(struct gpio_chip *chip, unsigned offset)
 	spin_unlock_irqrestore(&bank->lock, flags);
 }
 
-static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
+static void gpio_demux_inner(struct gpio_bank *bank, u32 isr)
 {
 	unsigned int gpio_irq, gpio_index;
 
@@ -1230,13 +1230,6 @@ static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
 		if (!(isr & 1))
 			continue;
 
-#ifdef CONFIG_IPIPE
-		if (!nonroot) {
-			local_irq_enable_hw();
-			local_irq_disable_hw();
-		}
-#endif /* CONFIG_IPIPE */
-
 #ifdef CONFIG_ARCH_OMAP1
 		gpio_index = get_gpio_index(irq_to_gpio(gpio_irq));
 
@@ -1272,11 +1265,7 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
 	u32 retrigger = 0;
 	int unmasked = 0;
 
-#ifndef CONFIG_IPIPE
 	desc->irq_data.chip->irq_ack(&desc->irq_data);
-#else /* CONFIG_IPIPE */
-	desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
-#endif /* CONFIG_IPIPE */
 
 	bank = get_irq_data(irq);
 #ifdef CONFIG_ARCH_OMAP1
@@ -1312,13 +1301,6 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
 		u32 isr_saved, level_mask = 0;
 		u32 enabled;
 
-#ifdef CONFIG_IPIPE
-		if (!bank->nonroot_gpios) {
-			local_irq_enable_hw();
-			local_irq_disable_hw();
-		}
-#endif /* CONFIG_IPIPE */
-
 		enabled = _get_gpio_irqbank_mask(bank);
 		isr_saved = isr = __raw_readl(isr_reg) & enabled;
 
@@ -1336,27 +1318,19 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
 		_clear_gpio_irqbank(bank, isr_saved & ~level_mask);
 		_enable_gpio_irqbank(bank, isr_saved & ~level_mask, 1);
 
-#ifndef CONFIG_IPIPE
 		/* if there is only edge sensitive GPIO pin interrupts
 		configured, we could unmask GPIO bank interrupt immediately */
 		if (!level_mask && !unmasked) {
 			unmasked = 1;
 			desc->irq_data.chip->irq_unmask(&desc->irq_data);
 		}
-#endif /* !CONFIG_IPIPE */
 
 		isr |= retrigger;
 		retrigger = 0;
 		if (!isr)
 			break;
 
-#ifdef CONFIG_IPIPE
-		if (bank->nonroot_gpios)
-			gpio_demux_inner(bank, isr & bank->nonroot_gpios, 1);
-		gpio_demux_inner(bank, isr & ~bank->nonroot_gpios, 0);
-#else /* !CONFIG_IPIPE */
-		gpio_demux_inner(bank, isr, 0);
-#endif /* !CONFIG_IPIPE */
+		gpio_demux_inner(bank, isr);
 
 	}
 	/* if bank has any level sensitive GPIO pin interrupt


-- 
					    Gilles.

[Xenomai] RE : RE : RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Jean-Pascal JULIEN]

Gilles,

In the adeos-ipipe-2.6.38.2-arm-1.18-04.patch this piece of code is not present :

>@@ -1272,11 +1265,7 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
>        u32 retrigger = 0;
>        int unmasked = 0;
>
>#ifndef CONFIG_IPIPE
>        desc->irq_data.chip->irq_ack(&desc->irq_data);
>#else /* CONFIG_IPIPE */
>       desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
>#endif /* CONFIG_IPIPE */
>
>        bank = get_irq_data(irq);
> #ifdef CONFIG_ARCH_OMAP1

I have :
	int unmasked = 0;

	desc->irq_data.chip->irq_ack(&desc->irq_data);

	bank = get_irq_data(irq);
#ifdef CONFIG_ARCH_OMAP1


In the adeos-ipipe-2.6.38.2-arm-1.18-04.patch this piece of code is not present :

>@@ -1336,27 +1318,19 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
>                _clear_gpio_irqbank(bank, isr_saved & ~level_mask);
>                _enable_gpio_irqbank(bank, isr_saved & ~level_mask, 1);
>
>#ifndef CONFIG_IPIPE
>                /* if there is only edge sensitive GPIO pin interrupts
>                configured, we could unmask GPIO bank interrupt immediately */
>                if (!level_mask && !unmasked) {
>                        unmasked = 1;
>                        desc->irq_data.chip->irq_unmask(&desc->irq_data);
>                }
>#endif /* !CONFIG_IPIPE */
>
>     isr |= retrigger;


I have : 
_clear_gpio_irqbank(bank, isr_saved & ~level_mask);
		_enable_gpio_irqbank(bank, isr_saved & ~level_mask, 1);

		/* if there is only edge sensitive GPIO pin interrupts
		configured, we could unmask GPIO bank interrupt immediately */
		if (!level_mask && !unmasked) {
			unmasked = 1;
			desc->irq_data.chip->irq_unmask(&desc->irq_data);
		}

		isr |= retrigger;


JP
________________________________________
De : Gilles Chanteperdrix [gilles.chanteperdrix@xenomai.org]
Date d'envoi : mardi 22 mai 2012 14:38
À : Jean-Pascal JULIEN
Cc : xenomai@xenomai.org
Objet : Re: RE : RE : [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

On 05/22/2012 01:52 PM, Jean-Pascal JULIEN wrote:
> Gilles,
>
> I have tested the new patch but i have the same result.
> i have a logic analyzer who monitored the irq and the answer to the irq.
> Below two  acquisitions with response latency higher than 1ms :
>
>        ******       ******       ******       ******       ******       ******
> ******       ******       ******       ******       ******       ******       ****** the irq input on GPIO (2KHz)
>
>        *****************        ***********               ************                first acquisition
> ******                      ******              ************               *********** the toggle of the output GPIO on IRQ
>
>
>        ***********************        ***********               ************        second acquisition
> ******                             ******              ************               ****** the toggle of the output GPIO on IRQ
>
> My irq on the linux space is : (cat /proc/interrupts).
>          CPU0
>   7:       6005        INTC  TWL4030-PIH
>  11:          0        INTC  prcm
>  12:      84772        INTC  DMA
>  25:          4        INTC  OMAP DSS
>  38:     589982        INTC  gp timer
>  56:     138661        INTC  omap_i2c
>  61:       1015        INTC  omap_i2c
>  72:          3        INTC  serial idle
>  73:          0        INTC  serial idle
>  74:        280        INTC  serial idle, OMAP UART2
>  77:          0        INTC  ehci_hcd:usb2
>  83:      31139        INTC  mmc0
>  86:         26        INTC  mmc1
>  92:          0        INTC  musb-hdrc
>  93:          0        INTC  musb-hdrc
> 336:     291209        GPIO  eth0
> 371:          0     twl4030  twl4030_madc
> 378:          0     twl4030  twl4030_usb
> 379:       6005     twl4030  rtc0
>
> My irq in the xenomai space are: (cat /proc/xenomai/irq).
> IRQ         CPU0
>  38:      603343         [timer]
> 307:     2956569         XENO_IRQ_RECEIVE
> 418:           0         [virtual]
>
>
> What kind of test do you need, for help me to solve my problem?

Please try the following (completely untested) patch. If it still
does not, try and do what Mitchell did: use the I-pipe tracer, and
show us the trace corresponding to the bug you observe.

diff --git a/arch/arm/plat-omap/gpio.c b/arch/arm/plat-omap/gpio.c
index 25c17a3..3e5a88d 100644
--- a/arch/arm/plat-omap/gpio.c
+++ b/arch/arm/plat-omap/gpio.c
@@ -1221,7 +1221,7 @@ static void omap_gpio_free(struct gpio_chip *chip, unsigned offset)
        spin_unlock_irqrestore(&bank->lock, flags);
 }

-static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
+static void gpio_demux_inner(struct gpio_bank *bank, u32 isr)
 {
        unsigned int gpio_irq, gpio_index;

@@ -1230,13 +1230,6 @@ static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
                if (!(isr & 1))
                        continue;

-#ifdef CONFIG_IPIPE
-               if (!nonroot) {
-                       local_irq_enable_hw();
-                       local_irq_disable_hw();
-               }
-#endif /* CONFIG_IPIPE */
-
 #ifdef CONFIG_ARCH_OMAP1
                gpio_index = get_gpio_index(irq_to_gpio(gpio_irq));


@@ -1312,13 +1301,6 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
                u32 isr_saved, level_mask = 0;
                u32 enabled;

-#ifdef CONFIG_IPIPE
-               if (!bank->nonroot_gpios) {
-                       local_irq_enable_hw();
-                       local_irq_disable_hw();
-               }
-#endif /* CONFIG_IPIPE */
-
                enabled = _get_gpio_irqbank_mask(bank);
                isr_saved = isr = __raw_readl(isr_reg) & enabled;

@@ -1336,27 +1318,19 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
                _clear_gpio_irqbank(bank, isr_saved & ~level_mask);
                _enable_gpio_irqbank(bank, isr_saved & ~level_mask, 1);

-#ifndef CONFIG_IPIPE
                /* if there is only edge sensitive GPIO pin interrupts
                configured, we could unmask GPIO bank interrupt immediately */
                if (!level_mask && !unmasked) {
                        unmasked = 1;
                        desc->irq_data.chip->irq_unmask(&desc->irq_data);
                }
-#endif /* !CONFIG_IPIPE */

                isr |= retrigger;
                retrigger = 0;
                if (!isr)
                        break;

-#ifdef CONFIG_IPIPE
-               if (bank->nonroot_gpios)
-                       gpio_demux_inner(bank, isr & bank->nonroot_gpios, 1);
-               gpio_demux_inner(bank, isr & ~bank->nonroot_gpios, 0);
-#else /* !CONFIG_IPIPE */
-               gpio_demux_inner(bank, isr, 0);
-#endif /* !CONFIG_IPIPE */
+               gpio_demux_inner(bank, isr);

        }
        /* if bank has any level sensitive GPIO pin interrupt


--
                                            Gilles.

Re: [Xenomai] RE : RE : RE : A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 03:23 PM, Jean-Pascal JULIEN wrote:
> Gilles,
> 
> In the adeos-ipipe-2.6.38.2-arm-1.18-04.patch this piece of code is not present :

Just look at what the patch does: it replaces what the current I-pipe
patch has, with what you already have. If you want the patch I sent to
apply correctly, please use the latest I-pipe patch.

-- 
					    Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

Gilles,

On 05/22/2012 04:07 AM, Gilles Chanteperdrix wrote:
> On 05/22/2012 02:31 AM, Mitchell Tasman wrote:
>> Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a
>> mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just
>> application) of the patch that Gilles referenced in the "IRQ latency"
>> thread last week:
>>
>> http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c
>
> Please try the following patch instead:

Thank you very much for your prompt and most helpful response.

I have applied the patch that you supplied, and find a dramatic 
reduction in the maximum latency experienced in servicing edge-triggered 
GPIO interrupts.  In testing so far today with the new patch, I have not 
encountered a single instance of the sort of extreme latency that 
prompted my query to the mailing list.

Might you consider applying the new patch to I-Pipe for ARM 2.6.38.8, 
and publishing a new revision of that patch set?  This could be of 
significant benefit to those members of the Xenomai community that are 
presently using a 2.6.38.8 kernel on ARM platforms.

You might also consider addressing/replacing the original "ipipe/arm: 
defer chained interrupts handling" patch that was applied to I-Pipe for 
ARM 3.0.13, if the new patch is relevant to that kernel as well.

Thanks again and Best Regards,
Mitch

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/22/2012 11:22 PM, Mitchell Tasman wrote:
> Gilles,
> 
> On 05/22/2012 04:07 AM, Gilles Chanteperdrix wrote:
>> On 05/22/2012 02:31 AM, Mitchell Tasman wrote:
>>> Hi.  I'm running an OMAP DM3725-based board with a 2.6.38.8 kernel, a
>>> mid-May 2012 git snapshot of xenomai-2.6, and a backport (really just
>>> application) of the patch that Gilles referenced in the "IRQ latency"
>>> thread last week:
>>>
>>> http://git.xenomai.org/?p=ipipe-gch.git;a=commit;h=81bfc05c4716b76e79f5e486baf4c52015a3b92c
>>
>> Please try the following patch instead:
> 
> Thank you very much for your prompt and most helpful response.
> 
> I have applied the patch that you supplied, and find a dramatic 
> reduction in the maximum latency experienced in servicing edge-triggered 
> GPIO interrupts.  In testing so far today with the new patch, I have not 
> encountered a single instance of the sort of extreme latency that 
> prompted my query to the mailing list.
> 
> Might you consider applying the new patch to I-Pipe for ARM 2.6.38.8, 
> and publishing a new revision of that patch set?  This could be of 
> significant benefit to those members of the Xenomai community that are 
> presently using a 2.6.38.8 kernel on ARM platforms.
> 
> You might also consider addressing/replacing the original "ipipe/arm: 
> defer chained interrupts handling" patch that was applied to I-Pipe for 
> ARM 3.0.13, if the new patch is relevant to that kernel as well.
> 
> Thanks again and Best Regards,

Yes, that's the plan. I am just waiting for Jean-Pascal's result to
confirm that everything is fine.

Regards.

-- 
                                                                Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

Gilles,

On 05/22/2012 05:30 PM, Gilles Chanteperdrix wrote:
> On 05/22/2012 11:22 PM, Mitchell Tasman wrote:
>> I have applied the patch that you supplied, and find a dramatic
>> reduction in the maximum latency experienced in servicing edge-triggered
>> GPIO interrupts.  In testing so far today with the new patch, I have not
>> encountered a single instance of the sort of extreme latency that
>> prompted my query to the mailing list.
>>
>> Might you consider applying the new patch to I-Pipe for ARM 2.6.38.8,
>> and publishing a new revision of that patch set?  This could be of
>> significant benefit to those members of the Xenomai community that are
>> presently using a 2.6.38.8 kernel on ARM platforms.
>>
>> You might also consider addressing/replacing the original "ipipe/arm:
>> defer chained interrupts handling" patch that was applied to I-Pipe for
>> ARM 3.0.13, if the new patch is relevant to that kernel as well.

> Yes, that's the plan. I am just waiting for Jean-Pascal's result to
> confirm that everything is fine.

I thought I'd report that testing of 
adeos-ipipe-2.6.38.8-arm-1.18-06.patch plus the revised "IRQ Chaining" 
patch that you e-mailed continues to show a dramatic reduction in the 
maximum latency in servicing edge-triggered GPIO interrupts.  Good news 
as well is that the load on the Linux domain (e.g. dohell vs. idle) 
appears to have much less effect on the latency in servicing such 
interrupts than was previously the case.

I noticed that Jean-Pascal reported success as well, on 23 May.

Thank you once again for your excellent support, and I very much look 
forward to the updated I-Pipe for ARM 2.6.38.8 patch set.

Best Regards,
Mitch

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/25/2012 11:18 AM, Mitchell Tasman wrote:
> Gilles,
> 
> On 05/22/2012 05:30 PM, Gilles Chanteperdrix wrote:
>> On 05/22/2012 11:22 PM, Mitchell Tasman wrote:
>>> I have applied the patch that you supplied, and find a dramatic
>>> reduction in the maximum latency experienced in servicing edge-triggered
>>> GPIO interrupts.  In testing so far today with the new patch, I have not
>>> encountered a single instance of the sort of extreme latency that
>>> prompted my query to the mailing list.
>>>
>>> Might you consider applying the new patch to I-Pipe for ARM 2.6.38.8,
>>> and publishing a new revision of that patch set?  This could be of
>>> significant benefit to those members of the Xenomai community that are
>>> presently using a 2.6.38.8 kernel on ARM platforms.
>>>
>>> You might also consider addressing/replacing the original "ipipe/arm:
>>> defer chained interrupts handling" patch that was applied to I-Pipe for
>>> ARM 3.0.13, if the new patch is relevant to that kernel as well.
> 
>> Yes, that's the plan. I am just waiting for Jean-Pascal's result to
>> confirm that everything is fine.
> 
> I thought I'd report that testing of 
> adeos-ipipe-2.6.38.8-arm-1.18-06.patch plus the revised "IRQ Chaining" 
> patch that you e-mailed continues to show a dramatic reduction in the 
> maximum latency in servicing edge-triggered GPIO interrupts.  Good news 
> as well is that the load on the Linux domain (e.g. dohell vs. idle) 
> appears to have much less effect on the latency in servicing such 
> interrupts than was previously the case.
> 
> I noticed that Jean-Pascal reported success as well, on 23 May.
> 
> Thank you once again for your excellent support, and I very much look 
> forward to the updated I-Pipe for ARM 2.6.38.8 patch set.

You are welcome. The patch should be done during the week-end.

-- 
					    Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

Gilles,

On 05/25/2012 08:20 AM, Gilles Chanteperdrix wrote:
> On 05/25/2012 11:18 AM, Mitchell Tasman wrote:

>> Thank you once again for your excellent support, and I very much look
>> forward to the updated I-Pipe for ARM 2.6.38.8 patch set.
>
> You are welcome. The patch should be done during the week-end.

Hi.  I noticed that you made some commits on the for-ipipe-2.6.38-arm 
branch of your ipipe-gch.git repository, and decided to take a quick 
look, leading to a question.

In arch/arm/plat-omap/gpio.c, I found that you reverted-out some changes 
relating to earlier fixes to handling of GPIO interrupts, bringing the 
source closer to the vanilla linux 2.6.38.8 version.

One change that had been made along the way on the for-ipipe-2.6.38-arm 
branch was to add a gpio_mask_ack_irq() function, and invoke it like so 
from gpio_irq_handler(), instead of invoking gpio_ack_irq():

#ifndef CONFIG_IPIPE
       desc->irq_data.chip->irq_ack(&desc->irq_data);
#else /* CONFIG_IPIPE */
       desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
#endif /* CONFIG_IPIPE */

With the latest commits, the gpio_mask_ack_irq() function remains, but 
it is no longer invoked, with gpio_ack_irq() being called 
unconditionally instead, as with the vanilla 2.6.38.8 source.

Looking at I-Pipe support for 3.0.13, I found that the inline function 
chained_irq_enter() is called at the equivalent place in 
gpio_irq_handler(), and that function appears to invoke 
gpio_mask_ack_irq() when available (which it is for OMAP in your tree), 
and otherwise calls gpio_ack_irq().

I just wanted to double-check with you that the vanilla 2.6.38.8 source 
is indeed correct (or at least harmless) in its unconditional calling of 
gpio_ack_irq() from gpio_irq_handler(), and that the apparent difference 
in behavior between the 3.0.13 and 2.6.38.8 I-Pipe patch sets is fine as 
well.

Thank you very much for updating I-Pipe support for ARM 2.6.38.8.

Best Regards,
Mitch

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 05/29/2012 09:20 AM, Mitchell Tasman wrote:
> Gilles,
> 
> On 05/25/2012 08:20 AM, Gilles Chanteperdrix wrote:
>> On 05/25/2012 11:18 AM, Mitchell Tasman wrote:
> 
>>> Thank you once again for your excellent support, and I very much look
>>> forward to the updated I-Pipe for ARM 2.6.38.8 patch set.
>>
>> You are welcome. The patch should be done during the week-end.
> 
> Hi.  I noticed that you made some commits on the for-ipipe-2.6.38-arm 
> branch of your ipipe-gch.git repository, and decided to take a quick 
> look, leading to a question.
> 
> In arch/arm/plat-omap/gpio.c, I found that you reverted-out some changes 
> relating to earlier fixes to handling of GPIO interrupts, bringing the 
> source closer to the vanilla linux 2.6.38.8 version.
> 
> One change that had been made along the way on the for-ipipe-2.6.38-arm 
> branch was to add a gpio_mask_ack_irq() function, and invoke it like so 
> from gpio_irq_handler(), instead of invoking gpio_ack_irq():
> 
> #ifndef CONFIG_IPIPE
>        desc->irq_data.chip->irq_ack(&desc->irq_data);
> #else /* CONFIG_IPIPE */
>        desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
> #endif /* CONFIG_IPIPE */
> 
> With the latest commits, the gpio_mask_ack_irq() function remains, but 
> it is no longer invoked, with gpio_ack_irq() being called 
> unconditionally instead, as with the vanilla 2.6.38.8 source.

desc->irq_data.chip->irq_mask_ack is the "mask_ack" method of the parent
irq, not gpio_mask_ack_irq. gpio_mask_ack_irq is called normally when
acknowledging the gpio irq.

> 
> Looking at I-Pipe support for 3.0.13, I found that the inline function 
> chained_irq_enter() is called at the equivalent place in 
> gpio_irq_handler(), and that function appears to invoke 
> gpio_mask_ack_irq() when available (which it is for OMAP in your tree), 
> and otherwise calls gpio_ack_irq().
> 
> I just wanted to double-check with you that the vanilla 2.6.38.8 source 
> is indeed correct (or at least harmless) in its unconditional calling of 
> gpio_ack_irq() from gpio_irq_handler(), and that the apparent difference 
> in behavior between the 3.0.13 and 2.6.38.8 I-Pipe patch sets is fine as 
> well.

Everything happens with irqs off, so masking is not strictly necessary I
guess.

-- 
                                                                Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

On 05/29/2012 04:06 AM, Gilles Chanteperdrix wrote:
> On 05/29/2012 09:20 AM, Mitchell Tasman wrote:
>> One change that had been made along the way on the for-ipipe-2.6.38-arm
>> branch was to add a gpio_mask_ack_irq() function, and invoke it like so
>> from gpio_irq_handler(), instead of invoking gpio_ack_irq():
>>
>> #ifndef CONFIG_IPIPE
>>         desc->irq_data.chip->irq_ack(&desc->irq_data);
>> #else /* CONFIG_IPIPE */
>>         desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
>> #endif /* CONFIG_IPIPE */
>>
>> With the latest commits, the gpio_mask_ack_irq() function remains, but
>> it is no longer invoked, with gpio_ack_irq() being called
>> unconditionally instead, as with the vanilla 2.6.38.8 source.

> desc->irq_data.chip->irq_mask_ack is the "mask_ack" method of the parent
> irq, not gpio_mask_ack_irq. gpio_mask_ack_irq is called normally when
> acknowledging the gpio irq.

Sorry about my confusion.  Assuming that I'm looking at the correct 
vector this time, in arch/arm/mach-omap2/irq.c for 2.6.38.8 as patched 
for I-Pipe:

> static struct irq_chip omap_irq_chip = {
>         .name           = "INTC",
>         .irq_ack        = omap_mask_ack_irq,
>         .irq_mask       = omap_mask_irq,
>         .irq_mask_ack   = omap_mask_ack_irq,
>         .irq_unmask     = omap_unmask_irq,
> };

both halves of the former #ifndef block quoted above appear equivalent, 
and the parent irq is indeed masked.  Likewise, for 3.0.13 with 
CONFIG_IPIPE defined, .irq_ack and .irq_mask_ack both appear to mask the 
parent irq.

Regarding your recent commits to ipipe-gch.git, I'm imagining that once 
the dust settles, the next step will be to cut new I-Pipe for ARM patch 
sets and update xenomai-2.6.git accordingly.  I'm planning an additional 
round of testing soon thereafter.

Thanks much,
Mitch

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Eric Eric]

Hi Gilles, I believe the problems I was having several months ago with
erratic timing on the Beagleboard when running dohell may be related
to this.  I'm wondering if there are still plans to formally wrap the
patch into the official Xenomai tree?

Thanks!

- Eric

On Fri, May 25, 2012 at 8:20 AM, Gilles Chanteperdrix
<gilles.chanteperdrix@xenomai.org> wrote:
> On 05/25/2012 11:18 AM, Mitchell Tasman wrote:
>> Gilles,
>>
>> On 05/22/2012 05:30 PM, Gilles Chanteperdrix wrote:
>>> On 05/22/2012 11:22 PM, Mitchell Tasman wrote:
>>>> I have applied the patch that you supplied, and find a dramatic
>>>> reduction in the maximum latency experienced in servicing edge-triggered
>>>> GPIO interrupts.  In testing so far today with the new patch, I have not
>>>> encountered a single instance of the sort of extreme latency that
>>>> prompted my query to the mailing list.
>>>>
>>>> Might you consider applying the new patch to I-Pipe for ARM 2.6.38.8,
>>>> and publishing a new revision of that patch set?  This could be of
>>>> significant benefit to those members of the Xenomai community that are
>>>> presently using a 2.6.38.8 kernel on ARM platforms.
>>>>
>>>> You might also consider addressing/replacing the original "ipipe/arm:
>>>> defer chained interrupts handling" patch that was applied to I-Pipe for
>>>> ARM 3.0.13, if the new patch is relevant to that kernel as well.
>>
>>> Yes, that's the plan. I am just waiting for Jean-Pascal's result to
>>> confirm that everything is fine.
>>
>> I thought I'd report that testing of
>> adeos-ipipe-2.6.38.8-arm-1.18-06.patch plus the revised "IRQ Chaining"
>> patch that you e-mailed continues to show a dramatic reduction in the
>> maximum latency in servicing edge-triggered GPIO interrupts.  Good news
>> as well is that the load on the Linux domain (e.g. dohell vs. idle)
>> appears to have much less effect on the latency in servicing such
>> interrupts than was previously the case.
>>
>> I noticed that Jean-Pascal reported success as well, on 23 May.
>>
>> Thank you once again for your excellent support, and I very much look
>> forward to the updated I-Pipe for ARM 2.6.38.8 patch set.
>
> You are welcome. The patch should be done during the week-end.
>
> --
>                                            Gilles.
>
> _______________________________________________
> Xenomai mailing list
> Xenomai@xenomai.org
> http://www.xenomai.org/mailman/listinfo/xenomai

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 06/07/2012 09:44 AM, Eric Eric wrote:
> Hi Gilles, I believe the problems I was having several months ago with
> erratic timing on the Beagleboard when running dohell may be related
> to this.  I'm wondering if there are still plans to formally wrap the
> patch into the official Xenomai tree?

Yes, the patches are not yet available in the xenomai tree, but are 
available on the gna download area:
http://download.gna.org/adeos/patches/v2.6/arm/adeos-ipipe-2.6.38.8-arm-1.18-08.patch
http://download.gna.org/adeos/patches/v3.x/arm/adeos-ipipe-3.0.13-arm-1.18-09.patch

-- 
					    Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Gilles Chanteperdrix]

On 06/07/2012 09:44 AM, Eric Eric wrote:
> Hi Gilles, I believe the problems I was having several months ago with
> erratic timing on the Beagleboard when running dohell may be related
> to this.  I'm wondering if there are still plans to formally wrap the
> patch into the official Xenomai tree?

The erratic timing you are talking about were variations in the
microsecond range. What this patch avoids are latencies in the
millisecond range involving GPIO interrupts. So, definitely no, this
patch will not change anything with your situation.

But anyway, I do not believe what you described was really a problem.
Yes, some things have more influence on latencies than other. This is
inherent to the dual kernel solution. And anyway, as I told you at the
time, if you want to know what cause the worst case latency of a latency
run, run latency -f with the I-pipe tracer enabled. If you post us a
latency trace, we may help you find if anything is wrong.

-- 
					    Gilles.

Re: [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Mitchell Tasman]

On 05/23/2012 04:22 AM, Jean-Pascal JULIEN wrote:

> The maximum latency response to the irq is less than 100us.
> I performed the test during more than 12 hours.
> I re do the first test with the internal timer which toggles a GPIO each 1ms and i have no regression.
>
> To avoid any confusion i give you the patches which were applied :

Overnight testing on my end also confirms that with Gilles' revised "IRQ 
Chaining" patch, outsized GPIO IRQ latencies are no longer present.

I do want to observe that Julien and I appear to have tested with 
slightly different I-Pipe for ARM patch sets.  I applied Gille's patch 
on top of the very latest patch set for 2.6.38.8, 
adeos-ipipe-3.0.13-arm-1.18-06.patch.

Based on Julien's diff below, it looks as if Julien  may have local 
changes to gpio_demux_inner() and to gpio_irq_handler() that don't 
exactly match the result of applying either 
adeos-ipipe-3.0.13-arm-1.18-06.patch or the earlier 
adeos-ipipe-3.0.13-arm-1.18-06.patch to a vanilla 2.6.38.8 kernel.

The good news, regardless, is that we both found Gille's revised "IRQ 
Chaining" patch to be of significant benefit.

Best Regards,
Mitch

> diff --git a/arch/arm/plat-omap/gpio.c b/arch/arm/plat-omap/gpio.c
> index 25c17a3..3e5a88d 100644
> --- a/arch/arm/plat-omap/gpio.c
> +++ b/arch/arm/plat-omap/gpio.c
> @@ -1221,7 +1221,7 @@ static void omap_gpio_free(struct gpio_chip *chip, unsigned offset)
>           spin_unlock_irqrestore(&bank->lock, flags);
>   }
>
> -static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
> +static void gpio_demux_inner(struct gpio_bank *bank, u32 isr)
>   {
>           unsigned int gpio_irq, gpio_index;
>
> @@ -1230,13 +1230,6 @@ static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
>                   if (!(isr&  1))
>                           continue;
>
> -#ifdef CONFIG_IPIPE
> -               if (!nonroot) {
> -                       local_irq_enable_hw();
> -                       local_irq_disable_hw();
> -               }
> -#endif /* CONFIG_IPIPE */
> -
>   #ifdef CONFIG_ARCH_OMAP1
>                   gpio_index = get_gpio_index(irq_to_gpio(gpio_irq));
>
> @@ -1272,11 +1265,7 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
>           u32 retrigger = 0;
>           int unmasked = 0;
>
> -#ifndef CONFIG_IPIPE
>           desc->irq_data.chip->irq_ack(&desc->irq_data);
> -#else /* CONFIG_IPIPE */
> -       desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
> -#endif /* CONFIG_IPIPE */
>
>           bank = get_irq_data(irq);
>   #ifdef CONFIG_ARCH_OMAP1
> @@ -1312,13 +1301,6 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
>                   u32 isr_saved, level_mask = 0;
>                   u32 enabled;
>
> -#ifdef CONFIG_IPIPE
> -               if (!bank->nonroot_gpios) {
> -                       local_irq_enable_hw();
> -                       local_irq_disable_hw();
> -               }
> -#endif /* CONFIG_IPIPE */
> -
>                   enabled = _get_gpio_irqbank_mask(bank);
>                   isr_saved = isr = __raw_readl(isr_reg)&  enabled;
>
> @@ -1336,27 +1318,19 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
>                   _clear_gpio_irqbank(bank, isr_saved&  ~level_mask);
>                   _enable_gpio_irqbank(bank, isr_saved&  ~level_mask, 1);
>
> -#ifndef CONFIG_IPIPE
>                   /* if there is only edge sensitive GPIO pin interrupts
>                   configured, we could unmask GPIO bank interrupt immediately */
>                   if (!level_mask&&  !unmasked) {
>                           unmasked = 1;
>                           desc->irq_data.chip->irq_unmask(&desc->irq_data);
>                   }
> -#endif /* !CONFIG_IPIPE */
>
>                   isr |= retrigger;
>                   retrigger = 0;
>                   if (!isr)
>                           break;
>
> -#ifdef CONFIG_IPIPE
> -               if (bank->nonroot_gpios)
> -                       gpio_demux_inner(bank, isr&  bank->nonroot_gpios, 1);
> -               gpio_demux_inner(bank, isr&  ~bank->nonroot_gpios, 0);
> -#else /* !CONFIG_IPIPE */
> -               gpio_demux_inner(bank, isr, 0);
> -#endif /* !CONFIG_IPIPE */
> +               gpio_demux_inner(bank, isr);
>
>           }
>           /* if bank has any level sensitive GPIO pin interrupt
>
>
> diff --git a/arch/arm/include/asm/ipipe.h b/arch/arm/include/asm/ipipe.h
> index 9b17c06..a510b09 100644
> --- a/arch/arm/include/asm/ipipe.h
> +++ b/arch/arm/include/asm/ipipe.h
> @@ -285,14 +285,15 @@ void __ipipe_grab_irq(int irq, struct pt_regs *regs);
>
>   void __ipipe_exit_irq(struct pt_regs *regs);
>
> -void __ipipe_handle_irq(int irq, struct pt_regs *regs);
> +#define IPIPE_IRQF_NOACK    0x1
> +#define IPIPE_IRQF_NOSYNC   0x2
> +
> +void __ipipe_handle_irq(int irq, int flags);
>
>   static inline void ipipe_handle_chained_irq(unsigned int irq)
>   {
> -       struct pt_regs regs;    /* dummy */
> -
>           ipipe_trace_irq_entry(irq);
> -       __ipipe_handle_irq(irq,&regs);
> +       __ipipe_handle_irq(irq, IPIPE_IRQF_NOSYNC);
>           ipipe_trace_irq_exit(irq);
>   }
>
> diff --git a/arch/arm/include/asm/smp_twd.h b/arch/arm/include/asm/smp_twd.h
> index b49c8ac..f655b27 100644
> --- a/arch/arm/include/asm/smp_twd.h
> +++ b/arch/arm/include/asm/smp_twd.h
> @@ -55,7 +55,7 @@ extern void __iomem *twd_base;
>   #define __ipipe_mach_relay_ipi(ipi, thiscpu)    \
>           ({                                      \
>                   (void)(thiscpu);                \
> -               __ipipe_handle_irq(ipi, NULL);  \
> +               __ipipe_handle_irq(ipi, IPIPE_IRQF_NOACK);      \
>           })
>
>   struct irq_desc;
> diff --git a/arch/arm/kernel/ipipe.c b/arch/arm/kernel/ipipe.c
> index f2f618c..ff93fa9 100644
> --- a/arch/arm/kernel/ipipe.c
> +++ b/arch/arm/kernel/ipipe.c
> @@ -254,the OMAP-specific implementations of g7 +254,7 @@ int ipipe_trigger_irq(unsigned irq)
>                   return -EINVAL;
>   #endif
>           local_irq_save_hw(flags);
> -       __ipipe_handle_irq(irq, NULL);
> +       __ipipe_handle_irq(irq, IPIPE_IRQF_NOACK);
>           local_irq_restore_hw(flags);
>
>           return 1;
> @@ -462,7 +462,7 @@ out:
>    * interrupt protection log is maintained here for each domain. Hw
>    * interrupts are off on entry.
>    */
> -void __ipipe_handle_irq(int irq, struct pt_regs *regs)
> +void __ipipe_handle_irq(int irq, int flags)
>   {
>           struct ipipe_domain *this_domain, *next_domain;
>           struct list_head *head, *pos;
> @@ -470,7 +470,7 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
>           int m_ack;
>
>           /* Software-triggered IRQs do not need any ack. */
> -       m_ack = (regs == NULL);
> +       m_ack = (flags&  IPIPE_IRQF_NOACK) != 0;
>
>   #ifdef CONFIG_IPIPE_DEBUG
>           if (unlikely(irq>= IPIPE_NR_IRQS) ||
> @@ -490,7 +490,11 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
>                                   desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
>                                   next_domain->irqs[irq].acknowledge(irq, desc);
>                           }
> -                       __ipipe_dispatch_wired(next_domain, irq);
> +                       if ((flags&  IPIPE_IRQF_NOSYNC) == 0)
> +                               __ipipe_dispatch_wired(next_domain, irq);
> +                       else
> +                               __ipipe_set_irq_pending(next_domain, irq);
> +
>                           return;
>                   }
>           }
> @@ -604,7 +608,7 @@ asmlinkage void __ipipe_grab_irq(int irq, struct pt_regs *regs)
>           ipipe_trace_begin(regs->ARM_ORIG_r0);
>   #endif
>
> -       __ipipe_handle_irq(irq, regs);
> +       __ipipe_handle_irq(irq, 0);
>
>   #ifdef CONFIG_IPIPE_TRACE_IRQSOFF
>           ipipe_trace_end(regs->ARM_ORIG_r0);
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 9ea207d..af02caa 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -527,7 +527,7 @@ __ipipe_grab_ipi(unsigned svc, struct pt_regs *regs) /* hw IRQs off */
>                   __ipipe_do_vnmi(IPIPE_SERVICE_VNMI, NULL);
>           else if ((1<<  svc)&  IPI_IPIPE_ALL) {
>                   virq = svc - IPI_IPIPE_CRITICAL + IPIPE_FIRST_IPI;
> -               __ipipe_handle_irq(virq, NULL);
> +               __ipipe_handle_irq(virq, IPIPE_IRQF_NOACK);
>           } else
>                   __ipipe_mach_relay_ipi(svc, cpu);
>
>
>
> Thanks you very much for your help.
>
>
> Best Regards.
>
> Jean-Pascal
>
> _______________________________________________
> Xenomai mailing list
> Xenomai@xenomai.org
> http://www.xenomai.org/mailman/listinfo/xenomai
>

[Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Jean-Pascal JULIEN]

Gilles,


The maximum latency response to the irq is less than 100us.
I performed the test during more than 12 hours.
I re do the first test with the internal timer which toggles a GPIO each 1ms and i have no regression.

To avoid any confusion i give you the patches which were applied :

diff --git a/arch/arm/plat-omap/gpio.c b/arch/arm/plat-omap/gpio.c
index 25c17a3..3e5a88d 100644
--- a/arch/arm/plat-omap/gpio.c
+++ b/arch/arm/plat-omap/gpio.c
@@ -1221,7 +1221,7 @@ static void omap_gpio_free(struct gpio_chip *chip, unsigned offset)
         spin_unlock_irqrestore(&bank->lock, flags);
 }
 
-static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
+static void gpio_demux_inner(struct gpio_bank *bank, u32 isr)
 {
         unsigned int gpio_irq, gpio_index;
 
@@ -1230,13 +1230,6 @@ static void gpio_demux_inner(struct gpio_bank *bank, u32 isr, int nonroot)
                 if (!(isr & 1))
                         continue;
 
-#ifdef CONFIG_IPIPE
-               if (!nonroot) {
-                       local_irq_enable_hw();
-                       local_irq_disable_hw();
-               }
-#endif /* CONFIG_IPIPE */
-
 #ifdef CONFIG_ARCH_OMAP1
                 gpio_index = get_gpio_index(irq_to_gpio(gpio_irq));
 
@@ -1272,11 +1265,7 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
         u32 retrigger = 0;
         int unmasked = 0;
 
-#ifndef CONFIG_IPIPE
         desc->irq_data.chip->irq_ack(&desc->irq_data);
-#else /* CONFIG_IPIPE */
-       desc->irq_data.chip->irq_mask_ack(&desc->irq_data);
-#endif /* CONFIG_IPIPE */
 
         bank = get_irq_data(irq);
 #ifdef CONFIG_ARCH_OMAP1
@@ -1312,13 +1301,6 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
                 u32 isr_saved, level_mask = 0;
                 u32 enabled;
 
-#ifdef CONFIG_IPIPE
-               if (!bank->nonroot_gpios) {
-                       local_irq_enable_hw();
-                       local_irq_disable_hw();
-               }
-#endif /* CONFIG_IPIPE */
-
                 enabled = _get_gpio_irqbank_mask(bank);
                 isr_saved = isr = __raw_readl(isr_reg) & enabled;
 
@@ -1336,27 +1318,19 @@ static void gpio_irq_handler(unsigned int irq, struct irq_desc *desc)
                 _clear_gpio_irqbank(bank, isr_saved & ~level_mask);
                 _enable_gpio_irqbank(bank, isr_saved & ~level_mask, 1);
 
-#ifndef CONFIG_IPIPE
                 /* if there is only edge sensitive GPIO pin interrupts
                 configured, we could unmask GPIO bank interrupt immediately */
                 if (!level_mask && !unmasked) {
                         unmasked = 1;
                         desc->irq_data.chip->irq_unmask(&desc->irq_data);
                 }
-#endif /* !CONFIG_IPIPE */
 
                 isr |= retrigger;
                 retrigger = 0;
                 if (!isr)
                         break;
 
-#ifdef CONFIG_IPIPE
-               if (bank->nonroot_gpios)
-                       gpio_demux_inner(bank, isr & bank->nonroot_gpios, 1);
-               gpio_demux_inner(bank, isr & ~bank->nonroot_gpios, 0);
-#else /* !CONFIG_IPIPE */
-               gpio_demux_inner(bank, isr, 0);
-#endif /* !CONFIG_IPIPE */
+               gpio_demux_inner(bank, isr);
 
         }
         /* if bank has any level sensitive GPIO pin interrupt


diff --git a/arch/arm/include/asm/ipipe.h b/arch/arm/include/asm/ipipe.h
index 9b17c06..a510b09 100644
--- a/arch/arm/include/asm/ipipe.h
+++ b/arch/arm/include/asm/ipipe.h
@@ -285,14 +285,15 @@ void __ipipe_grab_irq(int irq, struct pt_regs *regs);
 
 void __ipipe_exit_irq(struct pt_regs *regs);
 
-void __ipipe_handle_irq(int irq, struct pt_regs *regs);
+#define IPIPE_IRQF_NOACK    0x1
+#define IPIPE_IRQF_NOSYNC   0x2
+
+void __ipipe_handle_irq(int irq, int flags);
 
 static inline void ipipe_handle_chained_irq(unsigned int irq)
 {
-       struct pt_regs regs;    /* dummy */
-
         ipipe_trace_irq_entry(irq);
-       __ipipe_handle_irq(irq, &regs);
+       __ipipe_handle_irq(irq, IPIPE_IRQF_NOSYNC);
         ipipe_trace_irq_exit(irq);
 }
 
diff --git a/arch/arm/include/asm/smp_twd.h b/arch/arm/include/asm/smp_twd.h
index b49c8ac..f655b27 100644
--- a/arch/arm/include/asm/smp_twd.h
+++ b/arch/arm/include/asm/smp_twd.h
@@ -55,7 +55,7 @@ extern void __iomem *twd_base;
 #define __ipipe_mach_relay_ipi(ipi, thiscpu)    \
         ({                                      \
                 (void)(thiscpu);                \
-               __ipipe_handle_irq(ipi, NULL);  \
+               __ipipe_handle_irq(ipi, IPIPE_IRQF_NOACK);      \
         })
 
 struct irq_desc;
diff --git a/arch/arm/kernel/ipipe.c b/arch/arm/kernel/ipipe.c
index f2f618c..ff93fa9 100644
--- a/arch/arm/kernel/ipipe.c
+++ b/arch/arm/kernel/ipipe.c
@@ -254,7 +254,7 @@ int ipipe_trigger_irq(unsigned irq)
                 return -EINVAL;
 #endif
         local_irq_save_hw(flags);
-       __ipipe_handle_irq(irq, NULL);
+       __ipipe_handle_irq(irq, IPIPE_IRQF_NOACK);
         local_irq_restore_hw(flags);
 
         return 1;
@@ -462,7 +462,7 @@ out:
  * interrupt protection log is maintained here for each domain. Hw
  * interrupts are off on entry.
  */
-void __ipipe_handle_irq(int irq, struct pt_regs *regs)
+void __ipipe_handle_irq(int irq, int flags)
 {
         struct ipipe_domain *this_domain, *next_domain;
         struct list_head *head, *pos;
@@ -470,7 +470,7 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
         int m_ack;
 
         /* Software-triggered IRQs do not need any ack. */
-       m_ack = (regs == NULL);
+       m_ack = (flags & IPIPE_IRQF_NOACK) != 0;
 
 #ifdef CONFIG_IPIPE_DEBUG
         if (unlikely(irq >= IPIPE_NR_IRQS) ||
@@ -490,7 +490,11 @@ void __ipipe_handle_irq(int irq, struct pt_regs *regs)
                                 desc = ipipe_virtual_irq_p(irq) ? NULL : irq_to_desc(irq);
                                 next_domain->irqs[irq].acknowledge(irq, desc);
                         }
-                       __ipipe_dispatch_wired(next_domain, irq);
+                       if ((flags & IPIPE_IRQF_NOSYNC) == 0)
+                               __ipipe_dispatch_wired(next_domain, irq);
+                       else
+                               __ipipe_set_irq_pending(next_domain, irq);
+
                         return;
                 }
         }
@@ -604,7 +608,7 @@ asmlinkage void __ipipe_grab_irq(int irq, struct pt_regs *regs)
         ipipe_trace_begin(regs->ARM_ORIG_r0);
 #endif
 
-       __ipipe_handle_irq(irq, regs);
+       __ipipe_handle_irq(irq, 0);
 
 #ifdef CONFIG_IPIPE_TRACE_IRQSOFF
         ipipe_trace_end(regs->ARM_ORIG_r0);
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 9ea207d..af02caa 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -527,7 +527,7 @@ __ipipe_grab_ipi(unsigned svc, struct pt_regs *regs) /* hw IRQs off */
                 __ipipe_do_vnmi(IPIPE_SERVICE_VNMI, NULL);
         else if ((1 << svc) & IPI_IPIPE_ALL) {
                 virq = svc - IPI_IPIPE_CRITICAL + IPIPE_FIRST_IPI;
-               __ipipe_handle_irq(virq, NULL);
+               __ipipe_handle_irq(virq, IPIPE_IRQF_NOACK);
         } else
                 __ipipe_mach_relay_ipi(svc, cpu);
 


Thanks you very much for your help.


Best Regards.

Jean-Pascal

[Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

[Jean-Pascal JULIEN]

Gilles,

I added this last patch, its seem to be working fine.
I will run endurance testing all night long and give you the result tomorrow..
Thanks for your help.

JP
________________________________________
De : Gilles Chanteperdrix [gilles.chanteperdrix@xenomai.org]
Date d'envoi : mardi 22 mai 2012 15:33
À : Jean-Pascal JULIEN
Cc : xenomai@xenomai.org
Objet : Re: RE : RE : RE : [Xenomai] A possible mis-interaction between CONFIG_PREEMPT and GPIO IRQ handling for ARM, leading to extreme latency

On 05/22/2012 03:23 PM, Jean-Pascal JULIEN wrote:
> Gilles,
>
> In the adeos-ipipe-2.6.38.2-arm-1.18-04.patch this piece of code is not present :

Just look at what the patch does: it replaces what the current I-pipe
patch has, with what you already have. If you want the patch I sent to
apply correctly, please use the latest I-pipe patch.

--
                                            Gilles.