[Xenomai] RTDM driver init: interrupts and context objects

[Xenomai] RTDM driver init: interrupts and context objects

[andrija]

Hi,

I am using Xenomai 2.6.3 and I'm porting a Linux driver for an Acromag 
PCI DAQ board to RTDM.

Since RTDM in Xenomai 2.6.3 does not support minor device numbers, and 
there are possibly multiple PCI boards which must be accessed by this 
driver, I am following the example given for the 16550 UART driver in 
the Xenomai 2.6.3 source repository. There, we have a rtdm_device 
template with a blank name and inside the init_module() function the 
driver searches for multiple devices, and then assigns them names and 
registers each of them separately (i.e. rtserial0, rtserial1, rtserial2, 
etc.) so that each can be accessed as a separate device via the RTDM 
device table. This driver maintains a static array of rtdm_device, 
keeping there an instance of each detected device. So far so good. I am 
following the same approach in my driver.

However, I see some differences with which I am stuck.

The driver I am porting to RTDM scans for PCI devices in the 
init_module() method, and then, for each device found, performs some 
set-up stuff including - this is the most important thing - setting up 
an IRQ line.

The RTDM UART driver I am using as an example does not do this - it 
performs the IRQ line request (via rtdm_irq_request) in the open() 
method, NOT in the init_module() method. Is there any particular reason 
for this? Would it be incorrect to call rtdm_irq_request for each device 
in init_module() instead of in open(...)?

In order to call rtdm_irq_request, I require the irq handle, which is a 
field of the driver-specific context structure (e.g. in the UART driver 
case, rt_16550_context). I also have such a struct which stores some 
information about my device. Notwithstanding the IRQ setup, some of the 
information I would like to store in my context struct is arrived at 
during the operations in init_module(). However, I cannot see how to 
pass a context struct which I would create in init_module() properly 
when registering a device...? So far, it seems that any data that I 
obtain during init_module() I would have to save in e.g. static arrays, 
and then copy that into the context struct in open(...) where I can 
access the context struct for that particular device. Or do I have 
something wrong?

Any help is greatly appreciated!

Thanks,

Andrija Stupar
Univ. of Toronto

Re: [Xenomai] RTDM driver init: interrupts and context objects

[Philippe Gerum]

On 02/24/2017 01:27 AM, andrija@stupar.com wrote:
> Hi,
> 
> I am using Xenomai 2.6.3 and I'm porting a Linux driver for an Acromag
> PCI DAQ board to RTDM.
> 
> Since RTDM in Xenomai 2.6.3 does not support minor device numbers, and
> there are possibly multiple PCI boards which must be accessed by this
> driver, I am following the example given for the 16550 UART driver in
> the Xenomai 2.6.3 source repository. There, we have a rtdm_device
> template with a blank name and inside the init_module() function the
> driver searches for multiple devices, and then assigns them names and
> registers each of them separately (i.e. rtserial0, rtserial1, rtserial2,
> etc.) so that each can be accessed as a separate device via the RTDM
> device table. This driver maintains a static array of rtdm_device,
> keeping there an instance of each detected device. So far so good. I am
> following the same approach in my driver.
> 
> However, I see some differences with which I am stuck.
> 
> The driver I am porting to RTDM scans for PCI devices in the
> init_module() method, and then, for each device found, performs some
> set-up stuff including - this is the most important thing - setting up
> an IRQ line.
> 
> The RTDM UART driver I am using as an example does not do this - it
> performs the IRQ line request (via rtdm_irq_request) in the open()
> method, NOT in the init_module() method. Is there any particular reason
> for this? Would it be incorrect to call rtdm_irq_request for each device
> in init_module() instead of in open(...)?
> 

No issue with that. Any call site from a regular kernel context (i.e.
not Xenomai's primary/rt mode) will do.

> In order to call rtdm_irq_request, I require the irq handle, which is a
> field of the driver-specific context structure (e.g. in the UART driver
> case, rt_16550_context). I also have such a struct which stores some
> information about my device. Notwithstanding the IRQ setup, some of the
> information I would like to store in my context struct is arrived at
> during the operations in init_module(). However, I cannot see how to
> pass a context struct which I would create in init_module() properly
> when registering a device...? So far, it seems that any data that I
> obtain during init_module() I would have to save in e.g. static arrays,
> and then copy that into the context struct in open(...) where I can
> access the context struct for that particular device. Or do I have
> something wrong?
> 

A context in the usual sense applies to a particular _connection_ to a
device, i.e. per-fd, not to the device itself. In that sense, the open()
handler setting up a connection may refer to some global data for
retrieving the setup information about the particular device instance
the connection refers to. Since init_module() only knows about device
instances, you need some proxy data structure to expose the device
information to the open() handler.

-- 
Philippe.

Re: [Xenomai] RTDM driver init: interrupts and context objects

[andrija]

Hi Philippe,

Thank you very much for your earlier response. I have figured that part 
of the driver out and opening and requesting an interrupt line works 
well. Also, upon an interrupt, an interrupt service routine is executed 
as expected. However I have one remaining problem I hope you could help 
me out with. For interrupt-driven A/D conversion, the hardware I am 
using works like this: 1) I write to a register to initiate 
interrupt-driven A/D conversion 2) the write call then goes to sleep 
until data is ready 3) an interrupt handler awakens the write call 4) 
the write call returns and 5) the code which made the write call now 
knows that it should read data from an input register.

In the original driver, the code doing steps 1), 2) and 4) looks like 
this:

wqc = 0;       /* indicate waiting */
writew( (int)ldata, (void *)(*adata) );
wait_event_interruptible(ain_queue, wqc);
wqc = 2;       /* indicate complete */
.... return (...) ;

Since interrupts in RTDM work slightly differently and there is no wait 
queue in the RTDM/Xenomai version I am using, I ported the above code to 
RTDM like this:

writew( (int)ldata, (void *)(*adata) );
if (!my_context->data_flag) {
	 my_context->wait_flag = TRUE; /* indicate waiting */
          if (rtdm_event_wait(&my_context->apc8620_wait)) { /* wait */
                        my_context->wait_flag = FALSE;
                        return ( -ERESTARTSYS );
          }
}
/* indicate complete */
my_context->data_flag = FALSE;
my_context->wait_flag = FALSE;
.... return (...) ;

In the original driver, step 3) is the in the interrupt handler function 
and looks like this:

if( wqc == 0)        /* waiting for a blocked start convert AND an 
interrupt from the input board */
{
      wqc = 1;               /* when received, change the condition to 
true... */
       wake_up_interruptible(&ain_queue);	 /* ... and wake the blocked 
write */
}

My port of the above to RTDM is:

if (my_context->wait_flag && !(my_context->data_flag)) {
       	 my_context->data_flag = TRUE;
          my_context->wait_flag = FALSE;
        	 rtdm_event_signal(&my_context->apc8620_wait);
}
return( RTDM_IRQ_HANDLED);

My understanding is that the above two pairs of code snippets (vanilla 
Linux and RTDM) do the same thing, functionally. However they do not. 
The interrupt driven A/D conversion is defined by a timer which can be 
set to some range of values (e.g. 1 second). However it seems that the 
board generates interrupts for other reasons as well.

When I run this using the Linux driver, my write call that initiates 
sleeps for about a second (i.e. the timer period) and then returns, and 
then there is data to read. When I run this running my RTDM driver, my 
write call sleeps much less, it returns almost immediately, and there is 
no data to read.

My question is therefore - have I made some obvious mistake in porting 
the regular interrupts to RTDM interrupts in the code above? Am I using 
RTDM interrupts correctly here? Or is the code I've written expectedly 
behaving differently from the original code?

Thanks for any help.

Best regards,

Andrija Stupar



On 2017-02-24 04:03, Philippe Gerum wrote:

> No issue with that. Any call site from a regular kernel context (i.e.
> not Xenomai's primary/rt mode) will do.
> 
> A context in the usual sense applies to a particular _connection_ to a
> device, i.e. per-fd, not to the device itself. In that sense, the 
> open()
> handler setting up a connection may refer to some global data for
> retrieving the setup information about the particular device instance
> the connection refers to. Since init_module() only knows about device
> instances, you need some proxy data structure to expose the device
> information to the open() handler.

Re: [Xenomai] RTDM driver init: interrupts and context objects

[Andrija Stupar]

Well, I can answer my own question from a couple of weeks ago. Nothing 
was wrong with the way the RTDM interrupts were coded - the port from 
Linux to RTDM was done properly. The problem was a mode switch, falling 
into secondary (non-RT) mode. As I gather from the Xenomai RTDM API 
documentation, rtdm_event_wait(...) can only be called in RT mode. That 
is, if you call it non-RT mode it does nothing. When this call is made 
in non-RT mode, it just falls through (i.e. it does nothing). The actual 
interrupt occurs later and it has nothing to wake up since nothing went 
to sleep.

Once I found the sources of the mode switch in the code calling the 
driver, and fixed them so that I was always running in RT mode, the 
interrupts worked perfectly.


On 03/02/2017 11:10 PM, andrija@stupar.com wrote:
> Hi Philippe,
>
> Thank you very much for your earlier response. I have figured that 
> part of the driver out and opening and requesting an interrupt line 
> works well. Also, upon an interrupt, an interrupt service routine is 
> executed as expected. However I have one remaining problem I hope you 
> could help me out with. For interrupt-driven A/D conversion, the 
> hardware I am using works like this: 1) I write to a register to 
> initiate interrupt-driven A/D conversion 2) the write call then goes 
> to sleep until data is ready 3) an interrupt handler awakens the write 
> call 4) the write call returns and 5) the code which made the write 
> call now knows that it should read data from an input register.
>
> In the original driver, the code doing steps 1), 2) and 4) looks like 
> this:
>
> wqc = 0;       /* indicate waiting */
> writew( (int)ldata, (void *)(*adata) );
> wait_event_interruptible(ain_queue, wqc);
> wqc = 2;       /* indicate complete */
> .... return (...) ;
>
> Since interrupts in RTDM work slightly differently and there is no 
> wait queue in the RTDM/Xenomai version I am using, I ported the above 
> code to RTDM like this:
>
> writew( (int)ldata, (void *)(*adata) );
> if (!my_context->data_flag) {
>      my_context->wait_flag = TRUE; /* indicate waiting */
>          if (rtdm_event_wait(&my_context->apc8620_wait)) { /* wait */
>                        my_context->wait_flag = FALSE;
>                        return ( -ERESTARTSYS );
>          }
> }
> /* indicate complete */
> my_context->data_flag = FALSE;
> my_context->wait_flag = FALSE;
> .... return (...) ;
>
> In the original driver, step 3) is the in the interrupt handler 
> function and looks like this:
>
> if( wqc == 0)        /* waiting for a blocked start convert AND an 
> interrupt from the input board */
> {
>      wqc = 1;               /* when received, change the condition to 
> true... */
>       wake_up_interruptible(&ain_queue);     /* ... and wake the 
> blocked write */
> }
>
> My port of the above to RTDM is:
>
> if (my_context->wait_flag && !(my_context->data_flag)) {
>            my_context->data_flag = TRUE;
>          my_context->wait_flag = FALSE;
>             rtdm_event_signal(&my_context->apc8620_wait);
> }
> return( RTDM_IRQ_HANDLED);
>
> My understanding is that the above two pairs of code snippets (vanilla 
> Linux and RTDM) do the same thing, functionally. However they do not. 
> The interrupt driven A/D conversion is defined by a timer which can be 
> set to some range of values (e.g. 1 second). However it seems that the 
> board generates interrupts for other reasons as well.
>
> When I run this using the Linux driver, my write call that initiates 
> sleeps for about a second (i.e. the timer period) and then returns, 
> and then there is data to read. When I run this running my RTDM 
> driver, my write call sleeps much less, it returns almost immediately, 
> and there is no data to read.
>
> My question is therefore - have I made some obvious mistake in porting 
> the regular interrupts to RTDM interrupts in the code above? Am I 
> using RTDM interrupts correctly here? Or is the code I've written 
> expectedly behaving differently from the original code?
>
> Thanks for any help.
>
> Best regards,
>
> Andrija Stupar
>

Re: [Xenomai] RTDM driver init: interrupts and context objects

[Philippe Gerum]

On 03/18/2017 05:28 PM, Andrija Stupar wrote:
> Well, I can answer my own question from a couple of weeks ago. Nothing
> was wrong with the way the RTDM interrupts were coded - the port from
> Linux to RTDM was done properly. The problem was a mode switch, falling
> into secondary (non-RT) mode. As I gather from the Xenomai RTDM API
> documentation, rtdm_event_wait(...) can only be called in RT mode. That
> is, if you call it non-RT mode it does nothing. When this call is made
> in non-RT mode, it just falls through (i.e. it does nothing).

Actually, it does something wrong. You should always enable
CONFIG_XENO_DEBUG_COBALT to detect those spurious call sites.

-- 
Philippe.

Re: [Xenomai] RTDM driver init: interrupts and context objects

[andrija]

Hi Phillipe,

I was under the impression that I cannot, since Cobalt is a feature of 
Xenomai 3.x, and I am using Xenomai 2.6.5, where chasing the mode 
switches is a bit more complicated, as explained on the page 
https://xenomai.org//2014/06/finding-spurious-relaxes/

In any case, the causes of the mode switches that were troubling me were 
obvious, just buried in the code that I have been porting to Xenomai, 
and therefore took a while to find (doubtlessly using the debug flag 
would've been easier, but it did not take me weeks - this is not a 
project I work on full time).

Thanks,

Andrija

On 2017-03-19 05:51, Philippe Gerum wrote:

> 
> Actually, it does something wrong. You should always enable
> CONFIG_XENO_DEBUG_COBALT to detect those spurious call sites.

Re: [Xenomai] RTDM driver init: interrupts and context objects

[Philippe Gerum]

On 03/19/2017 07:08 PM, andrija@stupar.com wrote:
> Hi Phillipe,
> 
> I was under the impression that I cannot, since Cobalt is a feature of
> Xenomai 3.x, and I am using Xenomai 2.6.5, where chasing the mode
> switches is a bit more complicated, as explained on the page
> https://xenomai.org//2014/06/finding-spurious-relaxes/
> 
> In any case, the causes of the mode switches that were troubling me were
> obvious, just buried in the code that I have been porting to Xenomai,

It does not seem a mode switch problem, but merely a calling context
issue. There is no mode switch from kernel space, however Xenomai
switches mode automatically at the syscall boundary, as required to keep
the core system sane regardless of how well/bad an application behaves.
Whether such switch is acceptable latency-wise for such application is a
totally different matter.

> and therefore took a while to find (doubtlessly using the debug flag
> would've been easier, but it did not take me weeks - this is not a
> project I work on full time).
> 

The almost equivalent flag for Xenomai 2.6 is CONFIG_XENO_OPT_DEBUG_NUCLEUS.

-- 
Philippe.