(no subject)

[Richard B. Johnson]

To driver wizards:

I have a driver which needs to wait for some hardware.
Basically, it needs to have some code added to the run-queue
so it can get some CPU time even though it's not being called.

It needs to get some CPU time which can be "turned on" or
"turned off" as a result of an interrupt or some external
input from  an ioctl().

So I thought that the "tasklet" would be ideal. However, the
scheduler "thinks" that a tasklet is an interrupt, so any
attempt to sleep in the tasklet results in a kernel panic,
"ieee scheduling in an interrupt..., BUG sched.c line 688".

Next, I added code to try queue_task(). This has the same problem.

Basically the procedure needs to do:

procedure()
{
    if(some_event)
        schedule_timeout(n);               /* Needs to sleep */
    else if(something_else)
        do_something();
   queue_task(procedure, &tq_immediate);   /* Needs to queue itself again */
}

Since I'm running against a time-line, I temporarily  gave the module
some CPU time through an ioctl(), i.e., a separate task that does nothing
except repeatably execute ioctl(GIVE_CPU, NULL); This shows that the
driver actually works. It's a GPIB driver so it needs to get the
CPU to find out if it's addressed to listen, etc. These events don't
produce interrupts.

So, what am I supposed to do to add a piece of driver code to the
run queue so it gets scheduled occasionally?

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re: your mail

[Jens Axboe]

On Tue, May 08 2001, Richard B. Johnson wrote:
> 
> To driver wizards:
> 
> I have a driver which needs to wait for some hardware.
> Basically, it needs to have some code added to the run-queue
> so it can get some CPU time even though it's not being called.
> 
> It needs to get some CPU time which can be "turned on" or
> "turned off" as a result of an interrupt or some external
> input from  an ioctl().
> 
> So I thought that the "tasklet" would be ideal. However, the
> scheduler "thinks" that a tasklet is an interrupt, so any
> attempt to sleep in the tasklet results in a kernel panic,
> "ieee scheduling in an interrupt..., BUG sched.c line 688".

Use a kernel thread? If you don't need to access user space, context
switches are very cheap.

> So, what am I supposed to do to add a piece of driver code to the
> run queue so it gets scheduled occasionally?

Several, grep for kernel_thread.

-- 
Jens Axboe

Re: your mail

[Richard B. Johnson]

On Tue, 8 May 2001, Jens Axboe wrote:

> On Tue, May 08 2001, Richard B. Johnson wrote:
> > 
> > To driver wizards:
> > 
> > I have a driver which needs to wait for some hardware.
> > Basically, it needs to have some code added to the run-queue
> > so it can get some CPU time even though it's not being called.
> > 
> > It needs to get some CPU time which can be "turned on" or
> > "turned off" as a result of an interrupt or some external
> > input from  an ioctl().
> > 
> > So I thought that the "tasklet" would be ideal. However, the
> > scheduler "thinks" that a tasklet is an interrupt, so any
> > attempt to sleep in the tasklet results in a kernel panic,
> > "ieee scheduling in an interrupt..., BUG sched.c line 688".
> 
> Use a kernel thread? If you don't need to access user space, context
> switches are very cheap.
> 
> > So, what am I supposed to do to add a piece of driver code to the
> > run queue so it gets scheduled occasionally?
> 
> Several, grep for kernel_thread.
> 
> -- 
> Jens Axboe
> 

Okay. Thanks. I thought I would have to do that too. No problem.
It's a "tomorrow" thing. Ten hours it too long to stare at a
screen.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re: your mail

[Jens Axboe]

On Tue, May 08 2001, Richard B. Johnson wrote:
> > Use a kernel thread? If you don't need to access user space, context
> > switches are very cheap.
> > 
> > > So, what am I supposed to do to add a piece of driver code to the
> > > run queue so it gets scheduled occasionally?
> > 
> > Several, grep for kernel_thread.
> > 
> > -- 
> > Jens Axboe
> > 
> 
> Okay. Thanks. I thought I would have to do that too. No problem.

A small worker thread and a wait queue to sleeep on and you are all set,
10 minutes tops :-)

> It's a "tomorrow" thing. Ten hours it too long to stare at a
> screen.

Sissy!

-- 
Jens Axboe

Re: your mail

[Alan Cox]

> I have a driver which needs to wait for some hardware.
> Basically, it needs to have some code added to the run-queue
> so it can get some CPU time even though it's not being called.

Wht does it have to wait ? Why cant it just poll and come back next time ?

Re: your mail

[Richard B. Johnson]

On Tue, 8 May 2001, Alan Cox wrote:

> > I have a driver which needs to wait for some hardware.
> > Basically, it needs to have some code added to the run-queue
> > so it can get some CPU time even though it's not being called.
> 
> Wht does it have to wait ? Why cant it just poll and come back next time ?
> 

Good question. I wanted to be able to call the exact same routine(s)
that other routines (exected from read() and write()), execute.
These routines are complex and sleep while waiting for events. I
didn't want to duplicate that code with different time-out mechanisms.

GPIB is nasty because you can't do anything unless the 'controller'
tells you to do it. When "addressed to talk", you have to parse
all the stuff sent via interrupt (ATN bit set, control byte, which
address from the control byte, etc.), then let somebody sleeping
in poll() know that they can now "write()". That can all be handled
via interrupt. But, now for the receive <grin>. The user-mode code
needs to be sleeping until some data are available. That data
may never be available. Something in the driver needs to wait
until the hardware is addressed to receive. Since it is not now
receiving, there is no interrupt! It takes time for the controller
to tell you to listen and then tell somebody else to talk to you.
This means that I need some timeout to recover from the fact
that the other guy may never talk.

Once the other guy starts sending data, the interrupts can be
used to handle the data and, once there are valid data, the
device owner can be awakened, presumably sleeping in poll() or
select(). It's the intermediate time where there are no
interrupts that needs the CPU to determine that we've waited
too long for interrupts so the device had better get off the
bus to start the error recovery procedure.

Bright an early tommorrow, I will check out both ways. A kernel
thread might be "neat". However, I may just look to see if
I can just poll while using existing code.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re:

[george anzinger]

"Richard B. Johnson" wrote:
> 
> To driver wizards:
> 
> I have a driver which needs to wait for some hardware.
> Basically, it needs to have some code added to the run-queue
> so it can get some CPU time even though it's not being called.
> 
> It needs to get some CPU time which can be "turned on" or
> "turned off" as a result of an interrupt or some external
> input from  an ioctl().
> 
> So I thought that the "tasklet" would be ideal. However, the
> scheduler "thinks" that a tasklet is an interrupt, so any
> attempt to sleep in the tasklet results in a kernel panic,
> "ieee scheduling in an interrupt..., BUG sched.c line 688".
> 
> Next, I added code to try queue_task(). This has the same problem.
> 
> Basically the procedure needs to do:
> 
> procedure()
> {
>     if(some_event)
>         schedule_timeout(n);               /* Needs to sleep */
>     else if(something_else)
>         do_something();
>    queue_task(procedure, &tq_immediate);   /* Needs to queue itself again */
> }
> 
> Since I'm running against a time-line, I temporarily  gave the module
> some CPU time through an ioctl(), i.e., a separate task that does nothing
> except repeatably execute ioctl(GIVE_CPU, NULL); This shows that the
> driver actually works. It's a GPIB driver so it needs to get the
> CPU to find out if it's addressed to listen, etc. These events don't
> produce interrupts.
> 
> So, what am I supposed to do to add a piece of driver code to the
> run queue so it gets scheduled occasionally?
> 
> Cheers,
> Dick Johnson

How about something like:

#include <linux/timer.h>

void queue_task(void process_timeout(void), unsigned long timeout,
struct timer_list *timer, unsigned long data)
{
	unsigned long expire = timeout + jiffies;

	init_timer(&timer);
	timer->expires = expire;
	timer->data = data;
	timer->function = process_timeout;

	add_timer(&timer);
}


You will have to define the "struct timer_list timer".  This should
cause the function passed to be called after "timeout" jiffies (1/HZ,
not to be confused with 10 ms).  If you want to stop the timer early do:

	del_timer_sync(&timer);

"data" was not used in you example, but process_timeout will be passed
"data" when it is called.  This routine is called as part of the timer
interrupt, so it must be fast and should not do schedule() calls.  It
could queue a tasklet, however, to relax constraints a bit.

George

Re:

[Andrew Morton]

"Richard B. Johnson" wrote:
> 
> To driver wizards:
> 
> I have a driver which needs to wait for some hardware.
> Basically, it needs to have some code added to the run-queue
> so it can get some CPU time even though it's not being called.
> 
> It needs to get some CPU time which can be "turned on" or
> "turned off" as a result of an interrupt or some external
> input from  an ioctl().

schedule_task()?

Re:

[Richard B. Johnson]

On Tue, 8 May 2001, george anzinger wrote:

> "Richard B. Johnson" wrote:
> > 
> > To driver wizards:
> > 
> > I have a driver which needs to wait for some hardware.
> > Basically, it needs to have some code added to the run-queue
> > so it can get some CPU time even though it's not being called.
> > 
[SNIPPED...]

> How about something like:
> 
> #include <linux/timer.h>
> 
> void queue_task(void process_timeout(void), unsigned long timeout,
> struct timer_list *timer, unsigned long data)
> {
> 	unsigned long expire = timeout + jiffies;
> 
> 	init_timer(&timer);
> 	timer->expires = expire;
> 	timer->data = data;
> 	timer->function = process_timeout;
> 
> 	add_timer(&timer);
> }
> 
> 
> You will have to define the "struct timer_list timer".  This should
> cause the function passed to be called after "timeout" jiffies (1/HZ,
> not to be confused with 10 ms).  If you want to stop the timer early do:
> 
> 	del_timer_sync(&timer);
> 
> "data" was not used in you example, but process_timeout will be passed
> "data" when it is called.  This routine is called as part of the timer
> interrupt, so it must be fast and should not do schedule() calls.  It
> could queue a tasklet, however, to relax constraints a bit.
> 
> George

This is all very nice. This is basically what the 'tasklet' does.
The problem is that I have in my code something like this:


/*-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
/*
 *  This waits for an event-flag to be TRUE. It takes a pointer to
 *  that flag, plus the number of timer-ticks to wait. If it times-
 *  out, it returns -ETIME. Otherwise it returns 0.
 */
static int waitfor(volatile int *event, int mask, int ticks)
{
    unsigned long timer;
    int stat;
    DEB(printk("%s waitfor\n", info->dev));
    stat = -ETIME;
    timer = jiffies + (unsigned long) ticks;
    while(!!time_before(jiffies, timer))
    {
        if(!!(*event & mask))
        {
            stat = 0;
            break;
        }
        schedule();
    }
    return stat; 
}
/*-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=*/
/*
 *  This waits for a bit in a port to be TRUE. It takes the OFFSET of
 *  that port, plus the number of timer-ticks to wait. If it times-
 *  out, it returns -ETIME. Otherwise it returns 0.
 */
static int waitport(int offset, int mask, int ticks)
{
    unsigned long timer;
    int stat;
    DEB(printk("%s waitport\n", info->dev));
    stat = -ETIME;
    timer = jiffies + (unsigned long) ticks;
    while(!!time_before(jiffies, timer))
    {
        if(!!(READ_TNT(offset) & mask))
        {
            stat = 0;
            break;
        }
        schedule();
    }
    return stat; 
}


Both of these procedures schedule() while waiting for something to
happen. The wait can be very long (1 second) so I don't want to
just spin eating CPU cycles. I have to give the CPU to somebody.





Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re: your mail

[Richard B. Johnson]

On Tue, 8 May 2001, Jens Axboe wrote:

> On Tue, May 08 2001, Richard B. Johnson wrote:
> > > Use a kernel thread? If you don't need to access user space, context
> > > switches are very cheap.
> > > 
> > > > So, what am I supposed to do to add a piece of driver code to the
> > > > run queue so it gets scheduled occasionally?
> > > 
> > > Several, grep for kernel_thread.
> > > 
> > > -- 
> > > Jens Axboe
> > > 
> > 
> > Okay. Thanks. I thought I would have to do that too. No problem.
> 
> A small worker thread and a wait queue to sleeep on and you are all set,
> 10 minutes tops :-)
> 
> > It's a "tomorrow" thing. Ten hours it too long to stare at a
> > screen.
> 
> Sissy!
> 

Okay. I am now awake. I will now try the kernel thread. Looks
simple. Got to remember to kill it before/during module removal.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.

Re:

[Alan Cox]

>     while(!!time_before(jiffies, timer))
>     {
>         if(!!(*event & mask))
>         {
>             stat = 0;
>             break;
>         }
>         schedule();

You want to yield as well otherwise you may just spin anyway

> Both of these procedures schedule() while waiting for something to
> happen. The wait can be very long (1 second) so I don't want to
> just spin eating CPU cycles. I have to give the CPU to somebody.

So use a timer


void tick_tick_boom(unsigned long l)
{
	struct my_device *d = (struct my_device *)l;

	if(its_still_busy(d))
	{
		d->timer_count--;
		if(d->timer_count)
		{
			/* Try again until timer_count hits zero */
			add_timer(&t->timer, jiffies+1);
			return;
		}
		else
		{
			/* Lose some .. */
			d->event_status = TIMEOUT;
		}
	}
	else
	{
		/* Win some .. */
		d->event_status = OK;
	}
	/* Wake up the invoker */
	wake_up(&d->timer_wait);
}

Re:

[george anzinger]

Alan Cox wrote:
> 
> >     while(!!time_before(jiffies, timer))
> >     {
> >         if(!!(*event & mask))
> >         {
> >             stat = 0;
> >             break;
> >         }
> >         schedule();
> 
> You want to yield as well otherwise you may just spin anyway
> 
> > Both of these procedures schedule() while waiting for something to
> > happen. The wait can be very long (1 second) so I don't want to
> > just spin eating CPU cycles. I have to give the CPU to somebody.
> 
> So use a timer
> 
> void tick_tick_boom(unsigned long l)
> {
>         struct my_device *d = (struct my_device *)l;
> 
>         if(its_still_busy(d))
>         {
>                 d->timer_count--;
>                 if(d->timer_count)
>                 {
>                         /* Try again until timer_count hits zero */
>                         add_timer(&t->timer, jiffies+1);
>                         return;
>                 }
>                 else
>                 {
>                         /* Lose some .. */
>                         d->event_status = TIMEOUT;
>                 }
>         }
>         else
>         {
>                 /* Win some .. */
>                 d->event_status = OK;
>         }
>         /* Wake up the invoker */
>         wake_up(&d->timer_wait);
> }

To clarify this a bit, the above code invokes itself with the timer and
thus runs under the timer interrupt.  The first call to it would be made
from your driver which would then sleep waiting for the wake_up, which
will come either on success or when the timer_count has expired.  This
code will poll each jiffie.

The key here is to use the wake_up/ sleep combination to pass control
from the interrupt back to the driver.  This is not unlike what you must
already be doing for interrupt completion.

Do pay attention to getting the timer (&t->timer above) properly set up
(see my first response or most any usage in the kernel).

Have I got this right Alan?

George

Re:

[Alan Cox]

> from the interrupt back to the driver.  This is not unlike what you must
> already be doing for interrupt completion.
> 
> Do pay attention to getting the timer (&t->timer above) properly set up
> (see my first response or most any usage in the kernel).
> 
> Have I got this right Alan?

The other thing to watch is that you need to delete the timer before you unload
As you can safely del_timer() an initialised but already deleted timer that
isnt too onerous.

Waiting for a thread in the module unload is trickier. You cannot simply kill
the thread as it may run after cleanup_module() returns. Instead you do

static void cleanup_module(void)
{
	kill_thread();
	down(&thread_sem);
	printk("Thread dead\n");
}

and in the thread exit path do

	up_and_exit(&thread_sem, error_code);

This ensures that the thread of execution has left the module code space and
will not return.

Alan