Setting the priority of an IRQ thread

Setting the priority of an IRQ thread

[Martin Shepherd]

Is there a way for a device driver to set the priority of the IRQ thread that
services interrupts from its hardware? I am writing a device driver for a
digital I/O board with general purpose interrupt capability, and I would like
the application to be able to tell the driver what priority it needs those
interrupts to be given.

Currently it appears that preemp_rt spawns all IRQ threads with priority 50, and
that the only way to change the priority of an IRQ is for an application program
to execute the chrt program, after the IRQ thread has been created. However this
requires that the driver pass back information to the calling program about
which IRQ it is using, so that the program can then call popen to first search
for the PID of the corresponding thread, and then execute chrt to change its
priority. This is possible but messy, and relies on the name of IRQ threads not
changing in the future.

So is there a way for driver code in the kernel to identify the thread of a
given IRQ, and then change its priority?

Thanks,

Martin

RE: Setting the priority of an IRQ thread

[Suresh Kumar SHUKLA]

Hi Martin,

> Is there a way for a device driver to set the priority of the 
> IRQ thread that services interrupts from its hardware? I am 
> writing a device driver for a digital I/O board with general 
> purpose interrupt capability, and I would like the 
> application to be able to tell the driver what priority it 
> needs those interrupts to be given.
> 
> Currently it appears that preemp_rt spawns all IRQ threads 
> with priority 50, and that the only way to change the 
> priority of an IRQ is for an application program to execute 
> the chrt program, after the IRQ thread has been created. 
> However this requires that the driver pass back information 
> to the calling program about which IRQ it is using, so that 
> the program can then call popen to first search for the PID 
> of the corresponding thread, and then execute chrt to change 
> its priority. This is possible but messy, and relies on the 
> name of IRQ threads not changing in the future.
> 
> So is there a way for driver code in the kernel to identify 
> the thread of a given IRQ, and then change its priority?


I tried this exercise in a driver for 2.6.26. I succeeded in identifying the
thread for IRQ and its priority. I couldn't do the priority changes.

 #if 1
	struct sched_param param = { 0, };
	struct irq_desc *desc;

	// 2. Raise the priority of ISR thread
	param.sched_priority = 90;

	// get IRQ descriptor from IRQ, it contains pid
	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
	printk("Thread Priority = %u\n", desc->thread->prio);	
#endif

Along with this, some minor changes ('static' to 'extern' etc) for irq_desc
structure in corresponding files.

For changing the priority, you may try patches sent by Remy Bohmer
[linux@bohmer.net] on 28-May.

-Suresh

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
> 	// get IRQ descriptor from IRQ, it contains pid
> 	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];

That's a very useful clue. I was thinking that I would have to walk the process
tree to find the thread by name. Following up on this clue, it appears as though
in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
I'll try that out in the morning.

Thanks for the reply,

Martin

Re: Setting the priority of an IRQ thread

[Uwe Kleine-König]

On Tue, Jun 16, 2009 at 07:59:28AM +0000, Martin Shepherd wrote:
> Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
> > 	// get IRQ descriptor from IRQ, it contains pid
> > 	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
> 
> That's a very useful clue. I was thinking that I would have to walk the process
> tree to find the thread by name. Following up on this clue, it appears as though
> in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
> IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
> IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
> I'll try that out in the morning.
I wouldn't recommend calling sys_sched_setscheduler from kernel space.
That's the userspace API and you need to pass a __user pointer as third
argument.

The right way is to do this change from userspace.

Best regards
Uwe

-- 
Pengutronix e.K.                              | Uwe Kleine-König            |
Industrial Linux Solutions                    | http://www.pengutronix.de/  |
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1885 bytes --]


On Tue, 16 Jun 2009, Uwe Kleine-König wrote:
>> IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
>> I'll try that out in the morning.
> I wouldn't recommend calling sys_sched_setscheduler from kernel space.
> That's the userspace API and you need to pass a __user pointer as third
> argument.

Good point, although my ioctl() will be passing the scheduling
parameters from user-space. So the need for a __user pointer for this
would not be a problem. Alternatively one could call task_setprio(),
instead of sys_sched_setscheduler(), and omit the then reduntant step
of figuring out the PID of the task.

> The right way is to do this change from userspace.

Which is what I am trying to do.  I want the application thread that
uses my driver to be able to set the priority of its interrupt to
equal the priority of the calling thread.  The problem with calling
sched_setscheduler directly from user-space, is figuring out the PID
of the IRQ thread. First of all, I have multiple boards of the same
type, such that looking in /proc/interrupts won't tell me which IRQ is
being used by a given board. So I would have to implement an ioctl()
anyway, just to query the IRQ from the device driver. Secondly, the
PID of the corresponding IRQ thread changes each time that
request_irq() is called, and then later freed. So searching for the
IRQ thread couldn't be done by a wrapper script around the program.

It seems much more friendly and efficient for my driver to provide
applications with an ioctl that tells it to set the priority of its
IRQ thread.

Note that none of this would be necessary if the IRQ thread of a
device automatically inherited the priority of the highest priority
realtime thread that has requested (and not yet free'd) its IRQ. Not
doing so, leads to a form of priority inversion.

Thanks,

Martin

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

Uwe Kleine-König <u.kleine-koenig <at> pengutronix.de> writes:
> > IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
> > I'll try that out in the morning.
> I wouldn't recommend calling sys_sched_setscheduler from kernel space.
> That's the userspace API and you need to pass a __user pointer as third
> argument.

Good point, although my ioctl() will be passing the scheduling
parameters from user-space. So the need for a __user pointer for this
would not be a problem. Alternatively one could call task_setprio(),
instead of sys_sched_setscheduler(), and omit the then reduntant step
of figuring out the PID of the task.
 
> The right way is to do this change from userspace.

Which is what I am trying to do.  I want the application thread that
uses my driver to be able to set the priority of its interrupt to
equal the priority of the calling thread.  The problem with calling
sched_setscheduler directly from user-space, is figuring out the PID
of the IRQ thread. First of all, I have multiple boards of the same
type, such that looking in /proc/interrupts won't tell me which IRQ is
being used by a given board. So I would have to implement an ioctl()
anyway, just to query the IRQ from the device driver. Secondly, the
PID of the corresponding IRQ thread changes each time that
request_irq() is called, and then later freed. So searching for the
IRQ thread couldn't be done by a wrapper script around the program.

It seems much more friendly and efficient for my driver to provide
applications with an ioctl that tells it to set the priority of its
IRQ thread.

Note that none of this would be necessary if the IRQ thread of a
device automatically inherited the priority of the highest priority
realtime thread that has requested (and not yet free'd) its IRQ. Not
doing so, leads to a form of priority inversion.

Thanks,

Martin




--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1180 bytes --]

On Tue, 16 Jun 2009, Uwe Kleine-König wrote:
> On Tue, Jun 16, 2009 at 07:59:28AM +0000, Martin Shepherd wrote:
> > Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
> > > 	// get IRQ descriptor from IRQ, it contains pid
> > > 	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
> > 
> > That's a very useful clue. I was thinking that I would have to walk the process
> > tree to find the thread by name. Following up on this clue, it appears as though
> > in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
> > IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
> > IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
> > I'll try that out in the morning.
> I wouldn't recommend calling sys_sched_setscheduler from kernel space.
> That's the userspace API and you need to pass a __user pointer as third
> argument.
> 
> The right way is to do this change from userspace.

What a nonsense. Care to look at the code ?

The irq/softirq threads call sys_sched_setscheduler() already to
change their priorities.

The right way to do this is to have an interface
set_irq_thread_prio(irq, prio)

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[GeunSik Lim]

2009/6/17 Thomas Gleixner <tglx@linutronix.de>:
> On Tue, 16 Jun 2009, Uwe Kleine-König wrote:
>> On Tue, Jun 16, 2009 at 07:59:28AM +0000, Martin Shepherd wrote:
>> > Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
>> > >   // get IRQ descriptor from IRQ, it contains pid
>> > >   desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
>> >
>> > That's a very useful clue. I was thinking that I would have to walk the process
>> > tree to find the thread by name. Following up on this clue, it appears as though
>> > in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
>> > IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
>> > IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
>> > I'll try that out in the morning.
>> I wouldn't recommend calling sys_sched_setscheduler from kernel space.
>> That's the userspace API and you need to pass a __user pointer as third
>> argument.
>>
>> The right way is to do this change from userspace.
>
> What a nonsense. Care to look at the code ?
>
> The irq/softirq threads call sys_sched_setscheduler() already to
> change their priorities.
Yes, I agree with your explanation.
> The right way to do this is to have an interface
> set_irq_thread_prio(irq, prio)
You means that we have to utilize wrapper function using
sys_sched_setscheduler()
function for PID setting of IRQ thread. It's right?

Thks,
GeunSik Lim

> Thanks,
>
>        tglx



-- 
Regards,
GeunSik Lim ( Samsung Electronics )
Blog : http://blog.naver.com/invain/
e-Mail: geunsik.lim@samsung.com
           leemgs@gmail.com , leemgs1@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

On Wed, 17 Jun 2009, GeunSik Lim wrote:
>...
> You means that we have to utilize wrapper function using
> sys_sched_setscheduler() function for PID setting of IRQ thread. It's right?

I may be wrong, but from looking through the kernel code, it appears
as though sys_sched_setscheduler() is used in places where the target
thread could spontaneously dissapear, whereas its non-syscall variant,
sched_setscheduler(), is only used when it is clear that the
underlying thread can't dissapear, such as during the creation of a
new thread. I think that this must be because sys_sched_setscheduler()
takes a pid argument, which remains valid even if the underlying
thread dissapears, whereas sched_setscheduler() takes a raw
task_struct argument, which doesn't provide such guarantees.

I don't personally understand why the copy_from_user() call in the
do_sched_setscheduler() part of sys_sched_setscheduler() doesn't get
upset by being handed an argument from the kernel stack, instead of a
user pointer, but as Thomas says, sys_sched_setscheduler() is called
from a few places in the kernel, and apparently it works.

Note that my attempt to do this from my device-driver didn't work out,
because it turns out that irq_to_desc() isn't exported. So my driver
can't use this to find the IRQ thread. I'm currently having a quick go
at hacking the kernel to provide the set_irq_thread_prio(irq, prio)
function that Thomas suggested, but I haven't fiddled with the kernel
before. So it remains to be seen whether this works :-).

Martin

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

On Wed, 17 Jun 2009, Thomas Gleixner wrote:
> The irq/softirq threads call sys_sched_setscheduler() already to
> change their priorities.
>
> The right way to do this is to have an interface
> set_irq_thread_prio(irq, prio)

I just wrote such a function, and it appears to work. I couldn't get
it to work with sys_sched_setscheduler(), since this returned -EFAULT,
indicating that it didn't like the provenance of the sched_param
argument. I thus instead used the non-syscall variant,
sched_setscheduler(), protecting the target task structure from
deletion by bracketing the code with calls to
read_lock(&tasklist_lock), and read_unlock(&tasklist_lock). Hopefully
that is the right lock to take. The implemenation, which I added to
kernel/irq/manage.c, is as follows:

#ifdef CONFIG_PREEMPT_HARDIRQS

int set_irq_thread_prio(unsigned int irq, int prio)
{
         struct irq_desc *desc = irq_to_desc(irq);
         struct sched_param param = { 0, };
         int status;

         param.sched_priority = prio;

         if(desc) {
                 read_lock(&tasklist_lock);
                 if(desc->thread)
                         status = sched_setscheduler(desc->thread, SCHED_FIFO,
                                                     &param);
                 else
                         status = -ENODEV;
                 read_unlock(&tasklist_lock);
         } else {
                 status = -ENODEV;
         }
         return status;
}

EXPORT_SYMBOL(set_irq_thread_prio);

#endif

I also put the corresponding function prototype in
include/linux/irq.h. I called this from my device-driver, and ran my
test program with it a few times, and it worked fine.

Should I submit this as a patch as-is, or is there anything
problematic that I should fix first?

Thanks,

Martin

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Tue, 16 Jun 2009, Martin Shepherd wrote:
> Which is what I am trying to do.  I want the application thread that
> uses my driver to be able to set the priority of its interrupt to
> equal the priority of the calling thread.  The problem with calling
> sched_setscheduler directly from user-space, is figuring out the PID
> of the IRQ thread. First of all, I have multiple boards of the same
> type, such that looking in /proc/interrupts won't tell me which IRQ is
> being used by a given board. So I would have to implement an ioctl()
> anyway, just to query the IRQ from the device driver. Secondly, the
> PID of the corresponding IRQ thread changes each time that
> request_irq() is called, and then later freed. So searching for the
> IRQ thread couldn't be done by a wrapper script around the program.
>
> It seems much more friendly and efficient for my driver to provide
> applications with an ioctl that tells it to set the priority of its
> IRQ thread.

Wrong. That information is already available in sysfs and there is no
reason to use an ioctl for that. ioctls are neither friendly nor
efficient.

The only missing information is the thread id, which needs to
exposed. This is completely independent of a single driver and needs
to be implemented in a generic way.

> Note that none of this would be necessary if the IRQ thread of a
> device automatically inherited the priority of the highest priority
> realtime thread that has requested (and not yet free'd) its IRQ. Not
> doing so, leads to a form of priority inversion.

Eek. That's even more wrong. Why should the priority of a user task
influence the priority of a device driver irq thread ? That's a
question of policy and policies are set from user space. The kernel
merily provides the interfaces.

Also your claim that it leads to a form of priority inversion is not
necessarily true. I can see what you mean, but there are cases where
the high prio thread just wants an async notification from a driver
that something has changed. It maybe or maybe not required that this
information comes in with high prio. Again that's a question of policy
and needs to be decided by the system designer.

Also why do you claim that the user task requests/frees the drivers
irq ? That depends on the driver, i.e. if it is implemented to request
the irq on open(), and can not be generalized.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Tue, 16 Jun 2009, Martin Shepherd wrote:
> Note that my attempt to do this from my device-driver didn't work out,
> because it turns out that irq_to_desc() isn't exported. So my driver

For a good reaseon. The interrupt description structures are not to be
fiddled with by drivers.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Tue, 16 Jun 2009, Martin Shepherd wrote:
> #ifdef CONFIG_PREEMPT_HARDIRQS
> 
> int set_irq_thread_prio(unsigned int irq, int prio)
> {
>         struct irq_desc *desc = irq_to_desc(irq);
>         struct sched_param param = { 0, };
>         int status;
> 
>         param.sched_priority = prio;
> 
>         if(desc) {
>                 read_lock(&tasklist_lock);
>                 if(desc->thread)

That's racy. desc->thread can go away between the check and the
call. That needs locking of desc->lock, but that might have other
implications. Need to have a closer look at the semantics.

>                         status = sched_setscheduler(desc->thread, SCHED_FIFO,
>                                                     &param);
>                 else
>                         status = -ENODEV;
>                 read_unlock(&tasklist_lock);
>         } else {
>                 status = -ENODEV;
>         }
>         return status;
> }
> 
> EXPORT_SYMBOL(set_irq_thread_prio);

EXPORT_SYMBOL_GPL if any.

OTOH, I think that as I said in the other reply already that none of
this magic is really necessary. We need some sysfs based information
of the thread id in addition to the irq nr which is already
there. Then you can just use sched_setscheduler(2) or chrt(1) from user
space. You can do this already, the irq information in sysfs is
already there you just have to decode output of ps(1) to find the IRQ
thread.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

On Wed, 17 Jun 2009, Thomas Gleixner wrote:
>> It seems much more friendly and efficient for my driver to provide
>> applications with an ioctl that tells it to set the priority of its
>> IRQ thread.
>
> Wrong. That information is already available in sysfs and there is no
> reason to use an ioctl for that.

Could you point out where please? Searching sysfs I see the following
two files that contain the IRQ numbers of two of the boards that my
driver handles:

  /bus/pci/drivers/accpcidio/0000:03:02.0/irq
  /bus/pci/drivers/accpcidio/0000:03:03.0/irq

However, I don't see how a user application could figure out which of
these corresponded to the board that it opened using a particular
device file. Only the device driver knows which device file is
associated with a given PCI device ID, and that isn't recorded in
sysfs.

>> Note that none of this would be necessary if the IRQ thread of a
>> device automatically inherited the priority of the highest priority
>> realtime thread that has requested (and not yet free'd) its IRQ. Not
>> doing so, leads to a form of priority inversion.
>
> Eek. That's even more wrong. Why should the priority of a user task
> influence the priority of a device driver irq thread ? That's a
> question of policy and policies are set from user space. The kernel
> merily provides the interfaces.

Agreed. I didn't think this through properly.

> Also why do you claim that the user task requests/frees the drivers
> irq ? That depends on the driver, i.e. if it is implemented to request
> the irq on open(), and can not be generalized.

I was just saying that it is common for device drivers not to request
IRQs until a user application calls their open methods, and that in
such cases only the application and the driver know when is the right
time to search for the newly created IRQ thread. An external script
whose job it was to start the realtime program, and set the priorities
of associated IRQs, would have a hard time doing this reliably.

Martin

Re: Setting the priority of an IRQ thread

[Thomas Pfaff]

On Tue, 16 Jun 2009, Martin Shepherd wrote:

> Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
> > 	// get IRQ descriptor from IRQ, it contains pid
> > 	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
> 
> That's a very useful clue. I was thinking that I would have to walk the process
> tree to find the thread by name. Following up on this clue, it appears as though
> in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
> IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
> IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
> I'll try that out in the morning.
> 

FYI, i use following code to set the interrupt thread prio on first interrupt 
and it works so far:

atomic_t first_irq;

static irqreturn_t irq_handler (int irq, void *devp)
{
#ifdef CONFIG_PREEMPT_HARDIRQS
	if (atomic_cmpxchg (&first_irq, 1, 0))
	{
		struct sched_param param = { .sched_priority = 55 };

		/* raise thread prio on first interrupt */
		sched_setscheduler(current, SCHED_FIFO, &param);
	}
#endif

	/* handle interrupt */

	return IRQ_RETVAL(IRQ_HANDLED);
}

static int init_irqhandler (void)
{
	request_irq (...);
	atomic_set (&dev->first_irq, 1);

	...
}

Thomas

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Tue, 16 Jun 2009, Martin Shepherd wrote:
> On Wed, 17 Jun 2009, Thomas Gleixner wrote:
> > > It seems much more friendly and efficient for my driver to provide
> > > applications with an ioctl that tells it to set the priority of its
> > > IRQ thread.
> > 
> > Wrong. That information is already available in sysfs and there is no
> > reason to use an ioctl for that.
> 
> Could you point out where please? Searching sysfs I see the following
> two files that contain the IRQ numbers of two of the boards that my
> driver handles:
> 
>  /bus/pci/drivers/accpcidio/0000:03:02.0/irq
>  /bus/pci/drivers/accpcidio/0000:03:03.0/irq
> 
> However, I don't see how a user application could figure out which of
> these corresponded to the board that it opened using a particular
> device file. Only the device driver knows which device file is
> associated with a given PCI device ID, and that isn't recorded in
> sysfs.

Errm, and how does udev figure out which device node to create for
which device ?

/sys/devices/pci0000:00/0000:00:1d.7/usb1/dev

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1467 bytes --]

On Wed, 17 Jun 2009, Thomas Gleixner wrote:
> On Tue, 16 Jun 2009, Uwe Kleine-König wrote:
> > On Tue, Jun 16, 2009 at 07:59:28AM +0000, Martin Shepherd wrote:
> > > Suresh Kumar SHUKLA <suresh.shukla <at> st.com> writes:
> > > > 	// get IRQ descriptor from IRQ, it contains pid
> > > > 	desc = &irq_desc[IRQ_BASIC_TIMER_3_1];
> > > 
> > > That's a very useful clue. I was thinking that I would have to walk the process
> > > tree to find the thread by name. Following up on this clue, it appears as though
> > > in kernel 2.6.29.4 I should be able to use desc=irq_to_desc(irq) to look up the
> > > IRQ descriptor, then use pid=get_task_pid(desc->thread), to get the PID of the
> > > IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
> > > I'll try that out in the morning.
> > I wouldn't recommend calling sys_sched_setscheduler from kernel space.
> > That's the userspace API and you need to pass a __user pointer as third
> > argument.
> > 
> > The right way is to do this change from userspace.
> 
> What a nonsense. Care to look at the code ?
> 
> The irq/softirq threads call sys_sched_setscheduler() already to
> change their priorities.

More awake, it should be changed for correctness sake. :)

> The right way to do this is to have an interface
> set_irq_thread_prio(irq, prio)

This still stands, if we need a way to change the prio from the
driver.

But the more I think about it should be done from user space.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Remy Bohmer]

Hello Thomas,

>> > The right way is to do this change from userspace.
>> What a nonsense. Care to look at the code ?
>> The irq/softirq threads call sys_sched_setscheduler() already to
>> change their priorities.
> More awake, it should be changed for correctness sake. :)
>> The right way to do this is to have an interface
>> set_irq_thread_prio(irq, prio)
> This still stands, if we need a way to change the prio from the
> driver.
> But the more I think about it should be done from user space.

Some time ago we discussed on lkml this same subject.
I made several patches for it back then, but due to change of focus I
somewhat forgot these patches...
First post: http://www.nabble.com/-patch-0-3--add-kernel-cmdline-support-for-interrupt-thread-priorities-(repost:CC-to-LKML)-td14424269.html
I posted them recently again forward ported to 2.6.26:
http://www.spinics.net/lists/linux-rt-users/msg04237.html

I do not agree that it is a userspace issue, the rationale behind this
is that users care about functionality, not specific interrupt thread
ids.
Thread-ids can/will vary, and a lot of scripting is needed to map the
proper interrupt handler of the right device driver to the proper
kernel thread, and to set the priorities accordingly.
And what if there is not a userland at all? and init is the only
process in the system, or there is no shell installed?
Or some kernel developer change the name of the interrupt handler? All
different implementations in userspace has to follow as well... And
why is 50 the right default to use, and not 30 or 60?
For a realtime embedded device this could all be the case, and it is
not that strange.

This was for me the reasoning back then to provide patches that do it
from inside the kernel. The patches provide a means by configuring it
via the kernel commandline, or Kconfig.
The configuration is based on the name provided by the interrupt
handler name provided by request_irq(). If the irq-thread is created
it is looked up inside a map what the priority of the IRQ-thread needs
to be. The IRQ-thread is created with the required prio. It also
checks if the IRQ is shared, and it checks if there are conflicts as
well.

If there is interest I can rework this and make it mainline ready on
the short term for mainline 2.6.29-RT. (in that case comments are very
welcome on the 2.6.26 edition, see above)


Kind Regards,

Remy

Re: Setting the priority of an IRQ thread

[GeunSik Lim]

On Wed, Jun 17, 2009 at 8:21 PM, Remy Bohmer<linux@bohmer.net> wrote:
> Hello Thomas,
> Or some kernel developer change the name of the interrupt handler? All
> different implementations in userspace has to follow as well... And
> why is 50 the right default to use, and not 30 or 60?
> For a realtime embedded device this could all be the case, and it is
> not that strange.
Um..., I also don't know exact reason that seted irq priority with 50.
I think that  the value is irq priotiry  is promise in Realtime system.
In my case, When I have to change( or set) RT priority about RT Kernel thread or
Userspace RT applications in a realtime embedded device,
I  change( or set ) priority of  RT Process(or thread) after consider
that irq priority is 50.

-- 
Regards,
GeunSik Lim ( Samsung Electronics )
Blog : http://blog.naver.com/invain/
e-Mail: geunsik.lim@samsung.com
           leemgs@gmail.com , leemgs1@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Setting the priority of an IRQ thread

[Leon Woestenberg]

Hello,

On Wed, Jun 17, 2009 at 1:21 PM, Remy Bohmer<linux@bohmer.net> wrote:
> Thread-ids can/will vary, and a lot of scripting is needed to map the
> proper interrupt handler of the right device driver to the proper
> kernel thread, and to set the priorities accordingly.
> And what if there is not a userland at all? and init is the only
> process in the system, or there is no shell installed?
> Or some kernel developer change the name of the interrupt handler? All
> different implementations in userspace has to follow as well... And
> why is 50 the right default to use, and not 30 or 60?
> For a realtime embedded device this could all be the case, and it is
> not that strange.
>

To make sure the end solution covers all use cases, I re-submit my case:

>From my email: "[PATCH 2.6.24.7-rt14 v2] rt:
{queue,schedule}_work_prio() allowing work priorities other than
caller's priority".

"Adds schedule_work_prio() and queue_work_prio() so that work can be
scheduled with a different priority than the caller.

Currently under PREEMPT RT, queue_work() and therefore schedule_work() makes
the work inherit the caller's priority.

However, if a real-time kernel interrupt handler thread defers work it may want
the deferred work have lower priority so it does not contend for the CPU with
succeeding interrupt handling. In the non real-time kernel, this was guaranteed
because interrupts preempt workqueues.

The faulty case was a non-FIFO serial driver that deferred LED blinking to
a workqueue using schedule_work(). However, that work used GPIO bitbanged I2C,
which uses 50 usecs udelay()s. With the work inheriting the serial IRQ priority,
it easily missed the ~60 usec deadline of 115200 bps communications.

This patch solves the issue by introducing and using schedule_work_prio()."


Basically, what I did to fix our system, was run the workqueue with
lower priority (current - 1) than the interrupt handler.
So that when the interrupt handler started the work on the queue, that
work would not increase interrupt service latency.


What would be the solution forward for this case, for mainlined
drivers having the same problem?



Regards,
-- 
Leon

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Wed, 17 Jun 2009, Remy Bohmer wrote:
> I do not agree that it is a userspace issue, the rationale behind this
> is that users care about functionality, not specific interrupt thread
> ids.

It _is_ solely a user space issue. The kernel has absolutely no idea
what a system is used for and what the system designer decides is the
best priority balance. The kernel provides a default value which just
works for bootup.

We want drivers and the kernel to be as generic as possible to avoid
tons of extra special cases in the code which are no benefit at all.

> Thread-ids can/will vary, and a lot of scripting is needed to map the
> proper interrupt handler of the right device driver to the proper
> kernel thread, and to set the priorities accordingly.
> And what if there is not a userland at all? and init is the only
> process in the system, or there is no shell installed?

Can we talk about real world issues please ?

> Or some kernel developer change the name of the interrupt handler? All
> different implementations in userspace has to follow as well... And

As I said already, we need to expose the tid of the irq thread to
sysfs as we do with the irq number right now.

> why is 50 the right default to use, and not 30 or 60?

50 is a default startup value and the driver does not know in which
device it is built in. It does not care at all whether the device is a
radio a screwdriver or a blinkenlight.

> For a realtime embedded device this could all be the case, and it is
> not that strange.

There is no such thing "realtime embedded device". There is a device
which requires a real time kernel. Again the kernel does not care
about the overall system requirements. This can only be decided by the
system designer and the system designer has to do the same work for
the user space as well. It's all about policies and the kernel does
not implement policies except some safety ones.

The kernel provides selinux as well and the system designer/admin
decides which policies to use.

> If there is interest I can rework this and make it mainline ready on
> the short term for mainline 2.6.29-RT. (in that case comments are very
> welcome on the 2.6.26 edition, see above)

You can post it and I'm not going to take it for the above reasons.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

On Wed, 17 Jun 2009, GeunSik Lim wrote:
> On Wed, Jun 17, 2009 at 8:21 PM, Remy Bohmer<linux@bohmer.net> wrote:
> > Hello Thomas,
> > Or some kernel developer change the name of the interrupt handler? All
> > different implementations in userspace has to follow as well... And
> > why is 50 the right default to use, and not 30 or 60?
> > For a realtime embedded device this could all be the case, and it is
> > not that strange.

> Um..., I also don't know exact reason that seted irq priority with 50.
> I think that  the value is irq priotiry  is promise in Realtime system.

No, it depends on the system. If your super important device irq needs
to run above all others then you configure it, but it's not a kernel
problem. We have run unimportant irq threads on prio 1 to keep them
out of the way.

> In my case, When I have to change( or set) RT priority about RT
> Kernel thread or Userspace RT applications in a realtime embedded
> device, I change( or set ) priority of RT Process(or thread) after
> consider that irq priority is 50.
 
Again, 50 is a default value. There is no law which prohibits that the
irq priorities can be adjusted to the needs of the system.

The kernel treats by default all irq threads in the same way as it
resembles the mainline behaviour closely. But in contrary to the
mainline hardirq handlers you have now the possibility to change
that. If you are happy with the default then of course you can leave
it as is.

Also a vanilla non-rt kernel comes up with defaults and gives you
enough freedom to customize it to your needs. preempt-rt gives you
some more knobs to tweak and of course an easier way to shoot yourself
into the foot.

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Remy Bohmer]

Hello Thomas,

2009/6/17 Thomas Gleixner <tglx@linutronix.de>:
> On Wed, 17 Jun 2009, Remy Bohmer wrote:
>> I do not agree that it is a userspace issue, the rationale behind this
>> is that users care about functionality, not specific interrupt thread
>> ids.
> It _is_ solely a user space issue. The kernel has absolutely no idea
> what a system is used for and what the system designer decides is the
> best priority balance. The kernel provides a default value which just
> works for bootup.

No, it is a system designer issue. The type of users of the kernel is
much wider than only the person who is sitting between the keyboard
and chair. The RT-kernel is also used in embedded devices which are
only configured for 1 single purpose, without a 'user' that knows how
to configure the system, or even cares/knows that there is linux
running behind the surface. That same person might not even have a
login shell to be able to configure anything.

In these cases the system designer wants to be able to configure those
priorities at build time, just like he is used to configure
high-resolution-timers, tickless kernel, board support options, or
whatever dedicated kernel config option.
This type of user do not want to configure it 'runtime' over and over
on every boot. (for example, to reduce boot times as well)
Configuring the kernel IRQ-thread priorities falls in the same
category as the other kernel config options, or kernel commandline
options (why does the average user care to select a certain
clocksource from the kernel commandline? It can be done after boot as
well...)

> The kernel has absolutely no idea what a system is used for

This sounds very weird, the kconfig options determine what the kernel
is used for. It decides if it is full featured, or tiny. What
architecture it runs on, and even if it is purposed for a realtime or
regular application.

So, we agree that determining the kernel IRQ-prios by the _user_, but
the definition of the user is much wider than only the person behind
keyboard and chair at a login shell, and the RT-kernel is used in many
different areas than X86 desktop only. So, even it is a _user_ issue,
it is wrong to conclude that is therefor also a _userspace_ issue.

> We want drivers and the kernel to be as generic as possible to avoid
> tons of extra special cases in the code which are no benefit at all.

I completely agree on that, but there is nothing wrong by integrating
a _generic_ mechanism to allow the user to customise their kernel for
their needs. (To be clear: I am _only_ talking about the patch series
to configure kernel threads from Kconfig of kernel cmd-line, not about
fixed relations between workqueues, softirqs and irq-threads, that is
up to the system designer to work out while tweaking the kernel)
It is as wrong to select a default of 50 instead of 30,40,whatever
(hypothetical) as to integrate a generic mechanism to choose the
default at compile time.

>> Thread-ids can/will vary, and a lot of scripting is needed to map the
>> proper interrupt handler of the right device driver to the proper
>> kernel thread, and to set the priorities accordingly.
>> And what if there is not a userland at all? and init is the only
>> process in the system, or there is no shell installed?
>
> Can we talk about real world issues please ?

Who said it was not a real world example? I have seen systems that do
just that. Just to squeeze out the last tiny bytes of flash space, to
be able to choose a 2 cent cheaper flash, and to minimize the memory
footprint. I talk about dedicated embedded devices here, not multi
gigabytes X86 systems. You might believe it is crazy to do this, but
it is real life...
One example is where the complete realtime application was implemented
inside the kernel as well, as a kernel driver. (This is extremely good
for RT-behaviour -> less/no cache flushes due to process context
switches).
While it is using kernel threads as well and the driver chooses to use
certain kernel thread priorities, the default of 50 could even result
in an unbootable system. In that case the user that defines the
kernel-configuration and selects the driver in Kconfig also needs to
configure the IRQ/SIRQ thread prios.

>> Or some kernel developer change the name of the interrupt handler? All
>> different implementations in userspace has to follow as well... And
>
> As I said already, we need to expose the tid of the irq thread to
> sysfs as we do with the irq number right now.

But the irq-thread-id and irq-number are _not_ relevant _at all_, it
is about device drivers and their interrupt handlers.
If I want to, for example, boost the interrupt handler
'uhci_hcd:usb6', because I have RT requirements to the device
connected to this USB-port, I want the irq-thread boosted that handles
this particular interrupt handler. The irq-thread that belongs to this
driver is hardware deployment specific and can differ on different
revisions of hardware as well. By making the tid public in sysfs,
there still has to be made a mapping between /proc/interrupts and the
sysfs entries.
The same is valid for soft-irqs, I might want to boost 'sirq-net', or
'sirq-timer', regardless what tid it might have.

By adding the tid to sysfs only, it still requires tools like a shell,
cat, grep, cut, ps, and chrt in the filesystem as well, but this cost
memory footprint, and that can make the difference in flash type to
choose in an embedded device, where every penny counts up.
Following your opinion here: you can skip the tid in sysfs as well,
because it can be looked up with 'ps' also...

The kernel can no longer keep its own pants up for stuff that is
implemented inside the kernel for the kernel its own needs. The
default options, only might give a bootable system, but no _realtime_
system. To get a _usable_ realtime kernel there should be no
dependencies to user space.
realtime systems always require lots of tweaking by system-designers
who configure the kernel to their application needs.

>> why is 50 the right default to use, and not 30 or 60?
>
> 50 is a default startup value and the driver does not know in which
> device it is built in. It does not care at all whether the device is a
> radio a screwdriver or a blinkenlight.

uuhh, but there is a selection option in kconfig and kernel cmd-line
for the tickless kernel and highres-timers as well, but why should it?
it does not need to know where it is built in, so it does not need to
be configurable...???
Or is there a different reason for making things configurable: that
the location where the kernel is built in _does_ make sense to make
something configurable?

> The kernel treats by default all irq threads in the same way as it resembles the mainline behaviour closely.

Not completely true: On mainline interrupt handlers have prios in
hardware as well, based on the hardware layout and the behaviour of
the interrupt controller one interrupt can preempt another interrupt
if it has a higher priority. The RT-kernel default maps everything to
50 and handles them equally, ignoring the priority mechanism in
hardware.

>> For a realtime embedded device this could all be the case, and it is
>> not that strange.
>
> There is no such thing "realtime embedded device". There is a device

I said 'a' realtime embedded device, in the sense of 'an example'

> which requires a real time kernel. Again the kernel does not care
> about the overall system requirements. This can only be decided by the
> system designer and the system designer has to do the same work for
> the user space as well. It's all about policies and the kernel does
> not implement policies except some safety ones.
> The kernel provides selinux as well and the system designer/admin
> decides which policies to use.

You seem to talk about X86 only here, and seem to forget those tiny
little devices for which SE-Linux is completely overkill.

>> If there is interest I can rework this and make it mainline ready on
>> the short term for mainline 2.6.29-RT. (in that case comments are very
>> welcome on the 2.6.26 edition, see above)
>
> You can post it and I'm not going to take it for the above reasons.

Unfortunately you seem to be missing the point here.

I certainly not state that that patchset is mainline-ready as it is
now, although others were quite positive about it, as you can see in
the older threads. I would prefer to see useful remarks to be able to
create a patchset that is useful for everybody, honouring the generic
purpose of the kernel, and that helps users of non-X86 systems as
well.
I notice a need for it lately, so I am willing to invest some good
time in it, but I am not going to waste it...

Kind regards,

Remy

Re: Setting the priority of an IRQ thread

[Thomas Gleixner]

Remy,

On Wed, 17 Jun 2009, Remy Bohmer wrote:
> 2009/6/17 Thomas Gleixner <tglx@linutronix.de>:
> > On Wed, 17 Jun 2009, Remy Bohmer wrote:
> >> I do not agree that it is a userspace issue, the rationale behind this
> >> is that users care about functionality, not specific interrupt thread
> >> ids.
> > It _is_ solely a user space issue. The kernel has absolutely no idea
> > what a system is used for and what the system designer decides is the
> > best priority balance. The kernel provides a default value which just
> > works for bootup.
> 
> No, it is a system designer issue. The type of users of the kernel is

Care to read, what I write ? 

>> It _is_ solely a user space issue.

I did not say it is a user issue. 

And get it: A system is more than a kernel. It has a user space as
well. And the system designer cares about both.

> much wider than only the person who is sitting between the keyboard
> and chair. The RT-kernel is also used in embedded devices which are
> only configured for 1 single purpose, without a 'user' that knows how
> to configure the system, or even cares/knows that there is linux
> running behind the surface. That same person might not even have a
> login shell to be able to configure anything.

I did not talk about a person.
 
> In these cases the system designer wants to be able to configure those
> priorities at build time, just like he is used to configure
> high-resolution-timers, tickless kernel, board support options, or
> whatever dedicated kernel config option.
> This type of user do not want to configure it 'runtime' over and over
> on every boot. (for example, to reduce boot times as well)

Groan. We talk about how many ms to tweak the damned irq threads ? And
it's not the irq threads alone. There are softirq threads, kernel
threads for other purposes. 

You want to tweak them all via Kconfig or kernel command line ? Really
a great plan.

> Configuring the kernel IRQ-thread priorities falls in the same
> category as the other kernel config options, or kernel commandline
> options (why does the average user care to select a certain
> clocksource from the kernel commandline? It can be done after boot as
> well...)

No, in same cases it can't.

There is a completely different reason. clocksource selection is not
for system designers sake. It is for debugability of (mostly x86)
systems so users have a fair chance to boot a stock kernel which
selects an disfunctional TSC, HPET or whatever broken piece of crap
happens to be in the users box.

> > The kernel has absolutely no idea what a system is used for
> 
> This sounds very weird, the kconfig options determine what the kernel
> is used for. It decides if it is full featured, or tiny. What
> architecture it runs on, and even if it is purposed for a realtime or
> regular application.

Right. It restricts or widens the set of features which are exposed to
user space. That restricts or widens the number of knobs user space
can tweak. Not more and not less.

> So, we agree that determining the kernel IRQ-prios by the _user_, but
> the definition of the user is much wider than only the person behind
> keyboard and chair at a login shell, and the RT-kernel is used in many
> different areas than X86 desktop only. So, even it is a _user_ issue,
> it is wrong to conclude that is therefor also a _userspace_ issue.

You seem to be completely confused. 

    _user_       != Joe User
    _user space_ != desktop GUI.

The minimal user space needs to run some kind of init and starts the
application. So this setup can either be part of init or of the
application itself.

> > We want drivers and the kernel to be as generic as possible to avoid
> > tons of extra special cases in the code which are no benefit at all.
> 
> I completely agree on that, but there is nothing wrong by integrating
> a _generic_ mechanism to allow the user to customise their kernel for
> their needs. (To be clear: I am _only_ talking about the patch series
> to configure kernel threads from Kconfig of kernel cmd-line, not about
> fixed relations between workqueues, softirqs and irq-threads, that is
> up to the system designer to work out while tweaking the kernel)

Wrong again. The system designer should not tweak the kernel by
hacking around in it and compiling system policies into the kernel.

> It is as wrong to select a default of 50 instead of 30,40,whatever
> (hypothetical) as to integrate a generic mechanism to choose the
> default at compile time.

There is a lot wrong with that approach. You'd move policy into the
kernel which does not belong there by definition.
 
> >> Thread-ids can/will vary, and a lot of scripting is needed to map the
> >> proper interrupt handler of the right device driver to the proper
> >> kernel thread, and to set the priorities accordingly.
> >> And what if there is not a userland at all? and init is the only
> >> process in the system, or there is no shell installed?
> >
> > Can we talk about real world issues please ?
> 
> Who said it was not a real world example? I have seen systems that do
> just that. Just to squeeze out the last tiny bytes of flash space, to
> be able to choose a 2 cent cheaper flash, and to minimize the memory
> footprint. I talk about dedicated embedded devices here, not multi
> gigabytes X86 systems. You might believe it is crazy to do this, but
> it is real life...

Dude. I'm working in the embedded space since more than 20 years and I
know what's going on there. In my lab are probably more non x86 based
real world embedded machines than x86 based gigabyte systems.

> One example is where the complete realtime application was implemented
> inside the kernel as well, as a kernel driver. (This is extremely good

Great design. This is so uninteresting as it can be. The only
interesting point might be the GPL violation which is probably
happening there.

Those kernels are hacked anyway beyond recognition so why does it
matter if there is another ugly and horrible patch which allows that
driver/application to tweak the irq handler priority?

The mainline development cares about sane use cases and not about the
weirdness of some crackpot design which happens to use a completely
crippeled and unfixable kernel. I have seen such things already and
when I need to look at a patch which is larger than the preempt-rt
patch and even larger than the delta of two mainline versions then I
do not care at all. 

I even refuse to touch such shit for money, except they raise the
offer by some orders of magnitude.

The embedded space is nuts. Google for "Embedded nightmare" if you
care.

> for RT-behaviour -> less/no cache flushes due to process context
> switches).

Start using some ARM CPU which does not need that :)

> While it is using kernel threads as well and the driver chooses to use
> certain kernel thread priorities, the default of 50 could even result
> in an unbootable system. In that case the user that defines the
> kernel-configuration and selects the driver in Kconfig also needs to
> configure the IRQ/SIRQ thread prios.

Stop hand waving. Provide a real use case where the kernel does not
reach user space due to the default setting. And please do not provide
one of the kind you described above as I couldn't care less.
 
> >> Or some kernel developer change the name of the interrupt handler? All
> >> different implementations in userspace has to follow as well... And
> >
> > As I said already, we need to expose the tid of the irq thread to
> > sysfs as we do with the irq number right now.
> 
> But the irq-thread-id and irq-number are _not_ relevant _at all_, it
> is about device drivers and their interrupt handlers.
> If I want to, for example, boost the interrupt handler
> 'uhci_hcd:usb6', because I have RT requirements to the device
> connected to this USB-port, I want the irq-thread boosted that handles
> this particular interrupt handler. The irq-thread that belongs to this
> driver is hardware deployment specific and can differ on different
> revisions of hardware as well. By making the tid public in sysfs,
> there still has to be made a mapping between /proc/interrupts and the
> sysfs entries.

Again. You are confused. If you want to tweak uhci_hcd:usb6 then you
look at the very sysfs entry of uhci_hcd:usb6, get the tid and call
sched_setscheduler(2) or chrt(1) and be done.

> The same is valid for soft-irqs, I might want to boost 'sirq-net', or
> 'sirq-timer', regardless what tid it might have.

Where is the point? You can get the tid already today. There might be
some more intuitive interface, but that can be fixed and does not move
any policy decision into the kernel proper.

> By adding the tid to sysfs only, it still requires tools like a shell,
> cat, grep, cut, ps, and chrt in the filesystem as well, but this cost
> memory footprint, and that can make the difference in flash type to
> choose in an embedded device, where every penny counts up.

Yeah. I'm wiping my tears away.

Get some reality. Small FLASH parts which are suitable for Linux have
started years ago to be more expensive than larger ones.

> The kernel can no longer keep its own pants up for stuff that is
> implemented inside the kernel for the kernel its own needs. The
> default options, only might give a bootable system, but no _realtime_
> system. To get a _usable_ realtime kernel there should be no
> dependencies to user space.
> realtime systems always require lots of tweaking by system-designers
> who configure the kernel to their application needs.

Again, that's the wrong way. This is not your old lovely home brewn
bare metal OS where you stick everything in just because you can. It's
about having a sane default system which can be tuned from user space.

> >> why is 50 the right default to use, and not 30 or 60?
> >
> > 50 is a default startup value and the driver does not know in which
> > device it is built in. It does not care at all whether the device is a
> > radio a screwdriver or a blinkenlight.
> 
> uuhh, but there is a selection option in kconfig and kernel cmd-line
> for the tickless kernel and highres-timers as well, but why should it?
> it does not need to know where it is built in, so it does not need to
> be configurable...???

Both help to save your precious $pennies and the command line options
are for debugability (again mostly due to x86 hardware wreckage),
nothing else.

> > The kernel treats by default all irq threads in the same way as it
> > resembles the mainline behaviour closely.
>
> Not completely true: On mainline interrupt handlers have prios in
> hardware as well, based on the hardware layout and the behaviour of
> the interrupt controller one interrupt can preempt another interrupt
> if it has a higher priority. The RT-kernel default maps everything to
> 50 and handles them equally, ignoring the priority mechanism in
> hardware.

I said closely _not_ entirely. And I'm well aware of that. If you care
to look I even wrote a paper about that almost a decade ago.
 
> >> For a realtime embedded device this could all be the case, and it is
> >> not that strange.
> >
> > There is no such thing "realtime embedded device". There is a device
> 
> I said 'a' realtime embedded device, in the sense of 'an example'
> 
> > which requires a real time kernel. Again the kernel does not care
> > about the overall system requirements. This can only be decided by the
> > system designer and the system designer has to do the same work for
> > the user space as well. It's all about policies and the kernel does
> > not implement policies except some safety ones.
> > The kernel provides selinux as well and the system designer/admin
> > decides which policies to use.
> 
> You seem to talk about X86 only here, and seem to forget those tiny
> little devices for which SE-Linux is completely overkill.

I used selinux as an example to illustrate the difference of mechanism
and policy. We need this clear separation otherwise we end up with
tons of requests like yours which are simply not maintainable for the
mainline kernel. From a sane system design perspective there is no
place for policy in the kernel.

Can you please stop to see the world with your narrow minded embedded
world view ?

I'm doing and have been doing embedded work a lot and know very well
that there is a world outside of x86.

But embedded is not that different from x86. It's all the same kernel
and the same philosophy. 

Just you and some others of the embedded species insist that there is
a different philosophy. It's not. It's just different in the embedded
mindset which does not care about anything.

If you can not cope with the UNIX/Linux way, then use whatever OS fits
your philosophy best, really.

> >> If there is interest I can rework this and make it mainline ready on
> >> the short term for mainline 2.6.29-RT. (in that case comments are very
> >> welcome on the 2.6.26 edition, see above)
> >
> > You can post it and I'm not going to take it for the above reasons.
> 
> Unfortunately you seem to be missing the point here.

No, I'm not.
 
> I certainly not state that that patchset is mainline-ready as it is
> now, although others were quite positive about it, as you can see in
> the older threads. I would prefer to see useful remarks to be able to
> create a patchset that is useful for everybody, honouring the generic
> purpose of the kernel, and that helps users of non-X86 systems as
> well.
> I notice a need for it lately, so I am willing to invest some good
> time in it, but I am not going to waste it...

I don't care about others being positive about it. I'm refusing to put
policy into the kernel for two reasons:

       - there is no place for policy in the kernel by definition
       - policy in the kernel is not maintainable

Thanks,

	tglx

Re: Setting the priority of an IRQ thread

[Martin Shepherd]

On Wed, 17 Jun 2009, Thomas Gleixner wrote:
> Errm, and how does udev figure out which device node to create for
> which device ?
>
> /sys/devices/pci0000:00/0000:00:1d.7/usb1/dev

Thanks for the hint. I wasn't aware of the need to call
device_create() etc in my driver, and had been creating device files
by hand.  Having now remedied this omission, I have taken the
opportunity to add an irq attribute file under each of my device
entries in sysfs. Next I would like to add an attribute that indicates
the corresponding thread PID. To do that, I need a kernel function
that can be called to determine the IRQ thread ID.

I have just written and tested the following function for doing this:

#ifdef CONFIG_PREEMPT_HARDIRQS

pid_t get_irq_thread_pid(unsigned int irq, struct pid_namespace *ns)
{
         struct irq_desc *desc = irq_to_desc(irq);
         pid_t pid = 0;

         if(desc) {
                 unsigned long flags;
                 spin_lock_irqsave(&desc->lock, flags);
                 if(desc->thread)
                         pid = task_pid_nr_ns(desc->thread, ns);
                 spin_unlock_irqrestore(&desc->lock, flags);
         }
         return pid;
}

EXPORT_SYMBOL(get_irq_thread_pid);

#endif

This would most frequently be called like:

   pid = get_irq_thread_pid(irq, current->nsproxy->pid_ns);
   if(pid) {
     ...
   }

Would this function be acceptible for inclusion in the preempt-rt
kernel, or do you have something else in mind?

Martin

RE: Setting the priority of an IRQ thread

[Suresh Kumar SHUKLA]

> > It _is_ solely a user space issue. The kernel has 

I don't agree. Isn't this a policy of its own.

Mechanism for changing priority should be available within driver code too.
My desired functionality fits perfectly fine as driver/LKM, why force me to
create a user mode application.

Re: Setting the priority of an IRQ thread

[Peter Morreale]

Sorry for the top post. On BB. Network is down. :-(

Why not have the driver ioctl return the pid of the IRQ thread?  That way the driver stays 'clean' and the app is free to manipulate the prio as it sees fit.  

Just a thought...

-PWM 
-----Original Message-----
From: Martin Shepherd <mcs@astro.caltech.edu>
To:  <u.kleine-koenig@pengutronix.de>
Cc:  <linux-rt-users@vger.kernel.org>

Sent: 6/16/2009 3:25:18 PM
Subject: Re: Setting the priority of an IRQ thread


On Tue, 16 Jun 2009, Uwe Kleine-König wrote:
>> IRQ thread, then use sys_sched_setscheduler(pid,...) to change its priority.
>> I'll try that out in the morning.
> I wouldn't recommend calling sys_sched_setscheduler from kernel space.
> That's the userspace API and you need to pass a __user pointer as third
> argument.

Good point, although my ioctl() will be passing the scheduling
parameters from user-space. So the need for a __user pointer for this
would not be a problem. Alternatively one could call task_setprio(),
instead of sys_sched_setscheduler(), and omit the then reduntant step
of figuring out the PID of the task.

> The right way is to do this change from userspace.

Which is what I am trying to do.  I want the application thread that
uses my driver to be able to set the priority of its interrupt to
equal the priority of the calling thread.  The problem with calling
sched_setscheduler directly from user-space, is figuring out the PID
of the IRQ thread. First of all, I have multiple boards of the same
type, such that looking in /proc/interrupts won't tell me which IRQ is
being used by a given board. So I would have to implement an ioctl()
anyway, just to query the IRQ from the device driver. Secondly, the
PID of the corresponding IRQ thread changes each time that
request_irq() is called, and then later freed. So searching for the
IRQ thread couldn't be done by a wrapper script around the program.

It seems much more friendly and efficient for my driver to provide
applications with an ioctl that tells it to set the priority of its
IRQ thread.

Note that none of this would be necessary if the IRQ thread of a
device automatically inherited the priority of the highest priority
realtime thread that has requested (and not yet free'd) its IRQ. Not
doing so, leads to a form of priority inversion.

Thanks,

Martin
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html