* interrupt context [not found] <CA+bLfK5FPqFvU2xy7xKdV4LkAvmY6GAPFrB-4UBzn-cOunQ6Xg@mail.gmail.com> @ 2012-10-05 8:51 ` Iain Fraser 2012-10-05 9:32 ` Borislav Petkov 2012-10-05 13:27 ` Theodore Ts'o 0 siblings, 2 replies; 8+ messages in thread From: Iain Fraser @ 2012-10-05 8:51 UTC (permalink / raw) To: linux-kernel Hello, I understand the interrupts and softirq's run in interrupt context ( as opposed to process context ). But what I don't understand is why you cannot sleep in interrupt context? What I have read it states that it doesn't have a process to schedule out. But interrupts use the interrupted processes kernel stack just like a syscall. So surely it is possible to sleep using that stack. Understandably It would be unfair on the process that blocked through no fault of its own. Also if you are not allowed to sleep / schedule during interrupt context. Then how does the system timer pre-empt processes by calling schedule? I understand there are many reasons why you shouldn't: irq lines being disabled, quick completion, etc. But I don't understand why you cannot. Thanks for the help Iain ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 8:51 ` interrupt context Iain Fraser @ 2012-10-05 9:32 ` Borislav Petkov 2012-10-05 10:20 ` Iain Fraser 2012-10-05 13:27 ` Theodore Ts'o 1 sibling, 1 reply; 8+ messages in thread From: Borislav Petkov @ 2012-10-05 9:32 UTC (permalink / raw) To: Iain Fraser; +Cc: linux-kernel On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: > Hello, > > I understand the interrupts and softirq's run in interrupt context ( > as opposed to process context ). But what I > don't understand is why you cannot sleep in interrupt context? > > What I have read it states that it doesn't have a process to schedule > out. But interrupts use the interrupted processes > kernel stack just like a syscall. So surely it is possible to sleep > using that stack. Understandably It would be unfair on the process > that blocked through no fault of its own. > > Also if you are not allowed to sleep / schedule during interrupt > context. Then how does the system timer pre-empt processes by > calling schedule? > > I understand there are many reasons why you shouldn't: irq lines being > disabled, quick completion, etc. But I don't understand > why you cannot. Let's imagine for a second we could sleep in IRQ context and we had some dummy process entity we can schedule out from. So, we schedule out and run another process which enters the kernel and grabs a lock L. At that exact moment, another IRQ on the same line is raised, we execute the same IRQ handler which purely coincidentally starts busy-waiting on the same lock L. How long do you think we'll be busy waiting in IRQ context on that lock? :-) HTH. -- Regards/Gruss, Boris. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 9:32 ` Borislav Petkov @ 2012-10-05 10:20 ` Iain Fraser 2012-10-05 10:34 ` Borislav Petkov 0 siblings, 1 reply; 8+ messages in thread From: Iain Fraser @ 2012-10-05 10:20 UTC (permalink / raw) To: Borislav Petkov, Iain Fraser, linux-kernel Hi, Thanks for the response. But it appears your example illustrates why its bad to sleep while you have a lock. Which I understand, what I wanted to know is why in interrupt context under no circumstances are you allowed to sleep? Btw I have no intention of doing this, I'm just curious. After thinking about it further I'm starting to think its a "design" decision as opposed to a side effect of the kernel implementation. If you could answer: "How can the system timer ( obviously in interrupt context ) call schedule when it is impossible to sleep/schedule in interrupt context?" That would help me greatly. Thanks again iain On Fri, Oct 5, 2012 at 10:32 AM, Borislav Petkov <bp@alien8.de> wrote: > On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: >> Hello, >> >> I understand the interrupts and softirq's run in interrupt context ( >> as opposed to process context ). But what I >> don't understand is why you cannot sleep in interrupt context? >> >> What I have read it states that it doesn't have a process to schedule >> out. But interrupts use the interrupted processes >> kernel stack just like a syscall. So surely it is possible to sleep >> using that stack. Understandably It would be unfair on the process >> that blocked through no fault of its own. >> >> Also if you are not allowed to sleep / schedule during interrupt >> context. Then how does the system timer pre-empt processes by >> calling schedule? >> >> I understand there are many reasons why you shouldn't: irq lines being >> disabled, quick completion, etc. But I don't understand >> why you cannot. > > Let's imagine for a second we could sleep in IRQ context and we had some > dummy process entity we can schedule out from. > > So, we schedule out and run another process which enters the kernel and > grabs a lock L. > > At that exact moment, another IRQ on the same line is raised, we execute > the same IRQ handler which purely coincidentally starts busy-waiting on > the same lock L. > > How long do you think we'll be busy waiting in IRQ context on that lock? > > :-) > > HTH. > > -- > Regards/Gruss, > Boris. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 10:20 ` Iain Fraser @ 2012-10-05 10:34 ` Borislav Petkov 0 siblings, 0 replies; 8+ messages in thread From: Borislav Petkov @ 2012-10-05 10:34 UTC (permalink / raw) To: Iain Fraser; +Cc: linux-kernel On Fri, Oct 05, 2012 at 11:20:20AM +0100, Iain Fraser wrote: > Hi, > > Thanks for the response. But it appears your example illustrates why its bad > to sleep while you have a lock. Which I understand, what I wanted to know is > why in interrupt context under no circumstances are you allowed to sleep? > > Btw I have no intention of doing this, I'm just curious. After > thinking about it further > I'm starting to think its a "design" decision as opposed to a side effect of the > kernel implementation. There are more examples where sleeping in IRQ context is simply problematic. So prohibiting sleeping in such contexts turns out to be the simplest solution concerning code complexity, design, handling, etc. Btw, you can trigger some yourself by adding sleeping functions in an IRQ handler used on your system, disable the "sleeping-while-atomic" warnings and try to run it to see what happens. > If you could answer: > > "How can the system timer ( obviously in interrupt context ) call > schedule when it is impossible to sleep/schedule in interrupt > context?" I think the best way to learn these things is to stare at the code until you've understood exactly what it does when it does it, so I'll let you answer that. And you want to learn because you wouldn't be asking those questions in the first place. :-) HTH. -- Regards/Gruss, Boris. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 8:51 ` interrupt context Iain Fraser 2012-10-05 9:32 ` Borislav Petkov @ 2012-10-05 13:27 ` Theodore Ts'o 2012-10-05 14:03 ` Iain Fraser ` (2 more replies) 1 sibling, 3 replies; 8+ messages in thread From: Theodore Ts'o @ 2012-10-05 13:27 UTC (permalink / raw) To: Iain Fraser; +Cc: linux-kernel On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: > > I understand the interrupts and softirq's run in interrupt context ( > as opposed to process context ). But what I > don't understand is why you cannot sleep in interrupt context? Consider what happens with nested locks (and yes, we definitely need nested locks). In order to prevent deadlocks, it is critical to have lock ordering; that is, you always take locks in a certain order. If all processes take lock A, and then lock B, etc., then you won't have a problem where one process as lock A, and tries to get lock B, and another process has lock B, and tries to take lock A, and they wait for each other forever. If a process has a lock when it gets interrupted, the interrupt handler has no idea what locks may have already been taken. So if a process has taken a mutex (or some other sleeping lock) B, and then the interrupt handler tries to take lock A, that's a perscription for deadlock. In addition, you must never sleep while holding a (non-sleeping) spinlock. If the interrupt handler has interrupted a process which is holding a spinlock, then it simply may not sleep without triggering all sorts of other problems. > What I have read it states that it doesn't have a process to schedule > out. But interrupts use the interrupted processes > kernel stack just like a syscall. So surely it is possible to sleep > using that stack. Understandably It would be unfair on the process > that blocked through no fault of its own. > > Also if you are not allowed to sleep / schedule during interrupt > context. Then how does the system timer pre-empt processes by > calling schedule? The system timer sets the "need to reschedule" flag for that particular process. Then as the system timer returns from the interrupt, there is a common code path which is checked on the way out of any interrupt handler or system call. This code path checks to see if the "need to schedule" flag is set, and if so, at that point instead of returning to the original process, the kernel will simply return to some other process. I would suggest that you get a good introductory Linux book, such as "Linux Kernel Development" by Robert Love. You might also check out the kernelnewbies.org website and mailing list, where you are more likely to get answers to basic introductory questions like this. Regards, - Ted ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 13:27 ` Theodore Ts'o @ 2012-10-05 14:03 ` Iain Fraser 2012-10-05 18:05 ` anish kumar 2012-10-05 18:15 ` Iain Fraser 2 siblings, 0 replies; 8+ messages in thread From: Iain Fraser @ 2012-10-05 14:03 UTC (permalink / raw) To: Theodore Ts'o, Iain Fraser, linux-kernel Thanks for the response Ted. I've actual got that book, in fact it is the source of this question :) . I knew it wasn't a "nice" thing to do to the poor process. But your explanation explains why its also a dangerous thing so thanks. Also another problem with async sleeping a process, is that you could end up causing the idle thread to sleep. And end up in a scenario where there are no runnable tasks. Thanks for the help Iain On Fri, Oct 5, 2012 at 2:27 PM, Theodore Ts'o <tytso@mit.edu> wrote: > On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: >> >> I understand the interrupts and softirq's run in interrupt context ( >> as opposed to process context ). But what I >> don't understand is why you cannot sleep in interrupt context? > > Consider what happens with nested locks (and yes, we definitely need > nested locks). In order to prevent deadlocks, it is critical to have > lock ordering; that is, you always take locks in a certain order. If > all processes take lock A, and then lock B, etc., then you won't have > a problem where one process as lock A, and tries to get lock B, and > another process has lock B, and tries to take lock A, and they wait > for each other forever. > > If a process has a lock when it gets interrupted, the interrupt > handler has no idea what locks may have already been taken. So if a > process has taken a mutex (or some other sleeping lock) B, and then > the interrupt handler tries to take lock A, that's a perscription for > deadlock. > > In addition, you must never sleep while holding a (non-sleeping) > spinlock. If the interrupt handler has interrupted a process which is > holding a spinlock, then it simply may not sleep without triggering > all sorts of other problems. > >> What I have read it states that it doesn't have a process to schedule >> out. But interrupts use the interrupted processes >> kernel stack just like a syscall. So surely it is possible to sleep >> using that stack. Understandably It would be unfair on the process >> that blocked through no fault of its own. >> >> Also if you are not allowed to sleep / schedule during interrupt >> context. Then how does the system timer pre-empt processes by >> calling schedule? > > The system timer sets the "need to reschedule" flag for that > particular process. Then as the system timer returns from the > interrupt, there is a common code path which is checked on the way out > of any interrupt handler or system call. This code path checks to see > if the "need to schedule" flag is set, and if so, at that point > instead of returning to the original process, the kernel will simply > return to some other process. > > I would suggest that you get a good introductory Linux book, such as > "Linux Kernel Development" by Robert Love. You might also check out > the kernelnewbies.org website and mailing list, where you are more > likely to get answers to basic introductory questions like this. > > Regards, > > - Ted ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 13:27 ` Theodore Ts'o 2012-10-05 14:03 ` Iain Fraser @ 2012-10-05 18:05 ` anish kumar 2012-10-05 18:15 ` Iain Fraser 2 siblings, 0 replies; 8+ messages in thread From: anish kumar @ 2012-10-05 18:05 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Iain Fraser, linux-kernel On Fri, 2012-10-05 at 09:27 -0400, Theodore Ts'o wrote: > On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: > > > > I understand the interrupts and softirq's run in interrupt context ( > > as opposed to process context ). But what I > > don't understand is why you cannot sleep in interrupt context? > > Consider what happens with nested locks (and yes, we definitely need > nested locks). In order to prevent deadlocks, it is critical to have > lock ordering; that is, you always take locks in a certain order. If > all processes take lock A, and then lock B, etc., then you won't have > a problem where one process as lock A, and tries to get lock B, and > another process has lock B, and tries to take lock A, and they wait > for each other forever. > > If a process has a lock when it gets interrupted, the interrupt > handler has no idea what locks may have already been taken. So if a > process has taken a mutex (or some other sleeping lock) B, and then > the interrupt handler tries to take lock A, that's a perscription for > deadlock. > > In addition, you must never sleep while holding a (non-sleeping) > spinlock. If the interrupt handler has interrupted a process which is Just to get it right for me.Most of the time spinlock disables the local core interrupts(or all cores interrupts) and when you call the sleep function after holding the spinlock the interrupts will be enabled again(Am I right?).This is not desired and will have unknown results. > holding a spinlock, then it simply may not sleep without triggering > all sorts of other problems. > > > What I have read it states that it doesn't have a process to schedule > > out. But interrupts use the interrupted processes > > kernel stack just like a syscall. So surely it is possible to sleep > > using that stack. Understandably It would be unfair on the process > > that blocked through no fault of its own. > > > > Also if you are not allowed to sleep / schedule during interrupt > > context. Then how does the system timer pre-empt processes by > > calling schedule? > > The system timer sets the "need to reschedule" flag for that > particular process. Then as the system timer returns from the > interrupt, there is a common code path which is checked on the way out set_tsk_need_resched is the call? > of any interrupt handler or system call. This code path checks to see > if the "need to schedule" flag is set, and if so, at that point > instead of returning to the original process, the kernel will simply > return to some other process. > > I would suggest that you get a good introductory Linux book, such as > "Linux Kernel Development" by Robert Love. You might also check out > the kernelnewbies.org website and mailing list, where you are more > likely to get answers to basic introductory questions like this. > > Regards, > > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: interrupt context 2012-10-05 13:27 ` Theodore Ts'o 2012-10-05 14:03 ` Iain Fraser 2012-10-05 18:05 ` anish kumar @ 2012-10-05 18:15 ` Iain Fraser 2 siblings, 0 replies; 8+ messages in thread From: Iain Fraser @ 2012-10-05 18:15 UTC (permalink / raw) To: Theodore Ts'o, Iain Fraser, linux-kernel Actually Theodore I've been thinking about your reply. Surely the kernel doesn't care about a processes locks. Because a process can be pre-empted at any-time. Isn't that the same as allowing a process to be scheduled in interrupt context? I understand the other reason, i.e. not allowing the idle process to sleep is critical. And also appreciate that async scheduling a process ( other than preemption ) is very unfair to say the least :) Cheers Iain On Fri, Oct 5, 2012 at 2:27 PM, Theodore Ts'o <tytso@mit.edu> wrote: > On Fri, Oct 05, 2012 at 09:51:55AM +0100, Iain Fraser wrote: >> >> I understand the interrupts and softirq's run in interrupt context ( >> as opposed to process context ). But what I >> don't understand is why you cannot sleep in interrupt context? > > Consider what happens with nested locks (and yes, we definitely need > nested locks). In order to prevent deadlocks, it is critical to have > lock ordering; that is, you always take locks in a certain order. If > all processes take lock A, and then lock B, etc., then you won't have > a problem where one process as lock A, and tries to get lock B, and > another process has lock B, and tries to take lock A, and they wait > for each other forever. > > If a process has a lock when it gets interrupted, the interrupt > handler has no idea what locks may have already been taken. So if a > process has taken a mutex (or some other sleeping lock) B, and then > the interrupt handler tries to take lock A, that's a perscription for > deadlock. > > In addition, you must never sleep while holding a (non-sleeping) > spinlock. If the interrupt handler has interrupted a process which is > holding a spinlock, then it simply may not sleep without triggering > all sorts of other problems. > >> What I have read it states that it doesn't have a process to schedule >> out. But interrupts use the interrupted processes >> kernel stack just like a syscall. So surely it is possible to sleep >> using that stack. Understandably It would be unfair on the process >> that blocked through no fault of its own. >> >> Also if you are not allowed to sleep / schedule during interrupt >> context. Then how does the system timer pre-empt processes by >> calling schedule? > > The system timer sets the "need to reschedule" flag for that > particular process. Then as the system timer returns from the > interrupt, there is a common code path which is checked on the way out > of any interrupt handler or system call. This code path checks to see > if the "need to schedule" flag is set, and if so, at that point > instead of returning to the original process, the kernel will simply > return to some other process. > > I would suggest that you get a good introductory Linux book, such as > "Linux Kernel Development" by Robert Love. You might also check out > the kernelnewbies.org website and mailing list, where you are more > likely to get answers to basic introductory questions like this. > > Regards, > > - Ted ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-10-05 18:15 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CA+bLfK5FPqFvU2xy7xKdV4LkAvmY6GAPFrB-4UBzn-cOunQ6Xg@mail.gmail.com> 2012-10-05 8:51 ` interrupt context Iain Fraser 2012-10-05 9:32 ` Borislav Petkov 2012-10-05 10:20 ` Iain Fraser 2012-10-05 10:34 ` Borislav Petkov 2012-10-05 13:27 ` Theodore Ts'o 2012-10-05 14:03 ` Iain Fraser 2012-10-05 18:05 ` anish kumar 2012-10-05 18:15 ` Iain Fraser
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).