* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 @ 2005-06-28 17:27 Karsten Wiese 2005-06-28 20:21 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Karsten Wiese @ 2005-06-28 17:27 UTC (permalink / raw) To: linux-kernel; +Cc: mingo Hi Ingo, suffering (not really ;-) double-rated IO-APIC level-interrupts I found the following patch as a solution: --- arch/i386/kernel/io_apic.c~ 2005-06-28 19:07:49.000000000 +0200 +++ arch/i386/kernel/io_apic.c 2005-06-28 19:07:49.000000000 +0200 @@ -1959,6 +1959,7 @@ static void mask_and_ack_level_ioapic_irq(unsigned int irq) { move_irq(irq); + mask_IO_APIC_irq(irq); ack_APIC_irq(); } --- kernel/irq/handle.c~ 2005-06-28 19:19:32.000000000 +0200 +++ kernel/irq/handle.c 2005-06-28 19:19:32.000000000 +0200 @@ -214,7 +214,6 @@ * hardirq redirection to the irqd process context: */ if (redirect_hardirq(desc)) { - desc->handler->disable(irq); goto out_no_end; } it works here on a PREEMPT_RT 2.6.12-RT-V0.7.50-29 base. Level-interrupts are sanely rated again. Didn't check, if the patch breaks XT-PIC mode, which works ok without the patch. Mainline shows the same effect here (VIA K8T800 UP), didn't dig there yet. thanks, Karsten ___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-28 17:27 Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Karsten Wiese @ 2005-06-28 20:21 ` Ingo Molnar 2005-06-28 20:30 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-28 20:21 UTC (permalink / raw) To: Karsten Wiese; +Cc: linux-kernel * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > Hi Ingo, > > suffering (not really ;-) double-rated IO-APIC level-interrupts I > found the following patch as a solution: thanks. I've applied your patch but also tweaked this area a bit, to make the i8259A PIC work too. I've uploaded the -31 patch with these fixes included. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-28 20:21 ` Ingo Molnar @ 2005-06-28 20:30 ` Ingo Molnar 2005-06-28 23:51 ` Karsten Wiese 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-28 20:30 UTC (permalink / raw) To: Karsten Wiese; +Cc: linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > Hi Ingo, > > > > suffering (not really ;-) double-rated IO-APIC level-interrupts I > > found the following patch as a solution: > > thanks. I've applied your patch but also tweaked this area a bit, to > make the i8259A PIC work too. I've uploaded the -31 patch with these > fixes included. make that -50-32, had a leftover hack in io_apic.c. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-28 20:30 ` Ingo Molnar @ 2005-06-28 23:51 ` Karsten Wiese 2005-06-29 6:34 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Karsten Wiese @ 2005-06-28 23:51 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel Am Dienstag, 28. Juni 2005 22:30 schrieb Ingo Molnar: > > * Ingo Molnar <mingo@elte.hu> wrote: > > > > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > > > Hi Ingo, > > > > > > suffering (not really ;-) double-rated IO-APIC level-interrupts I > > > found the following patch as a solution: > > > > thanks. I've applied your patch but also tweaked this area a bit, to > > make the i8259A PIC work too. I've uploaded the -31 patch with these > > fixes included. > > make that -50-32, had a leftover hack in io_apic.c. > looked at -50-33 now and wonder why is mask_IO_APIC_irq() called twice from __do_IRQ()? given a threaded interrupt: __do_IRQ() calls desc->handler->ack(irq). ack points to mask_and_ack_level_ioapic_irq(), which calls mask_IO_APIC_irq(irq). some lines later in __do_IRQ() desc->handler->disable(irq) is called. disable points to mask_IO_APIC_irq(), now being called a 2nd time. I think this 2nd call isn't necessary. Is there a difference between masking an interrupt line and disabling it? What am I missing? Back at 2.6.12-rc5-RT-48-16 mask_and_ack_level_ioapic_irq() also contained the mask_IO_APIC_irq(irq) call and level interrupt-rates where fine. Some versions later it vanished there. Why was that? Karsten ___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-28 23:51 ` Karsten Wiese @ 2005-06-29 6:34 ` Ingo Molnar 2005-06-29 7:00 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-29 6:34 UTC (permalink / raw) To: Karsten Wiese; +Cc: linux-kernel * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > looked at -50-33 now and wonder why is mask_IO_APIC_irq() called twice > from __do_IRQ()? given a threaded interrupt: __do_IRQ() calls > desc->handler->ack(irq). ack points to > mask_and_ack_level_ioapic_irq(), which calls mask_IO_APIC_irq(irq). > some lines later in __do_IRQ() desc->handler->disable(irq) is called. > disable points to mask_IO_APIC_irq(), now being called a 2nd time. I > think this 2nd call isn't necessary. Is there a difference between > masking an interrupt line and disabling it? What am I missing? you are not missing anything - but i found no easy way for the time being to get rid of the second masking. > Back at 2.6.12-rc5-RT-48-16 mask_and_ack_level_ioapic_irq() also > contained the mask_IO_APIC_irq(irq) call and level interrupt-rates > where fine. Some versions later it vanished there. Why was that? i reorganized how redirection is being done, and i've implemented auto-ACK for the i8259A, to reduce IRQ handling costs. One goal was to avoid the masking of the interrupt line for the timer interrupt on i8259A - but i think i'm going to revert that, it's causing too many problems all around. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 6:34 ` Ingo Molnar @ 2005-06-29 7:00 ` Ingo Molnar 2005-06-29 9:15 ` William Weston 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-29 7:00 UTC (permalink / raw) To: Karsten Wiese; +Cc: linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > [...] but i think i'm going to revert that, it's causing too many > problems all around. reverted it and this enabled the removal of the extra ->disable() you noticed - this should further speed up the IOAPIC code. These changes are in the -50-34 kernel i just uploaded. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 7:00 ` Ingo Molnar @ 2005-06-29 9:15 ` William Weston 2005-06-29 12:56 ` Ingo Molnar 2005-06-29 14:48 ` Karsten Wiese 0 siblings, 2 replies; 76+ messages in thread From: William Weston @ 2005-06-29 9:15 UTC (permalink / raw) To: Ingo Molnar; +Cc: Karsten Wiese, linux-kernel On Wed, 29 Jun 2005, Ingo Molnar wrote: > > [...] but i think i'm going to revert that, it's causing too many > > problems all around. > > reverted it and this enabled the removal of the extra ->disable() you > noticed - this should further speed up the IOAPIC code. These changes > are in the -50-34 kernel i just uploaded. -50-34 fixed the wakeup latency regression I was seeing on my Athlon box with -50-33, and seems a bit more responsive than -50-25. Max wakeup latency is back down to 14us (from 39us), even while running JACK (xrun free) and two instances of burnK7. Overall system response is probably the best I've seen with the RT kernels ;-} --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 9:15 ` William Weston @ 2005-06-29 12:56 ` Ingo Molnar 2005-06-30 1:50 ` William Weston 2005-06-29 14:48 ` Karsten Wiese 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-29 12:56 UTC (permalink / raw) To: William Weston; +Cc: Karsten Wiese, linux-kernel * William Weston <weston@sysex.net> wrote: > On Wed, 29 Jun 2005, Ingo Molnar wrote: > > > > [...] but i think i'm going to revert that, it's causing too many > > > problems all around. > > > > reverted it and this enabled the removal of the extra ->disable() you > > noticed - this should further speed up the IOAPIC code. These changes > > are in the -50-34 kernel i just uploaded. > > -50-34 fixed the wakeup latency regression I was seeing on my Athlon > box with -50-33, and seems a bit more responsive than -50-25. Max > wakeup latency is back down to 14us (from 39us), even while running > JACK (xrun free) and two instances of burnK7. Overall system response > is probably the best I've seen with the RT kernels ;-} great! The SMP box running BurnP6 is another system, right? Could you sum up the remaining regressions you are seeing under -RT? (the latency.c warning is one, what others are remaining?) Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 12:56 ` Ingo Molnar @ 2005-06-30 1:50 ` William Weston 0 siblings, 0 replies; 76+ messages in thread From: William Weston @ 2005-06-30 1:50 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Wed, 29 Jun 2005, Ingo Molnar wrote: > great! The SMP box running BurnP6 is another system, right? Could you > sum up the remaining regressions you are seeing under -RT? (the > latency.c warning is one, what others are remaining?) Right. I'm doing the bulk of my RT testing on three machines: Home: UP Athlon 2GHz VIA KT400 desktop, audio & MIDI dev Work: UP Xeon 1.8GHz Intel 845 video encoding & mcast streaming Work: SMT Xeon/HT 3.2GHz Intel 865 desktop, video streaming testing Here's the max wakeup latencies I'm seeing on each machine with -50-35: Athlon 15us Xeon 22us Xeon/HT 290us In all fairness, the Xeon/HT machine is still running with the debug and trace options enabled, while the other two have all debug except wakeup latency timing disabled. On UP, everything looks good. No JACK xruns, even while running burnK7. Streaming with VLC from a Hauppauge card to UDP/multicast is almost flawless (usually takes at least a day before any UDP packets are sent out too late). I only saw one warning in my logs on the Athlon box for the whole -50 series, and that was in -50-15. The non-HT Xeon box hasn't seen any bug warnings since plist_init() on -48-33. On SMT, everything works well under normal desktop load (X, wmaker, 10 dockapps, 10-20 xterms, and firefox). VLC runs very smoothly when I'm not stress-testing. System responsiveness is much better than the vanilla kernels. Wakeup latencies have decreased since the -48 series (generally <100us, spiking to <300us now instead of generally <200us spiking to <1000us). When it comes to stress testing, I'm still finding conditions where CPU hungry processes (like burnP6) can make the box completely unresponsive. So far, I've narrowed it down to the following conditions: X is running (xorg-6.8.2), along with two burnP6 instances, and X apps that are actively updating the screen (like VLC or a collection of wmaker dockapps). If one of these conditions is absent, there's no meltdown. I've been toying with a script to fire up two instances of burnP6, grab some traces with your trace-it code, and kill off those burnP6 processes. The script generally takes the form: #!/bin/bash trace-it > trace1.out sync burn & usleep 100000 trace-it > trace2.out sync burn & usleep 100000 trace-it > trace2.out sync killall burn I copied 'burnP6' to 'burn' so that the process name shows up in the traces. Every time system response disappears, I never get any traces after the second burnP6 fires up. Most of the time, I can still move the mouse pointer for a few minutes (with serio lost synchro warnings on the serial console) before it goes dead, too. Keyboard is almost always lost. The results of two tests (one resulting in a meltdown) are located at: http://sysex.net/testing/2.6.12-RT-V0.7.50-35 The 'test' directory has some traces with X and burnP6 (no VLC or dockapps) running. These traces have chunks of dead time ranging from roughly 500 to 900 us. The 'crash' directory has a trace with X, burnP6, and VLC running, and a bug warning. As soon as the second burnP6 instance started up, the system showed no signs of being alive other than the mouse synchro warnings on the serial console. Overall, I would have to say that RT_PREEMPT is nearly ready for primetime, especially for audio, MIDI, and video. SMT needs some help when it comes to doing anything that requires X. I'm not sure about true SMP, since I can't find a true SMP system to play with right now. Great Work, Ingo! Best Regards, --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 9:15 ` William Weston 2005-06-29 12:56 ` Ingo Molnar @ 2005-06-29 14:48 ` Karsten Wiese 2005-06-29 19:38 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Karsten Wiese @ 2005-06-29 14:48 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel [-- Attachment #1: Type: text/plain, Size: 937 bytes --] Am Mittwoch, 29. Juni 2005 11:15 schrieb William Weston: > On Wed, 29 Jun 2005, Ingo Molnar wrote: > > > > [...] but i think i'm going to revert that, it's causing too many > > > problems all around. > > > > reverted it and this enabled the removal of the extra ->disable() you > > noticed - this should further speed up the IOAPIC code. These changes > > are in the -50-34 kernel i just uploaded. > > -50-34 fixed the wakeup latency regression I was seeing on my Athlon box > with -50-33, and seems a bit more responsive than -50-25. Max wakeup > latency is back down to 14us (from 39us), even while running JACK (xrun > free) and two instances of burnK7. Overall system response is probably > the best I've seen with the RT kernels ;-} > attached patch for io_apic.c lets 1. gcc 3.4.3 optimize io_apic access a little better. 2. CONFIG_X86_UP_IOAPIC_FAST work here. Didn't check, if it really speeds up things. Karsten [-- Attachment #2: io_apic-RT-50-35.diff --] [-- Type: text/x-diff, Size: 3593 bytes --] --- arch/i386/kernel/io_apic.c-50-33 2005-06-29 16:27:40.000000000 +0200 +++ arch/i386/kernel/io_apic.c 2005-06-29 16:27:40.000000000 +0200 @@ -138,14 +138,22 @@ /* * Cache register values: */ -static unsigned int io_apic_cache[MAX_IO_APICS][MAX_IOAPIC_CACHE] +static struct { + unsigned int reg; + unsigned int val[MAX_IOAPIC_CACHE]; +} io_apic_cache[MAX_IO_APICS] ____cacheline_aligned_in_smp; #endif inline unsigned int __raw_io_apic_read(unsigned int apic, unsigned int reg) { - *IO_APIC_BASE(apic) = reg; - return *(IO_APIC_BASE(apic)+4); + volatile unsigned int *io_apic; + io_apic = IO_APIC_BASE(apic); +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = reg; +#endif + io_apic[0] = reg; + return io_apic[4]; } unsigned int raw_io_apic_read(unsigned int apic, unsigned int reg) @@ -153,7 +161,7 @@ unsigned int val = __raw_io_apic_read(apic, reg); #ifdef IOAPIC_CACHE - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif return val; } @@ -172,14 +180,17 @@ } return __raw_io_apic_read(apic, reg); } - if (io_apic_cache[apic][reg] && !sis_apic_bug) - return io_apic_cache[apic][reg]; + if (io_apic_cache[apic].val[reg] && !sis_apic_bug) { + io_apic_cache[apic].reg = -1; + return io_apic_cache[apic].val[reg]; + } #endif return raw_io_apic_read(apic, reg); } void io_apic_write(unsigned int apic, unsigned int reg, unsigned int val) { + volatile unsigned int *io_apic; #ifdef IOAPIC_CACHE if (unlikely(reg >= MAX_IOAPIC_CACHE)) { static int once = 1; @@ -191,10 +202,14 @@ dump_stack(); } } else - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif - *IO_APIC_BASE(apic) = reg; - *(IO_APIC_BASE(apic)+4) = val; + io_apic = IO_APIC_BASE(apic); +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = reg; +#endif + io_apic[0] = reg; + io_apic[4] = val; } /* @@ -214,34 +229,42 @@ */ void io_apic_modify(unsigned int apic, unsigned int reg, unsigned int val) { + volatile unsigned int *io_apic; #ifdef IOAPIC_CACHE - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif - if (unlikely(sis_apic_bug)) - *IO_APIC_BASE(apic) = reg; - *(IO_APIC_BASE(apic)+4) = val; + io_apic = IO_APIC_BASE(apic); +#ifdef IOAPIC_CACHE + if (io_apic_cache[apic].reg != reg || sis_apic_bug) { + io_apic_cache[apic].reg = reg; +#else + if (unlikely(sis_apic_bug)) { +#endif + io_apic[0] = reg; + } + io_apic[4] = val; #ifndef IOAPIC_POSTFLUSH if (unlikely(sis_apic_bug)) #endif /* * Force POST flush by reading: */ - val = *(IO_APIC_BASE(apic)+4); + val = io_apic[4]; } static void __modify_IO_APIC_irq (unsigned int irq, unsigned long enable, unsigned long disable) { struct irq_pin_list *entry = irq_2_pin + irq; - unsigned int pin, reg; + unsigned int pin, val; for (;;) { pin = entry->pin; if (pin == -1) break; - reg = io_apic_read(entry->apic, 0x10 + pin*2); - reg &= ~disable; - reg |= enable; - io_apic_modify(entry->apic, 0x10 + pin*2, reg); + val = io_apic_read(entry->apic, 0x10 + pin*2); + val &= ~disable; + val |= enable; + io_apic_modify(entry->apic, 0x10 + pin*2, val); if (!entry->next) break; entry = irq_2_pin + entry->next; @@ -306,9 +329,13 @@ { int apic, pin; - for (apic = 0; apic < nr_ioapics; apic++) + for (apic = 0; apic < nr_ioapics; apic++) { +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = -1; +#endif for (pin = 0; pin < nr_ioapic_registers[apic]; pin++) clear_IO_APIC_pin(apic, pin); + } } static void set_ioapic_affinity_irq(unsigned int irq, cpumask_t cpumask) ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 14:48 ` Karsten Wiese @ 2005-06-29 19:38 ` Ingo Molnar 2005-06-29 23:36 ` Karsten Wiese 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-29 19:38 UTC (permalink / raw) To: Karsten Wiese; +Cc: William Weston, linux-kernel * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > attached patch for io_apic.c lets > 1. gcc 3.4.3 optimize io_apic access a little better. > 2. CONFIG_X86_UP_IOAPIC_FAST work here. > Didn't check, if it really speeds up things. which change made CONFIG_X86_UP_IOAPIC_FAST work on your box? It seems you've changed the per-register frontside read-cache to something else - was that on purpose? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 19:38 ` Ingo Molnar @ 2005-06-29 23:36 ` Karsten Wiese 2005-06-30 17:52 ` Karsten Wiese 0 siblings, 1 reply; 76+ messages in thread From: Karsten Wiese @ 2005-06-29 23:36 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel Am Mittwoch, 29. Juni 2005 21:38 schrieb Ingo Molnar: > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > attached patch for io_apic.c lets > > 1. gcc 3.4.3 optimize io_apic access a little better. > > 2. CONFIG_X86_UP_IOAPIC_FAST work here. > > Didn't check, if it really speeds up things. > > which change made CONFIG_X86_UP_IOAPIC_FAST work on your box? It seems > you've changed the per-register frontside read-cache to something else - > was that on purpose? > CONFIG_X86_UP_IOAPIC_FAST started working here, when I made io_apic_modify() look like that: void io_apic_modify(unsigned int apic, unsigned int reg, unsigned int val) { #ifdef IOAPIC_CACHE io_apic_cache[apic][reg] = val; #endif // if (unlikely(sis_apic_bug)) commented this ^^ out *IO_APIC_BASE(apic) = reg; *(IO_APIC_BASE(apic)+4) = val; #ifndef IOAPIC_POSTFLUSH if (unlikely(sis_apic_bug)) #endif /* * Force POST flush by reading: */ val = *(IO_APIC_BASE(apic)+4); } This change does it, 'cause when we read a cached value instead of from the ioapic, the ioapic's address register isn't set and thus the following write in io_apic_modify might not scratch the right ioapic register. On top of the above the patch adds unsigned int reg; to io_apic_cache. with that "reg" struct member a mark -1 can be set, when we read from cache, or the reg-number, when we read from the ioapic. Then by comparing parameter reg with io_apic_cache[apic].reg the patched io_apic_modify() knows, if it has to set the address register or not. The register caching in the patch should be the same as before, only the cache changed from a 2 dimensional array to an array holding structs, which contain an array: io_apic_cache[apic][reg] should be equivalent to io_apic_cache[apic].val[reg], no? Karsten ___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-29 23:36 ` Karsten Wiese @ 2005-06-30 17:52 ` Karsten Wiese 2005-06-30 19:46 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 William Weston 2005-06-30 20:50 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Ingo Molnar 0 siblings, 2 replies; 76+ messages in thread From: Karsten Wiese @ 2005-06-30 17:52 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel [-- Attachment #1: Type: text/plain, Size: 3389 bytes --] Am Donnerstag, 30. Juni 2005 01:36 schrieb Karsten Wiese: > Am Mittwoch, 29. Juni 2005 21:38 schrieb Ingo Molnar: > > > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > > > attached patch for io_apic.c lets > > > 1. gcc 3.4.3 optimize io_apic access a little better. > > > 2. CONFIG_X86_UP_IOAPIC_FAST work here. > > > Didn't check, if it really speeds up things. > > > > which change made CONFIG_X86_UP_IOAPIC_FAST work on your box? It seems > > you've changed the per-register frontside read-cache to something else - > > was that on purpose? > > > CONFIG_X86_UP_IOAPIC_FAST started working here, when I made io_apic_modify() > look like that: > > void io_apic_modify(unsigned int apic, unsigned int reg, unsigned int val) > { > #ifdef IOAPIC_CACHE > io_apic_cache[apic][reg] = val; > #endif > // if (unlikely(sis_apic_bug)) > commented this ^^ out > > *IO_APIC_BASE(apic) = reg; > *(IO_APIC_BASE(apic)+4) = val; > #ifndef IOAPIC_POSTFLUSH > if (unlikely(sis_apic_bug)) > #endif > /* > * Force POST flush by reading: > */ > val = *(IO_APIC_BASE(apic)+4); > } > > This change does it, 'cause when we read a cached value instead of from the ioapic, > the ioapic's address register isn't set and thus the following write in io_apic_modify > might not scratch the right ioapic register. > > On top of the above the patch adds > unsigned int reg; > to io_apic_cache. > with that "reg" struct member a mark -1 can be set, when we read from cache, > or the reg-number, when we read from the ioapic. > Then by comparing parameter reg with io_apic_cache[apic].reg > the patched io_apic_modify() knows, if it has to set the address register or not. > The register caching in the patch should be the same as before, > only the cache changed from a 2 dimensional array > to an array holding structs, which contain an array: > io_apic_cache[apic][reg] should be equivalent to io_apic_cache[apic].val[reg], no? > Here come some numbers to back up the usefullness of CONFIG_X86_UP_IOAPIC_FAST. (and to show that my patch actually works ;-)) All measurement where taken on an UP Athlon64 2Ghz running 32bit 2.6.12-RT-50-35 PREEMPT_RT on a K8T800 mobo. I measured the tsc-cycles spent in __do_IRQ() plus those used for interruptcontroller clean-up after the irq-thread finished the handler. Did that with 3 Setups: XT-PIC IO-APIC uncached IO-APIC cached IO-APIC cached + cpu-cache thrashing application hogging the cpu Characteristic values found: XT-PIC irq# | cycles/irq -----------|------------ 0 timer | 18733 5 VIA8237 | 19089 11 eth0 | 22676 14 ide0 | 23057 IO-APIC uncached irq# | cycles/irq -----------|------------ 0 timer | 4711 22 VIA8237 | 32107 16 eth0 | 32680 14 ide0 | 13404 IO-APIC cached irq# | cycles/irq -----------|------------ 0 timer | 5004 22 VIA8237 | 1484 16 eth0 | 1598 14 ide0 | 905 IO-APIC cached + cpu-cache thrashing application hogging the cpu irq# | cycles/irq -----------|------------ 0 timer | 6344 22 VIA8237 | 2512 16 eth0 | 2774 14 ide0 | 1723 Clear winner is IO-APIC cached. The patch used + full /proc/interrupt logs are attached. The patch differs from the previously sent one by precalculation of the ioapic's virtual address and the cycle-measurement code. Cheers, Karsten [-- Attachment #2: ints_io_apic_cached_fullmemload --] [-- Type: text/plain, Size: 810 bytes --] CPU0 0: 7390558 IO-APIC-edge 0/ 558 6344 3540429, timer 1: 14651 IO-APIC-edge 0/ 4651 1583 7364358, i8042 8: 12 IO-APIC-edge 0/ 12 1462 17550, rtc 9: 0 IO-APIC-level 0/ 0 0 0, acpi 12: 120985 IO-APIC-edge 0/ 985 1251 1232698, i8042 14: 117323 IO-APIC-edge 0/ 7321 1723 12621315, ide0 15: 328390 IO-APIC-edge 0/ 8388 619 5194067, ide1 16: 458342 IO-APIC-level 0/ 8342 2774 23146219, eth0, nvidia 21: 0 IO-APIC-level 0/ 0 0 0, ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, uhci_hcd:usb4, uhci_hcd:usb5 22: 7826 IO-APIC-level 0/ 7826 2512 19665441, VIA8237 NMI: 0 LOC: 7390775 ERR: 0 MIS: 0 [-- Attachment #3: ints_xt-pic_noload --] [-- Type: text/plain, Size: 869 bytes --] CPU0 0: 622269 XT-PIC 0/ 2269 18733 42507219, timer 1: 886 XT-PIC 0/ 886 19145 16962916, i8042 2: 0 XT-PIC 0/ 0 0 0, cascade 3: 0 XT-PIC 0/ 0 0 0, uhci_hcd:usb4, uhci_hcd:usb5 5: 6381 XT-PIC 0/ 6381 19089 121807578, VIA8237, ehci_hcd:usb1 8: 1 XT-PIC 0/ 1 23697 23697, rtc 9: 0 XT-PIC 0/ 0 0 0, acpi 11: 1086 XT-PIC 0/ 1086 22676 24626747, uhci_hcd:usb2, uhci_hcd:usb3, eth0 12: 93 XT-PIC 0/ 93 22848 2124875, i8042 14: 3393 XT-PIC 0/ 3392 23057 78211017, ide0 15: 5410 XT-PIC 0/ 5409 22818 123423742, ide1 NMI: 0 LOC: 622240 ERR: 0 MIS: 0 [-- Attachment #4: ints_io_apic_cached_noload --] [-- Type: text/plain, Size: 806 bytes --] CPU0 0: 7520343 IO-APIC-edge 0/ 343 5004 1716578, timer 1: 15028 IO-APIC-edge 0/ 5028 1569 7890363, i8042 8: 12 IO-APIC-edge 0/ 12 1462 17550, rtc 9: 0 IO-APIC-level 0/ 0 0 0, acpi 12: 122185 IO-APIC-edge 0/ 2185 998 2181980, i8042 14: 133059 IO-APIC-edge 0/ 3057 905 2769148, ide0 15: 334204 IO-APIC-edge 0/ 4202 515 2166602, ide1 16: 466139 IO-APIC-level 0/ 6139 1598 9813868, eth0, nvidia 21: 0 IO-APIC-level 0/ 0 0 0, ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, uhci_hcd:usb4, uhci_hcd:usb5 22: 10619 IO-APIC-level 0/ 619 1484 919075, VIA8237 NMI: 0 LOC: 7520565 ERR: 0 MIS: 0 [-- Attachment #5: ints_io_apic_noload --] [-- Type: text/plain, Size: 801 bytes --] CPU0 0: 260074 IO-APIC-edge 0/ 74 4711 348679, timer 1: 596 IO-APIC-edge 0/ 596 11293 6731173, i8042 8: 1 IO-APIC-edge 0/ 1 22686 22686, rtc 9: 0 IO-APIC-level 0/ 0 0 0, acpi 12: 93 IO-APIC-edge 0/ 93 11070 1029547, i8042 14: 3305 IO-APIC-edge 0/ 3303 13404 44275707, ide0 15: 2135 IO-APIC-edge 0/ 2133 10937 23329953, ide1 16: 298 IO-APIC-level 0/ 298 32680 9738917, eth0 21: 0 IO-APIC-level 0/ 0 0 0, ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb3, uhci_hcd:usb4, uhci_hcd:usb5 22: 1294 IO-APIC-level 0/ 1294 32107 41547375, VIA8237 NMI: 0 LOC: 260001 ERR: 0 MIS: 0 [-- Attachment #6: io_apic-RT-50-35.2.diff --] [-- Type: text/x-diff, Size: 10320 bytes --] diff -ru linux-2.6.12-RT-50-35/arch/i386/kernel/io_apic.c linux-2.6.12-RT/arch/i386/kernel/io_apic.c --- linux-2.6.12-RT-50-35/arch/i386/kernel/io_apic.c 2005-06-30 16:38:19.000000000 +0200 +++ linux-2.6.12-RT/arch/i386/kernel/io_apic.c 2005-06-29 20:01:10.000000000 +0200 @@ -138,14 +138,24 @@ /* * Cache register values: */ -static unsigned int io_apic_cache[MAX_IO_APICS][MAX_IOAPIC_CACHE] +static struct { + unsigned int reg; + unsigned int val[MAX_IOAPIC_CACHE]; +} io_apic_cache[MAX_IO_APICS] ____cacheline_aligned_in_smp; #endif -inline unsigned int __raw_io_apic_read(unsigned int apic, unsigned int reg) +volatile unsigned int *io_apic_base[MAX_IO_APICS]; + +static inline unsigned int __raw_io_apic_read(unsigned int apic, unsigned int reg) { - *IO_APIC_BASE(apic) = reg; - return *(IO_APIC_BASE(apic)+4); + volatile unsigned int *io_apic; +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = reg; +#endif + io_apic = io_apic_base[apic]; + io_apic[0] = reg; + return io_apic[4]; } unsigned int raw_io_apic_read(unsigned int apic, unsigned int reg) @@ -153,7 +163,7 @@ unsigned int val = __raw_io_apic_read(apic, reg); #ifdef IOAPIC_CACHE - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif return val; } @@ -172,14 +182,17 @@ } return __raw_io_apic_read(apic, reg); } - if (io_apic_cache[apic][reg] && !sis_apic_bug) - return io_apic_cache[apic][reg]; + if (io_apic_cache[apic].val[reg] && !sis_apic_bug) { + io_apic_cache[apic].reg = -1; + return io_apic_cache[apic].val[reg]; + } #endif return raw_io_apic_read(apic, reg); } void io_apic_write(unsigned int apic, unsigned int reg, unsigned int val) { + volatile unsigned int *io_apic; #ifdef IOAPIC_CACHE if (unlikely(reg >= MAX_IOAPIC_CACHE)) { static int once = 1; @@ -191,10 +204,14 @@ dump_stack(); } } else - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif - *IO_APIC_BASE(apic) = reg; - *(IO_APIC_BASE(apic)+4) = val; + io_apic = io_apic_base[apic]; +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = reg; +#endif + io_apic[0] = reg; + io_apic[4] = val; } /* @@ -214,34 +231,42 @@ */ void io_apic_modify(unsigned int apic, unsigned int reg, unsigned int val) { + volatile unsigned int *io_apic; #ifdef IOAPIC_CACHE - io_apic_cache[apic][reg] = val; + io_apic_cache[apic].val[reg] = val; #endif - if (unlikely(sis_apic_bug)) - *IO_APIC_BASE(apic) = reg; - *(IO_APIC_BASE(apic)+4) = val; + io_apic = io_apic_base[apic]; +#ifdef IOAPIC_CACHE + if (io_apic_cache[apic].reg != reg || sis_apic_bug) { + io_apic_cache[apic].reg = reg; +#else + if (unlikely(sis_apic_bug)) { +#endif + io_apic[0] = reg; + } + io_apic[4] = val; #ifndef IOAPIC_POSTFLUSH if (unlikely(sis_apic_bug)) #endif /* * Force POST flush by reading: */ - val = *(IO_APIC_BASE(apic)+4); + val = io_apic[4]; } static void __modify_IO_APIC_irq (unsigned int irq, unsigned long enable, unsigned long disable) { struct irq_pin_list *entry = irq_2_pin + irq; - unsigned int pin, reg; + unsigned int pin, val; for (;;) { pin = entry->pin; if (pin == -1) break; - reg = io_apic_read(entry->apic, 0x10 + pin*2); - reg &= ~disable; - reg |= enable; - io_apic_modify(entry->apic, 0x10 + pin*2, reg); + val = io_apic_read(entry->apic, 0x10 + pin*2); + val &= ~disable; + val |= enable; + io_apic_modify(entry->apic, 0x10 + pin*2, val); if (!entry->next) break; entry = irq_2_pin + entry->next; @@ -249,13 +274,13 @@ } /* mask = 1 */ -static void __mask_IO_APIC_irq (unsigned int irq) +static inline void __mask_IO_APIC_irq (unsigned int irq) { __modify_IO_APIC_irq(irq, 0x00010000, 0); } /* mask = 0 */ -static void __unmask_IO_APIC_irq (unsigned int irq) +static inline void __unmask_IO_APIC_irq (unsigned int irq) { __modify_IO_APIC_irq(irq, 0, 0x00010000); } @@ -306,9 +331,13 @@ { int apic, pin; - for (apic = 0; apic < nr_ioapics; apic++) + for (apic = 0; apic < nr_ioapics; apic++) { +#ifdef IOAPIC_CACHE + io_apic_cache[apic].reg = -1; +#endif for (pin = 0; pin < nr_ioapic_registers[apic]; pin++) clear_IO_APIC_pin(apic, pin); + } } static void set_ioapic_affinity_irq(unsigned int irq, cpumask_t cpumask) Nur in linux-2.6.12-RT/arch/i386/kernel: io_apic.c~. Nur in linux-2.6.12-RT/arch/i386/kernel: io_apic.c-50-33. diff -ru linux-2.6.12-RT-50-35/arch/i386/kernel/irq.c linux-2.6.12-RT/arch/i386/kernel/irq.c --- linux-2.6.12-RT-50-35/arch/i386/kernel/irq.c 2005-06-30 16:38:19.000000000 +0200 +++ linux-2.6.12-RT/arch/i386/kernel/irq.c 2005-06-30 16:11:18.000000000 +0200 @@ -217,6 +217,8 @@ } if (i < NR_IRQS) { + unsigned int irq_count; + cycles_t cycles; spin_lock_irqsave(&irq_desc[i].lock, flags); action = irq_desc[i].action; if (!action) @@ -230,12 +232,16 @@ seq_printf(p, "%10u ", kstat_cpu(j).irqs[i]); #endif seq_printf(p, " %14s", irq_desc[i].handler->typename); - seq_printf(p, " %s", action->name); - for (action=action->next; action; action = action->next) - seq_printf(p, ", %s", action->name); - seq_printf(p, " %d/%d", irq_desc[i].irqs_unhandled, irq_desc[i].irq_count); + irq_count = irq_desc[i].irq_count; + cycles = irq_desc[i].cycles; + if (irq_count) { + do_div(cycles, irq_count); + } + seq_printf(p, " %5d/%5d %5lld %lld", irq_desc[i].irqs_unhandled, irq_count, cycles, irq_desc[i].cycles); + for (; action; action = action->next) + seq_printf(p, ", %s", action->name); seq_putc(p, '\n'); skip: spin_unlock_irqrestore(&irq_desc[i].lock, flags); Nur in linux-2.6.12-RT/arch/i386/kernel: irq.c~. diff -ru linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c linux-2.6.12-RT/arch/i386/kernel/mpparse.c --- linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c 2005-06-30 16:38:00.000000000 +0200 +++ linux-2.6.12-RT/arch/i386/kernel/mpparse.c 2005-06-29 20:30:50.000000000 +0200 @@ -263,6 +263,7 @@ return; } mp_ioapics[nr_ioapics] = *m; + io_apic_base[nr_ioapics] = IO_APIC_BASE(nr_ioapics); nr_ioapics++; } @@ -914,6 +915,7 @@ mp_ioapics[idx].mpc_apicaddr = address; set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address); + io_apic_base[idx] = IO_APIC_BASE(idx); mp_ioapics[idx].mpc_apicid = io_apic_get_unique_id(idx, id); mp_ioapics[idx].mpc_apicver = io_apic_get_version(idx); Nur in linux-2.6.12-RT/arch/i386/kernel: mpparse.c~. Nur in linux-2.6.12-RT/arch/i386/kernel: mpparse.c-RT-50-35. Nur in linux-2.6.12-RT: cscope.files. Nur in linux-2.6.12-RT: cscope.files~. Nur in linux-2.6.12-RT: cscope.out. Nur in linux-2.6.12-RT/Documentation: realtime-lsm.txt. diff -ru linux-2.6.12-RT-50-35/include/asm-i386/io_apic.h linux-2.6.12-RT/include/asm-i386/io_apic.h --- linux-2.6.12-RT-50-35/include/asm-i386/io_apic.h 2005-06-30 16:38:20.000000000 +0200 +++ linux-2.6.12-RT/include/asm-i386/io_apic.h 2005-06-29 19:59:17.000000000 +0200 @@ -155,6 +155,8 @@ /* MP IRQ source entries */ extern struct mpc_config_intsrc mp_irqs[MAX_IRQ_SOURCES]; +extern volatile unsigned int *io_apic_base[MAX_IO_APICS]; + /* non-0 if default (table-less) MP configuration */ extern int mpc_default_type; Nur in linux-2.6.12-RT/include/asm-i386: io_apic.h~. diff -ru linux-2.6.12-RT-50-35/include/linux/irq.h linux-2.6.12-RT/include/linux/irq.h --- linux-2.6.12-RT-50-35/include/linux/irq.h 2005-06-30 16:38:20.000000000 +0200 +++ linux-2.6.12-RT/include/linux/irq.h 2005-06-30 13:24:46.000000000 +0200 @@ -79,7 +79,7 @@ unsigned int irqs_unhandled; struct task_struct *thread; wait_queue_head_t wait_for_handler; - cycles_t timestamp; + cycles_t cycles; raw_spinlock_t lock; } ____cacheline_aligned irq_desc_t; Nur in linux-2.6.12-RT/include/linux: irq.h~. Nur in linux-2.6.12-RT/include: sound. Nur in linux-2.6.12-RT/kernel: cscope.files. Nur in linux-2.6.12-RT/kernel: cscope.out. diff -ru linux-2.6.12-RT-50-35/kernel/irq/handle.c linux-2.6.12-RT/kernel/irq/handle.c --- linux-2.6.12-RT-50-35/kernel/irq/handle.c 2005-06-30 16:38:20.000000000 +0200 +++ linux-2.6.12-RT/kernel/irq/handle.c 2005-06-30 13:36:11.000000000 +0200 @@ -143,10 +143,10 @@ return retval; } -cycles_t irq_timestamp(unsigned int irq) -{ - return irq_desc[irq].timestamp; -} +/* cycles_t irq_timestamp(unsigned int irq) */ +/* { */ +/* return irq_desc[irq].timestamp; */ +/* } */ /* * do_IRQ handles all normal device IRQ's (the special @@ -160,7 +160,7 @@ unsigned int status; #ifdef CONFIG_PREEMPT_RT unsigned long flags; - + cycles_t timestamp; /* * Disable the soft-irq-flag: */ @@ -178,7 +178,8 @@ desc->handler->end(irq); return 1; } - desc->timestamp = get_cycles(); + + timestamp = get_cycles(); spin_lock(&desc->lock); desc->handler->ack(irq); @@ -256,6 +257,7 @@ local_irq_restore(flags); /* re-disable interrupts because callers expect irqs off: */ // raw_local_irq_disable(); + desc->cycles += get_cycles() - timestamp; #endif return 1; } Nur in linux-2.6.12-RT/kernel/irq: handle.c~. diff -ru linux-2.6.12-RT-50-35/kernel/irq/manage.c linux-2.6.12-RT/kernel/irq/manage.c --- linux-2.6.12-RT-50-35/kernel/irq/manage.c 2005-06-30 16:38:20.000000000 +0200 +++ linux-2.6.12-RT/kernel/irq/manage.c 2005-06-30 13:39:35.000000000 +0200 @@ -412,6 +412,7 @@ spin_lock_irq(&desc->lock); if (desc->status & IRQ_INPROGRESS) { + cycles_t timestamp; action = desc->action; for (;;) { irqreturn_t action_ret = 0; @@ -434,9 +435,11 @@ * The ->end() handler has to deal with interrupts which got * disabled while the handler was running. */ + timestamp = get_cycles(); desc->handler->end(irq); if (!(desc->status & IRQ_DISABLED)) desc->handler->enable(irq); + desc->cycles += get_cycles() - timestamp; } spin_unlock_irq(&desc->lock); Nur in linux-2.6.12-RT/kernel/irq: manage.c~. diff -ru linux-2.6.12-RT-50-35/kernel/irq/spurious.c linux-2.6.12-RT/kernel/irq/spurious.c --- linux-2.6.12-RT-50-35/kernel/irq/spurious.c 2005-06-30 16:38:20.000000000 +0200 +++ linux-2.6.12-RT/kernel/irq/spurious.c 2005-06-30 15:41:54.000000000 +0200 @@ -74,11 +74,12 @@ } desc->irq_count++; - if (desc->irq_count < 100000) + if (desc->irq_count < 10000) return; desc->irq_count = 0; - if (desc->irqs_unhandled > 99900) { + desc->cycles = 0; + if (desc->irqs_unhandled > 9990) { /* * The interrupt is stuck */ ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 2005-06-30 17:52 ` Karsten Wiese @ 2005-06-30 19:46 ` William Weston 2005-06-30 19:52 ` Ingo Molnar 2005-06-30 20:38 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 Michal Schmidt 2005-06-30 20:50 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Ingo Molnar 1 sibling, 2 replies; 76+ messages in thread From: William Weston @ 2005-06-30 19:46 UTC (permalink / raw) To: Karsten Wiese; +Cc: Ingo Molnar, linux-kernel Hi Ingo, -50-37 wouldn't compile out of the box on my debug config. Here's a couple minor cleanups: --- linux-2.6.12-RT-V0.7.50-37.orig/include/linux/preempt.h 2005-06-30 12:36:37.000000000 -0700 +++ linux-2.6.12-RT-V0.7.50-37/include/linux/preempt.h 2005-06-30 12:23:38.000000000 -0700 @@ -8,6 +8,7 @@ #include <linux/config.h> #include <linux/linkage.h> +#include <linux/thread_info.h> #if defined(CONFIG_DEBUG_PREEMPT) || defined(CONFIG_CRITICAL_TIMING) extern void notrace add_preempt_count(unsigned int val); --- linux-2.6.12-RT-V0.7.50-37.orig/kernel/rt.c 2005-06-30 12:36:37.000000000 -0700 +++ linux-2.6.12-RT-V0.7.50-37/kernel/rt.c 2005-06-30 12:30:55.000000000 -0700 @@ -450,7 +450,7 @@ #ifdef CONFIG_DEBUG_PREEMPT if (task->lock_count) { static int once = 1; - if (once > 0) + if (once > 0) { once--; printk("BUG: nonzero lock count %d at exit time?\n", task->lock_count); @@ -925,7 +925,7 @@ static int capture_lock(struct rt_mutex_waiter *waiter, struct task_struct *task) { - struct thread_info *ti = current_thread_info(); + //struct thread_info *ti = current_thread_info(); struct rt_mutex *lock = waiter->lock; unsigned long flags; int ret = 0; @@ -1411,7 +1411,7 @@ */ static void __up_mutex(struct rt_mutex *lock, int save_state __EIP_DECL__) { - struct thread_info *ti = current_thread_info(); + //struct thread_info *ti = current_thread_info(); struct task_struct *old_owner, *new_owner; struct rt_mutex_waiter *w; unsigned long flags; FYI, a diff against include/linux/compile.h also made it into the the -50-37 patch. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 2005-06-30 19:46 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 William Weston @ 2005-06-30 19:52 ` Ingo Molnar 2005-06-30 20:48 ` Ingo Molnar 2005-06-30 22:15 ` Gene Heskett 2005-06-30 20:38 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 Michal Schmidt 1 sibling, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-06-30 19:52 UTC (permalink / raw) To: William Weston; +Cc: Karsten Wiese, linux-kernel * William Weston <weston@sysex.net> wrote: > Hi Ingo, > > -50-37 wouldn't compile out of the box on my debug config. > Here's a couple minor cleanups: thanks, applied. except these: > - struct thread_info *ti = current_thread_info(); > + //struct thread_info *ti = current_thread_info(); > - struct thread_info *ti = current_thread_info(); > + //struct thread_info *ti = current_thread_info(); they are needed in the nondebug build - i'll fix this up. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 2005-06-30 19:52 ` Ingo Molnar @ 2005-06-30 20:48 ` Ingo Molnar 2005-06-30 22:17 ` Gene Heskett 2005-06-30 22:15 ` Gene Heskett 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-30 20:48 UTC (permalink / raw) To: William Weston; +Cc: Karsten Wiese, linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > > * William Weston <weston@sysex.net> wrote: > > > Hi Ingo, > > > > -50-37 wouldn't compile out of the box on my debug config. > > Here's a couple minor cleanups: > > thanks, applied. [...] i've uploaded -39 with your fixes and other fixes - could you check that it compiles cleanly for you now? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 2005-06-30 20:48 ` Ingo Molnar @ 2005-06-30 22:17 ` Gene Heskett 0 siblings, 0 replies; 76+ messages in thread From: Gene Heskett @ 2005-06-30 22:17 UTC (permalink / raw) To: linux-kernel; +Cc: Ingo Molnar, William Weston, Karsten Wiese On Thursday 30 June 2005 16:48, Ingo Molnar wrote: >* Ingo Molnar <mingo@elte.hu> wrote: >> * William Weston <weston@sysex.net> wrote: >> > Hi Ingo, >> > >> > -50-37 wouldn't compile out of the box on my debug config. >> > Here's a couple minor cleanups: >> >> thanks, applied. [...] > >i've uploaded -39 with your fixes and other fixes - could you check > that it compiles cleanly for you now? > > Ingo I'll go try this one after dinner, we been w/o power here for about 3 hours. The only machine that rebooted ok was the one NOT on a ups, but running the LVM (FC4 box). Interesting. >- >To unsubscribe from this list: send the line "unsubscribe > linux-kernel" in the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html >Please read the FAQ at http://www.tux.org/lkml/ -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.35% setiathome rank, not too shabby for a WV hillbilly Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 2005-06-30 19:52 ` Ingo Molnar 2005-06-30 20:48 ` Ingo Molnar @ 2005-06-30 22:15 ` Gene Heskett 1 sibling, 0 replies; 76+ messages in thread From: Gene Heskett @ 2005-06-30 22:15 UTC (permalink / raw) To: linux-kernel; +Cc: Ingo Molnar, William Weston, Karsten Wiese On Thursday 30 June 2005 15:52, Ingo Molnar wrote: >* William Weston <weston@sysex.net> wrote: >> Hi Ingo, >> >> -50-37 wouldn't compile out of the box on my debug config. >> Here's a couple minor cleanups: > >thanks, applied. except these: >> - struct thread_info *ti = current_thread_info(); >> + //struct thread_info *ti = current_thread_info(); >> >> - struct thread_info *ti = current_thread_info(); >> + //struct thread_info *ti = current_thread_info(); > >they are needed in the nondebug build - i'll fix this up. > > Ingo >- Humm, I can't get the -37 to apply, something about the wrong -p0/-p1 option to patch, but neither one finds the file to patch. PEBKAC? or what Ingo? >To unsubscribe from this list: send the line "unsubscribe > linux-kernel" in the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html >Please read the FAQ at http://www.tux.org/lkml/ -- Cheers, Gene "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) 99.35% setiathome rank, not too shabby for a WV hillbilly Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2005 by Maurice Eugene Heskett, all rights reserved. ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 2005-06-30 19:46 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 William Weston 2005-06-30 19:52 ` Ingo Molnar @ 2005-06-30 20:38 ` Michal Schmidt 2005-06-30 20:47 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Michal Schmidt @ 2005-06-30 20:38 UTC (permalink / raw) To: William Weston; +Cc: Karsten Wiese, Ingo Molnar, linux-kernel [-- Attachment #1: Type: text/plain, Size: 98 bytes --] Here are some more fixes for -RT-2.6.12-final-V0.7.50-38 needed to compile it on x86_64. Michal [-- Attachment #2: rt-amd64-PER_CPU_LOCKED.diff --] [-- Type: text/plain, Size: 1203 bytes --] diff -Nurp -X dontdiff.new linux-RT/arch/x86_64/mm/init.c linux-RT.mich/arch/x86_64/mm/init.c --- linux-RT/arch/x86_64/mm/init.c 2005-06-30 21:22:33.000000000 +0200 +++ linux-RT.mich/arch/x86_64/mm/init.c 2005-06-30 22:28:16.000000000 +0200 @@ -47,7 +47,7 @@ extern int swiotlb; extern char _stext[]; -DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); +DEFINE_PER_CPU_LOCKED(struct mmu_gather, mmu_gathers); /* * NOTE: pagetable_init alloc all the fixmap pagetables contiguous on the diff -Nurp -X dontdiff.new linux-RT/include/asm-x86_64/percpu.h linux-RT.mich/include/asm-x86_64/percpu.h --- linux-RT/include/asm-x86_64/percpu.h 2005-06-30 21:44:37.000000000 +0200 +++ linux-RT.mich/include/asm-x86_64/percpu.h 2005-06-30 22:26:10.000000000 +0200 @@ -63,6 +63,9 @@ extern void setup_per_cpu_areas(void); #endif /* SMP */ #define DECLARE_PER_CPU(type, name) extern __typeof__(type) per_cpu__##name +#define DECLARE_PER_CPU_LOCKED(type, name) \ + extern spinlock_t per_cpu_lock__##name##_locked; \ + extern __typeof__(type) per_cpu__##name##_locked #define EXPORT_PER_CPU_SYMBOL(var) EXPORT_SYMBOL(per_cpu__##var) #define EXPORT_PER_CPU_SYMBOL_GPL(var) EXPORT_SYMBOL_GPL(per_cpu__##var) ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 2005-06-30 20:38 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 Michal Schmidt @ 2005-06-30 20:47 ` Ingo Molnar 0 siblings, 0 replies; 76+ messages in thread From: Ingo Molnar @ 2005-06-30 20:47 UTC (permalink / raw) To: Michal Schmidt; +Cc: William Weston, Karsten Wiese, linux-kernel * Michal Schmidt <xschmi00@stud.feec.vutbr.cz> wrote: > Here are some more fixes for -RT-2.6.12-final-V0.7.50-38 needed to > compile it on x86_64. thanks - i've put them into the -39 patch. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 17:52 ` Karsten Wiese 2005-06-30 19:46 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 William Weston @ 2005-06-30 20:50 ` Ingo Molnar 2005-06-30 22:27 ` Karsten Wiese 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-06-30 20:50 UTC (permalink / raw) To: Karsten Wiese; +Cc: William Weston, linux-kernel * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > Here come some numbers to back up the usefullness of > CONFIG_X86_UP_IOAPIC_FAST. (and to show that my patch actually works > ;-)) All measurement where taken on an UP Athlon64 2Ghz running 32bit > 2.6.12-RT-50-35 PREEMPT_RT on a K8T800 mobo. thanks - the numbers are pretty convincing. I've applied most of your patch (except the instrumentation bits), and it seems to work quite well - one of my testsystems that had interrupt storms before can now run IOAPIC_FAST. (i also enabled the option to be selectable for SMP kernels too. If things work out fine we can make it default-on.) I've uploaded the -50-39 patch with these changes included. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 20:50 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Ingo Molnar @ 2005-06-30 22:27 ` Karsten Wiese 2005-06-30 22:59 ` William Weston ` (2 more replies) 0 siblings, 3 replies; 76+ messages in thread From: Karsten Wiese @ 2005-06-30 22:27 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel Am Donnerstag, 30. Juni 2005 22:50 schrieb Ingo Molnar: > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > Here come some numbers to back up the usefullness of > > CONFIG_X86_UP_IOAPIC_FAST. (and to show that my patch actually works > > ;-)) All measurement where taken on an UP Athlon64 2Ghz running 32bit > > 2.6.12-RT-50-35 PREEMPT_RT on a K8T800 mobo. > > thanks - the numbers are pretty convincing. I've applied most of your > patch (except the instrumentation bits), and it seems to work quite well > - one of my testsystems that had interrupt storms before can now run > IOAPIC_FAST. (i also enabled the option to be selectable for SMP kernels > too. If things work out fine we can make it default-on.) I've uploaded > the -50-39 patch with these changes included. > Ooops, you didn't apply this part of the patch: <snip> --- linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c 2005-06-30 16:38:00.000000000 +0200 +++ linux-2.6.12-RT/arch/i386/kernel/mpparse.c 2005-06-29 20:30:50.000000000 +0200 @@ -263,6 +263,7 @@ return; } mp_ioapics[nr_ioapics] = *m; + io_apic_base[nr_ioapics] = IO_APIC_BASE(nr_ioapics); nr_ioapics++; } @@ -914,6 +915,7 @@ mp_ioapics[idx].mpc_apicaddr = address; set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address); + io_apic_base[idx] = IO_APIC_BASE(idx); mp_ioapics[idx].mpc_apicid = io_apic_get_unique_id(idx, id); mp_ioapics[idx].mpc_apicver = io_apic_get_version(idx); </snip> It provides the precalculation of the ioapics's virtual address. Or does IO_APIC_BASE() compile to an indexed lookup? then io_apic_base[] wouldn't be needed... Karsten ___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 22:27 ` Karsten Wiese @ 2005-06-30 22:59 ` William Weston 2005-07-01 1:00 ` William Weston 2005-07-01 0:15 ` Karsten Wiese 2005-07-01 7:18 ` Ingo Molnar 2 siblings, 1 reply; 76+ messages in thread From: William Weston @ 2005-06-30 22:59 UTC (permalink / raw) To: Karsten Wiese; +Cc: Ingo Molnar, linux-kernel On Fri, 1 Jul 2005, Karsten Wiese wrote: > > <snip> > --- linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c 2005-06-30 16:38:00.000000000 +0200 > +++ linux-2.6.12-RT/arch/i386/kernel/mpparse.c 2005-06-29 20:30:50.000000000 +0200 > @@ -263,6 +263,7 @@ > return; > } > mp_ioapics[nr_ioapics] = *m; > + io_apic_base[nr_ioapics] = IO_APIC_BASE(nr_ioapics); > nr_ioapics++; > } > > @@ -914,6 +915,7 @@ > mp_ioapics[idx].mpc_apicaddr = address; > > set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address); > + io_apic_base[idx] = IO_APIC_BASE(idx); > mp_ioapics[idx].mpc_apicid = io_apic_get_unique_id(idx, id); > mp_ioapics[idx].mpc_apicver = io_apic_get_version(idx); > </snip> > > It provides the precalculation of the ioapics's virtual address. > Or does IO_APIC_BASE() compile to an indexed lookup? > then io_apic_base[] wouldn't be needed... Thanks, Karsten. This patch was needed to get my Xeon/HT (i865) to boot -50-39. Next I'll see if IOAPIC_FAST will work. I'll keep you posted. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 22:59 ` William Weston @ 2005-07-01 1:00 ` William Weston 0 siblings, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-01 1:00 UTC (permalink / raw) Cc: Karsten Wiese, Ingo Molnar, linux-kernel On Thu, 30 Jun 2005, William Weston wrote: > On Fri, 1 Jul 2005, Karsten Wiese wrote: > > --- linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c 2005-06-30 16:38:00.000000000 +0200 > > +++ linux-2.6.12-RT/arch/i386/kernel/mpparse.c 2005-06-29 20:30:50.000000000 +0200 > > @@ -263,6 +263,7 @@ > > return; > > } > > mp_ioapics[nr_ioapics] = *m; > > + io_apic_base[nr_ioapics] = IO_APIC_BASE(nr_ioapics); > > nr_ioapics++; > > } > > > > @@ -914,6 +915,7 @@ > > mp_ioapics[idx].mpc_apicaddr = address; > > > > set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address); > > + io_apic_base[idx] = IO_APIC_BASE(idx); > > mp_ioapics[idx].mpc_apicid = io_apic_get_unique_id(idx, id); > > mp_ioapics[idx].mpc_apicver = io_apic_get_version(idx); > > Thanks, Karsten. This patch was needed to get my Xeon/HT (i865) to boot > -50-39. Next I'll see if IOAPIC_FAST will work. I'll keep you posted. After applying this patch to -50-39, IOAPIC_FAST now works on both the Xeon/HT and the non-HT Xeon boxes (i865 and i845 respectively). Wakeup latencies are looking better, especially on the Xeon/HT (max of 53us so far, as compared to 183us with -50-37 without IOAPIC_FAST). I'll test this out on the Athlon box (which hasn't done well with IOAPIC_FAST previously) when I get home. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 22:27 ` Karsten Wiese 2005-06-30 22:59 ` William Weston @ 2005-07-01 0:15 ` Karsten Wiese 2005-07-01 7:18 ` Ingo Molnar 2 siblings, 0 replies; 76+ messages in thread From: Karsten Wiese @ 2005-07-01 0:15 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel, Gene Heskett Am Freitag, 1. Juli 2005 00:27 schrieb Karsten Wiese: > Am Donnerstag, 30. Juni 2005 22:50 schrieb Ingo Molnar: > > > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > > > > > Here come some numbers to back up the usefullness of > > > CONFIG_X86_UP_IOAPIC_FAST. (and to show that my patch actually works > > > ;-)) All measurement where taken on an UP Athlon64 2Ghz running 32bit > > > 2.6.12-RT-50-35 PREEMPT_RT on a K8T800 mobo. > > > > thanks - the numbers are pretty convincing. I've applied most of your > > patch (except the instrumentation bits), and it seems to work quite well > > - one of my testsystems that had interrupt storms before can now run > > IOAPIC_FAST. (i also enabled the option to be selectable for SMP kernels > > too. If things work out fine we can make it default-on.) I've uploaded > > the -50-39 patch with these changes included. > > > Ooops, you didn't apply this part of the patch: > > <snip> > --- linux-2.6.12-RT-50-35/arch/i386/kernel/mpparse.c 2005-06-30 16:38:00.000000000 +0200 > +++ linux-2.6.12-RT/arch/i386/kernel/mpparse.c 2005-06-29 20:30:50.000000000 +0200 > @@ -263,6 +263,7 @@ > return; > } > mp_ioapics[nr_ioapics] = *m; > + io_apic_base[nr_ioapics] = IO_APIC_BASE(nr_ioapics); > nr_ioapics++; > } > > @@ -914,6 +915,7 @@ > mp_ioapics[idx].mpc_apicaddr = address; > > set_fixmap_nocache(FIX_IO_APIC_BASE_0 + idx, address); > + io_apic_base[idx] = IO_APIC_BASE(idx); > mp_ioapics[idx].mpc_apicid = io_apic_get_unique_id(idx, id); > mp_ioapics[idx].mpc_apicver = io_apic_get_version(idx); > </snip> > > It provides the precalculation of the ioapics's virtual address. > Or does IO_APIC_BASE() compile to an indexed lookup? > then io_apic_base[] wouldn't be needed... > It is needed (only for optimization): There are ioapics that have baseaddresses not on a page boundary. Another idea: to merge the virtual ioapic base addresses into the io_apic_cache struct to use the cpu-cache more efficiently ;-) Karsten ___________________________________________________________ Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-06-30 22:27 ` Karsten Wiese 2005-06-30 22:59 ` William Weston 2005-07-01 0:15 ` Karsten Wiese @ 2005-07-01 7:18 ` Ingo Molnar 2005-07-01 19:34 ` Chuck Harding 2005-07-02 1:46 ` William Weston 2 siblings, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-01 7:18 UTC (permalink / raw) To: Karsten Wiese; +Cc: William Weston, linux-kernel * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > Ooops, you didn't apply this part of the patch: oops - i had it in my tree (so all my tests passed), but it escaped the -39 patch. I've uploaded -50-40 with this included too. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-01 7:18 ` Ingo Molnar @ 2005-07-01 19:34 ` Chuck Harding 2005-07-02 1:46 ` William Weston 1 sibling, 0 replies; 76+ messages in thread From: Chuck Harding @ 2005-07-01 19:34 UTC (permalink / raw) To: Linux Kernel Discussion List On Fri, 1 Jul 2005, Ingo Molnar wrote: > > * Karsten Wiese <annabellesgarden@yahoo.de> wrote: > >> Ooops, you didn't apply this part of the patch: > > oops - i had it in my tree (so all my tests passed), but it escaped the > -39 patch. I've uploaded -50-40 with this included too. > > Ingo > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > Well, the -50-42 still has the problem with sox locking up too tightly to kill. Just saw that the sox processes are all in state 'D', FWIW. -- Charles D. (Chuck) Harding <charding@llnl.gov> Voice: 925-423-8879 Senior Computer Associate ICCD Fax: 925-423-6961 Lawrence Livermore National Laboratory Computation Directorate Livermore, CA USA http://www.llnl.gov GPG Public Key ID: B9EB6601 ------------------ http://tinyurl.com/5w5ey ----------------------- -- Bad breath is better than no breath. -- ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-01 7:18 ` Ingo Molnar 2005-07-01 19:34 ` Chuck Harding @ 2005-07-02 1:46 ` William Weston 2005-07-02 2:02 ` Lee Revell 2005-07-03 14:04 ` Ingo Molnar 1 sibling, 2 replies; 76+ messages in thread From: William Weston @ 2005-07-02 1:46 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Fri, 1 Jul 2005, Ingo Molnar wrote: > oops - i had it in my tree (so all my tests passed), but it escaped the > -39 patch. I've uploaded -50-40 with this included too. FWIW, I'm still seeing the SMT scheduling? meltdown issues with -50-42. Running two instances of 'dd if=/dev/zero of=/dev/null bs=65536' instead of 'burnP6' results in the same behavior. Here's a quick recap: - Start (or login to ) X. - Start an X app that constantly updates the screen, like wmcube, or vlc. - Run 'burnP6' or 'dd if=/dev/zero of=/dev/null bs=65536'. - Run trace-it. Trace completes without any troubles. - Run another 'burnP6' or 'dd if=/dev/zero of=/dev/null bs=65536'. At this point, most of the system is unresponsive: - Most X apps are frozen (even top in its own xterm). - Mouse lost synchro serio warnings show up on serial console. - Serial console is otherwise unresponsive (no SysRq). - X server quits responding to keyboard input. - Kbd input makes mouse temporarily unresponsive (for .1 to >5 secs). - Mouse immediately after kbd triggers more 'mouse lost synchro' messages. - Networking is lost (box won't respond to pings). - Any script automating starting burnP6 or dd and then trace-it hangs. A few things are left working (but not enough to get the system back): - Mouse pointer (movements are chunky) and window focus. - Mouse scroll wheel can still scroll xterms and switch workspaces. - SysRq-B All of this behavior is 100% reproduceable on recent realtime SMP/SMT kernels, and is _not_ seen with vanilla SMP/SMT, Fedora SMP, or realtime UP kernels. I'll be running realtime UP kernels on this Xeon/HT box for a while unless someone gives me reason to enable SMT again. You can tell me to shut up now.... at least until I decide to hunt down another corner case ;-} Best Regards, --William Weston ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-02 1:46 ` William Weston @ 2005-07-02 2:02 ` Lee Revell 2005-07-04 8:53 ` William Weston 2005-07-03 14:04 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Lee Revell @ 2005-07-02 2:02 UTC (permalink / raw) To: William Weston; +Cc: Ingo Molnar, linux-kernel On Fri, 2005-07-01 at 18:46 -0700, William Weston wrote: > FWIW, I'm still seeing the SMT scheduling? meltdown issues with > -50-42. > Running two instances of 'dd if=/dev/zero of=/dev/null bs=65536' > instead > of 'burnP6' results in the same behavior. Here's a quick recap: > > - Start (or login to ) X. > - Start an X app that constantly updates the screen, like wmcube, or > vlc. Which video driver is X using? What nice value is the X server running at? Does adding: Option "NoAccel" to the Device section of your X config file make any difference? (on most systems X is the only thing besides the kernel that can access hardware directly, which can cause problems) Lee ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-02 2:02 ` Lee Revell @ 2005-07-04 8:53 ` William Weston 2005-07-04 11:16 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-04 8:53 UTC (permalink / raw) To: Lee Revell; +Cc: Ingo Molnar, linux-kernel On Fri, 1 Jul 2005, Lee Revell wrote: > On Fri, 2005-07-01 at 18:46 -0700, William Weston wrote: > > FWIW, I'm still seeing the SMT scheduling? meltdown issues with > > -50-42. > > Running two instances of 'dd if=/dev/zero of=/dev/null bs=65536' > > instead > > of 'burnP6' results in the same behavior. Here's a quick recap: > > > > - Start (or login to ) X. > > - Start an X app that constantly updates the screen, like wmcube, or > > vlc. > > Which video driver is X using? What nice value is the X server running > at? Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump the priority up? To realtime? > Does adding: > > Option "NoAccel" > > to the Device section of your X config file make any difference? Disabling the dri and drm modules didn't help. I'll turn on NoAccel when I'm back in the office on Tuesday. > (on most systems X is the only thing besides the kernel that can access > hardware directly, which can cause problems) So would running X through the framebuffer device be the way to go for stability under realtime? It's been a couple years since I've used fb. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-04 8:53 ` William Weston @ 2005-07-04 11:16 ` Ingo Molnar 2005-07-13 1:18 ` Lee Revell 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-04 11:16 UTC (permalink / raw) To: William Weston; +Cc: Lee Revell, linux-kernel * William Weston <weston@sysex.net> wrote: > > Which video driver is X using? What nice value is the X server running > > at? > > Hardware is Intel 82865G (integrated) with DRM i915 1.1.0 20040405 and > xorg-3.8.2 i810 driver, running at nice 0, priority 15. Should I bump > the priority up? To realtime? i'd first suggest to try the latest kernel, before changing your X config - i think the bug might have been fixed meanwhile. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-04 11:16 ` Ingo Molnar @ 2005-07-13 1:18 ` Lee Revell 2005-07-13 1:28 ` Lee Revell 0 siblings, 1 reply; 76+ messages in thread From: Lee Revell @ 2005-07-13 1:18 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: > i'd first suggest to try the latest kernel, before changing your X > config - i think the bug might have been fixed meanwhile. I have found that heavy network traffic really kills the interactive performance. In the top excerpt below, gtk-gnutella is generating about 320KB/sec of traffic. These priorities do not look right: 14 root -50 -5 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [IRQ 9] irqd 686 root -49 -5 0 0 S 0.0 0.0 87:58.18 0 0 0 0 [IRQ 8] irqd 689 root -48 -5 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [IRQ 7] irqd 694 root -47 -5 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [IRQ 12] irqd 714 root -46 -5 0 0 S 2.5 0.0 5:25.33 0 0 0 0 [IRQ 15] irqd 731 root -45 -5 0 0 S 0.0 0.0 0:04.74 0 0 0 0 [IRQ 1] irqd 3 root -44 -10 0 0 S 0.0 0.0 1:42.55 0 0 0 0 [softirq-timer/0] ksoftirqd 5 root -44 -10 0 0 S 2.5 0.0 5:06.49 0 0 0 0 [softirq-net-rx/] ksoftirqd 1541 root -44 -5 0 0 S 0.0 0.0 9:39.03 0 0 0 0 [IRQ 10] irqd 5650 rlrevell -44 0 2804 1616 S 14.8 0.4 5:52.83 1188 52 784 1 /usr/lib/gamin/gam_server stext 30313 rlrevell -44 0 24572 19m S 99.9 4.4 68:15.42 4520 1800 12m 31 gtk-gnutella stext 2285 root -43 -5 0 0 S 0.0 0.0 3:47.34 0 0 0 0 [IRQ 11] irqd 2807 root -42 -5 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [IRQ 14] irqd 9 root -2 -5 0 0 S 0.0 0.0 0:16.05 0 0 0 0 [events/0] worker_th 4 root 5 -10 0 0 S 0.0 0.0 0:25.95 0 0 0 0 [softirq-net-tx/] ksoftirqd 7 root 5 -10 0 0 S 0.0 0.0 0:01.22 0 0 0 0 [softirq-tasklet] ksoftirqd 8 root 5 -10 0 0 S 0.0 0.0 0:01.30 0 0 0 0 [desched/0] desched_t 28988 root 5 -10 106m 30m S 4.9 6.9 242:59.04 76m 1556 32m 3 /usr/X11R6/bin/X :0 -audit 0 -auth /var/lib/gd stext 2 root 10 -10 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [softirq-high/0] ksoftirqd 6 root 10 -10 0 0 S 0.0 0.0 0:00.00 0 0 0 0 [softirq-scsi/0] Why do gtk-gnutella and gam_server get a higher priority than the disk (14) and network (11) IRQs? And shouldn't the softirq threads run at lower priority than all the hardirq threads? Looking at the top output I'd expect the machine to be crawling, and that's what it does. Lee ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-13 1:18 ` Lee Revell @ 2005-07-13 1:28 ` Lee Revell 2005-07-13 6:35 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Lee Revell @ 2005-07-13 1:28 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel On Tue, 2005-07-12 at 21:18 -0400, Lee Revell wrote: > On Mon, 2005-07-04 at 13:16 +0200, Ingo Molnar wrote: > > i'd first suggest to try the latest kernel, before changing your X > > config - i think the bug might have been fixed meanwhile. > > I have found that heavy network traffic really kills the interactive > performance. In the top excerpt below, gtk-gnutella is generating about > 320KB/sec of traffic. > > These priorities do not look right: Never mind, I just rediscovered the RT priority leakage bug. Lee ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-13 1:28 ` Lee Revell @ 2005-07-13 6:35 ` Ingo Molnar 2005-07-13 17:20 ` Lee Revell 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-13 6:35 UTC (permalink / raw) To: Lee Revell; +Cc: William Weston, linux-kernel * Lee Revell <rlrevell@joe-job.com> wrote: > > I have found that heavy network traffic really kills the interactive > > performance. In the top excerpt below, gtk-gnutella is generating about > > 320KB/sec of traffic. > > > > These priorities do not look right: > > Never mind, I just rediscovered the RT priority leakage bug. ok. And is it fixed by recent kernels? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-13 6:35 ` Ingo Molnar @ 2005-07-13 17:20 ` Lee Revell 0 siblings, 0 replies; 76+ messages in thread From: Lee Revell @ 2005-07-13 17:20 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel On Wed, 2005-07-13 at 08:35 +0200, Ingo Molnar wrote: > * Lee Revell <rlrevell@joe-job.com> wrote: > > > > I have found that heavy network traffic really kills the interactive > > > performance. In the top excerpt below, gtk-gnutella is generating about > > > 320KB/sec of traffic. > > > > > > These priorities do not look right: > > > > Never mind, I just rediscovered the RT priority leakage bug. > > ok. And is it fixed by recent kernels? > Yes, I have not seen the problem yet with V0.7.51-28. Lee ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-02 1:46 ` William Weston 2005-07-02 2:02 ` Lee Revell @ 2005-07-03 14:04 ` Ingo Molnar 2005-07-03 18:12 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-03 14:04 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > FWIW, I'm still seeing the SMT scheduling? meltdown issues with > -50-42. Running two instances of 'dd if=/dev/zero of=/dev/null > bs=65536' instead of 'burnP6' results in the same behavior. Here's a > quick recap: > > - Start (or login to ) X. > - Start an X app that constantly updates the screen, like wmcube, or vlc. > - Run 'burnP6' or 'dd if=/dev/zero of=/dev/null bs=65536'. > - Run trace-it. Trace completes without any troubles. > - Run another 'burnP6' or 'dd if=/dev/zero of=/dev/null bs=65536'. > > At this point, most of the system is unresponsive: > > - Most X apps are frozen (even top in its own xterm). > - Mouse lost synchro serio warnings show up on serial console. > - Serial console is otherwise unresponsive (no SysRq). > - X server quits responding to keyboard input. > - Kbd input makes mouse temporarily unresponsive (for .1 to >5 secs). > - Mouse immediately after kbd triggers more 'mouse lost synchro' messages. > - Networking is lost (box won't respond to pings). > - Any script automating starting burnP6 or dd and then trace-it hangs. > > A few things are left working (but not enough to get the system back): > > - Mouse pointer (movements are chunky) and window focus. > - Mouse scroll wheel can still scroll xterms and switch workspaces. > - SysRq-B hm, i can reproduce a variant of this, by starting enough 'dd' tasks. (it needed more than two on a 2-way/4-way HT testbox though) Indeed everything seems to be starved, but SysRq still worked so i was able to SysRq-kIll all tasks and thus the system recovered. i'm debugging this now. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-03 14:04 ` Ingo Molnar @ 2005-07-03 18:12 ` Ingo Molnar 2005-07-05 20:43 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 William Weston 2005-07-07 1:26 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 William Weston 0 siblings, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-03 18:12 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > > A few things are left working (but not enough to get the system back): > > > > - Mouse pointer (movements are chunky) and window focus. > > - Mouse scroll wheel can still scroll xterms and switch workspaces. > > - SysRq-B > > hm, i can reproduce a variant of this, by starting enough 'dd' tasks. > (it needed more than two on a 2-way/4-way HT testbox though) Indeed > everything seems to be starved, but SysRq still worked so i was able > to SysRq-kIll all tasks and thus the system recovered. > > i'm debugging this now. ok, found a bug that could explain the situation: mutex sleeps+wakeups were incorrectly credited as 'interactive sleep' periods, causing the dd processes to be boosted incorrectly. The dd processes created a workload in which they blocked each other in such a pattern that they got boosted periodically, starving pretty much every other task. the fix is significant and affects alot of workloads, and should further improve interactivity in noticeable ways. I'm not 100% sure it solves all the starvation problems (e.g. how could normal-prio dd tasks starve the SCHED_FIFO irq threads that drove SysRq?), but the results so far look promising. i've uploaded the -50-45 patch, can you under this kernel trigger a 'meltdown' on your SMT box? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-03 18:12 ` Ingo Molnar @ 2005-07-05 20:43 ` William Weston 2005-07-06 10:04 ` Ingo Molnar 2005-07-07 1:26 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 William Weston 1 sibling, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-05 20:43 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Sun, 3 Jul 2005, Ingo Molnar wrote: > > i've uploaded the -50-45 patch, can you under this kernel trigger a > 'meltdown' on your SMT box? Hi Ingo, Here's the results of trying out everything from -50-45 through -51-01 on the SMT box: -50-47 looks better. 4x burnP6 + wmcube doesn't bring the box to a halt anymore. Running VLC along with a kernel compile brought all running processes to a halt. SysRq-I couldn't recover the system (unless I just wasn't patient enough), but SysRq-S + SysRq-U + SysRq-B was able to sync, unmount, and reboot. -50-45 and -50-46 seemed to have similar behavior. -50-48 won't boot: softirq-timer/0/4[CPU#0]: BUG in __up_mutex at kernel/rt.c:1485 [<c0103ec7>] dump_stack+0x23/0x25 (20) [<c0121580>] __WARN_ON+0x66/0x86 (52) [<c013bd49>] __up_mutex+0x14e/0x304 (40) [<c013c4a3>] up_mutex+0x39/0xe3 (28) [<c0131aca>] delayed_work_timer_fn+0x3a/0x41 (24) [<c012a7f6>] run_timer_softirq+0x208/0x46d (60) [<c0126833>] ksoftirqd+0xee/0x1ad (36) [<c0136786>] kthread+0xb6/0xba (44) [<c01010c5>] kernel_thread_helper+0x5/0xb (554565660) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c012153b>] .. ( <= __WARN_ON+0x21/0x86) .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c013ffc4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (softirq-timer/0/4 [def10030, 98]): ------------------------------ ACPI: Processor [CPU1] (supports 8 throttling states) ACPI: Processor [CPU2] (supports 8 throttling states) khelper/163[CPU#1]: BUG in set_new_owner at kernel/rt.c:822 NMI watchdog detected lockup on CPU#1 (50000/50000) Pid: 163, comm: khelper EIP: 0060:[<c013b7c4>] CPU: 1 EIP is at __down_trylock+0x88/0x369 EFLAGS: 00000082 Not tainted (2.6.12-RT-V0.7.50-48-debug) EAX: c03da258 EBX: c03da244 ECX: c03da250 EDX: c03da268 ESI: c03da268 EDI: 00000000 EBP: dec69d1c DS: 007b ES: 007b CR0: 8005003b CR2: 00000000 CR3: 004f8000 CR4: 000006d0 [<c01010b6>] show_regs+0x164/0x16e (36) [<c0112f94>] nmi_watchdog_tick+0x189/0x2b7 (52) [<c0104e80>] default_do_nmi+0x7a/0x18a (52) [<c0104ff4>] do_nmi+0x5d/0x6f (24) [<c0103b52>] nmi_stack_correct+0x1d/0x22 (92) [<c013c84f>] rt_down_trylock+0x29/0x179 (36) [<c0120e42>] vprintk+0x22e/0x2c3 (116) [<c0120c12>] printk+0x20/0x22 (20) [<c012157b>] __WARN_ON+0x61/0x86 (52) [<c037173b>] __down_mutex+0x4e2/0x689 (108) [<c037258b>] _spin_lock_irqsave+0x24/0x89 (32) [<c0129ccf>] lock_timer_base+0x1f/0x3a (20) [<c0129fa5>] try_to_del_timer_sync+0x1f/0x5f (28) [<c012a00f>] del_timer_sync+0x2a/0xb1 (44) [<c0135d06>] exit_itimers+0x117/0x1a9 (56) [<c012b66a>] __exit_signal+0xfd/0x1c8 (32) [<c012210e>] release_task+0x81/0x16a (40) [<c01232a8>] exit_notify+0x64f/0x972 (60) [<c0123816>] do_exit+0x24b/0x40d (44) [<c013178b>] wait_for_helper+0x0/0xb0 (24) [<c01010c5>] kernel_thread_helper+0x5/0xb (557408284) --------------------------- | preempt count: 00010004 ] | 4-level deep critical section nesting: ---------------------------------------- .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c012153b>] .. ( <= __WARN_ON+0x21/0x86) .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c0120c3a>] .. ( <= vprintk+0x26/0x2c3) .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c0112f5f>] .. ( <= nmi_watchdog_tick+0x154/0x2b7) .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c013ffc4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (khelper/163 [dec67930, 115]): ------------------------------ NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 22, comm: khelper EIP: 0060:[<c0371315>] CPU: 0 EIP is at __down_mutex+0xbc/0x689 EFLAGS: 00000082 Not tainted (2.6.12-RT-V0.7.50-48-debug) EAX: c14cde94 EBX: 00000000 ECX: 00000000 EDX: c14cdde8 ESI: c14cde80 EDI: c14cdea4 EBP: c14cde04 DS: 007b ES: 007b CR0: 8005003b CR2: fffdf000 CR3: 004f8000 CR4: 000006d0 [<c01010b6>] show_regs+0x164/0x16e (36) [<c0112f94>] nmi_watchdog_tick+0x189/0x2b7 (52) [<c0104e80>] default_do_nmi+0x7a/0x18a (52) [<c0104ff4>] do_nmi+0x5d/0x6f (24) [<c0103b52>] nmi_stack_correct+0x1d/0x22 (160) [<c03724cc>] _spin_lock_irq+0x24/0x7b (28) [<c03701aa>] wait_for_completion+0xa6/0xdb (48) [<c011f7a4>] do_fork+0xf1/0x1e2 (132) [<c0101151>] kernel_thread+0x86/0x8c (96) [<c01318a1>] __call_usermodehelper+0x66/0x68 (24) [<c0131d2f>] worker_thread+0x1a1/0x266 (108) [<c0136786>] kthread+0xb6/0xba (44) [<c01010c5>] kernel_thread_helper+0x5/0xb (1051926556) --------------------------- | preempt count: 00010002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c0112f5f>] .. ( <= nmi_watchdog_tick+0x154/0x2b7) .. [<c013efaa>] .... add_preempt_count+0x1c/0x1e .....[<c013ffc4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (khelper/22 [c14c32f0, 114]): ------------------------------ -50-49 through -50-51 won't compile on my SMT debug config: arch/i386/kernel/vm86.c:701: error: syntax error before ',' token -51-01 won't boot: softirq-timer/1/13[CPU#1]: BUG in ____up_mutex at kernel/rt.c:1302 [<c0103ea7>] dump_stack+0x23/0x25 (20) [<c0121410>] __WARN_ON+0x66/0x86 (52) [<c013ca44>] up_mutex+0x160/0x3b6 (36) [<c013195a>] delayed_work_timer_fn+0x3a/0x41 (24) [<c012a686>] run_timer_softirq+0x208/0x46d (60) [<c01266c3>] ksoftirqd+0xee/0x1ad (36) [<c0136616>] kthread+0xb6/0xba (44) [<c01010c5>] kernel_thread_helper+0x5/0xb (1052131356) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c01402ca>] .... add_preempt_count+0x1c/0x1e .....[<c01213cb>] .. ( <= __WARN_ON+0x21/0x86) .. [<c01402ca>] .... add_preempt_count+0x1c/0x1e .....[<c01412e4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (softirq-timer/1/13 [c1490670, 98]): ------------------------------ ACPI: Processor [CPU1] (supports 8 throttling states) ACPI: Processor [CPU2] (supports 8 throttling states) khelper/22[CPU#1]: BUG in set_new_owner at kernel/rt.c:869 NMI watchdog detected lockup on CPU#1 (50000/50000) BTW, -50-44 through -50-51 wouldn't compile on the UP Athlon box. -50-43 appears to be rock-solid under the UP non-debug config. After hammering on it all weekend, Maximum wakeup latency was 14us (from running JACK, phasex, burnK7, dd, and switching from X to text console all at once). --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-05 20:43 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 William Weston @ 2005-07-06 10:04 ` Ingo Molnar 2005-07-06 21:12 ` William Weston ` (2 more replies) 0 siblings, 3 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-06 10:04 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > Here's the results of trying out everything from -50-45 through -51-01 on > the SMT box: thanks for the extensive testing! > -51-01 won't boot: > > softirq-timer/1/13[CPU#1]: BUG in ____up_mutex at kernel/rt.c:1302 i have digged out an older HT-box .config of yours and have reproduced an assert quite similar to the one above. Found one bug in that area: the assert (conditional on RT_DEADLOCK_DETECT) was done a bit too early, i have fixed this in my tree and have uploaded -51-02. > BTW, -50-44 through -50-51 wouldn't compile on the UP Athlon box. do the 51-02 (and later) kernels work on the UP Athlon box? > -50-43 appears to be rock-solid under the UP non-debug config. After > hammering on it all weekend, Maximum wakeup latency was 14us (from > running JACK, phasex, burnK7, dd, and switching from X to text console > all at once). great! Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-06 10:04 ` Ingo Molnar @ 2005-07-06 21:12 ` William Weston 2005-07-07 14:56 ` Steven Rostedt 2005-07-07 19:05 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 William Weston 2 siblings, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-06 21:12 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Wed, 6 Jul 2005, Ingo Molnar wrote: > i have digged out an older HT-box .config of yours and have reproduced > an assert quite similar to the one above. Found one bug in that area: > the assert (conditional on RT_DEADLOCK_DETECT) was done a bit too early, > i have fixed this in my tree and have uploaded -51-02. On -51-06, the vlc+dd test works ok (video is a bit choppy after the third dd instance). With the vlc+burnP6 test, the keyboard is lost first (for everything except SysRq), then the serial console (for everything except SysRq), then the mouse. SysRq-I hangs X, but brings the serial console back for a while. At this point, I should be able to get more debug info than previously. FWIW, I got this stack dump with SysRq-K: SysRq : SAK IRQ 4/307[CPU#0]: BUG in set_palette at drivers/char/vt.c:2918 [<c0103ea7>] dump_stack+0x23/0x25 (20) [<c0121550>] __WARN_ON+0x66/0x86 (52) [<c02975df>] set_palette+0x5a/0x5c (24) [<c02980a8>] __handle_sysrq+0x9a/0x139 (48) [<c029817f>] handle_sysrq+0x38/0x3a (24) [<c02b40bf>] receive_chars+0x250/0x2ec (60) [<c02b4474>] serial8250_nterrupt+0xea/0x101 (44) [<c014b85b>] handle_IRQ_event+0x71/0xef (52) [<c014bfcb>] do_hardirq+0x53/0xda (40) [<c014c128>] do_irqd+0xd6/0x1d4 (40) [<c0136736>] kthread+0xb6/0xba (44) [<c01010c1>] kernel_thread_helper+0x5/0xb (556269596) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c01406ba>] .... add_preempt_count+0x1c/0x1e .....[<c012150b>] .. ( <= __WARN_ON+0x21/0x86) .. [<c01406ba>] .... add_preempt_count+0x1c/0x1e .....[<c01416d4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (IRQ 4/307 [ded7d930, 57]): ----------------------------- #001: [c087b2d0] {&irq_lists[i].lock} ... acquired at: serial8250_interrupt+0x16/0x101 #002: [c087ed40] {&port->lock} ... acquired at: serial8250_interrupt+0x62/0x101 #003: [c04bf380] {sysrq_key_table_lock} ... acquired at: __handle_sysrq+0x2a/0x139 SysRq : Terminate All Tasks SysRq : Kill All Tasks SysRq : Emergency Sync Emergency Sync complete SysRq : Emergency Remount R/O Emergency Remount complete SysRq : SAK IRQ 1/298[CPU#1]: BUG in set_palette at drivers/char/vt.c:2918 [<c0103ea7>] dump_stack+0x23/0x25 (20) [<c0121550>] __WARN_ON+0x66/0x86 (52) [<c02975df>] set_palette+0x5a/0x5c (24) [<c02980a8>] __handle_sysrq+0x9a/0x139 (48) [<c029817f>] handle_sysrq+0x38/0x3a (24) [<c02928d3>] kbd_event+0xb9/0xdc (40) [<c02f2f06>] input_event+0xe6/0x478 (44) [<c02f6c45>] atkbd_report_key+0x3c/0xd4 (32) [<c02f6eb8>] atkbd_interrupt+0x1db/0x65f (56) [<c02adb6a>] serio_interrupt+0x49/0x8f (40) [<c02ae8a6>] i8042_interrupt+0x12a/0x238 (56) [<c014b85b>] handle_IRQ_event+0x71/0xef (52) [<c014bfcb>] do_hardirq+0x53/0xda (40) [<c014c128>] do_irqd+0xd6/0x1d4 (40) [<c0136736>] kthread+0xb6/0xba (44) [<c01010c1>] kernel_thread_helper+0x5/0xb (557096988) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c01406ba>] .... add_preempt_count+0x1c/0x1e .....[<c012150b>] .. ( <= __WARN_ON+0x21/0x86) .. [<c01406ba>] .... add_preempt_count+0x1c/0x1e .....[<c01416d4>] .. ( <= print_traces+0x1b/0x52) ------------------------------ | showing all locks held by: | (IRQ 1/298 [c14c2cb0, 56]): ------------------------------ #001: [dec8404c] {&serio->lock} ... acquired at: serio_interrupt+0x1f/0x8f #002: [c04bf380] {sysrq_key_table_lock} ... acquired at: __handle_sysrq+0x2a/0x139 SysRq : Resetting > > BTW, -50-44 through -50-51 wouldn't compile on the UP Athlon box. Correction: There were a couple that wouldn't compile in there (-50-49 was one of them). -50-48 and -51-01 on the Athlon give me a lot of these with debugging enabled: Jul 6 01:09:08 astarte kernel: delayed_work_timer_fn+0x1d/0x20softirq-timer/0/3[CPU#0]: BUG in ____up_mutex at kernel/rt.c:1302 Jul 6 01:09:08 astarte kernel: [<c0104313>] dump_stack+0x23/0x30 (20) Jul 6 01:09:08 astarte kernel: [<c0120857>] __WARN_ON+0x77/0xc0 (56) Jul 6 01:09:08 astarte kernel: [<c013de4c>] rt_up+0x30c/0x470 (40) Jul 6 01:09:08 astarte kernel: [<c01201d2>] release_console_sem+0xc2/0x140 (36) Jul 6 01:09:08 astarte kernel: [<c011ff10>] vprintk+0x180/0x2a0 (120) Jul 6 01:09:08 astarte kernel: [<c011fd8d>] printk+0x1d/0x20 (16) Jul 6 01:09:08 astarte kernel: [<c01461d3>] __print_symbol+0x93/0x110 (400) Jul 6 01:09:08 astarte kernel: [<c01041e9>] show_trace+0x79/0xe0 (36) Jul 6 01:09:08 astarte kernel: [<c0104313>] dump_stack+0x23/0x30 (20) Jul 6 01:09:08 astarte kernel: [<c0120857>] __WARN_ON+0x77/0xc0 (56) Jul 6 01:09:08 astarte kernel: [<c013c989>] up_mutex+0x1f9/0x470 (36) Jul 6 01:09:08 astarte kernel: [<c0130ecd>] delayed_work_timer_fn+0x1d/0x20 (16) Jul 6 01:09:08 astarte kernel: [<c0129948>] run_timer_softirq+0x1f8/0x420 (56) Jul 6 01:09:08 astarte kernel: [<c012577f>] ksoftirqd+0xcf/0x170 (28) Jul 6 01:09:08 astarte kernel: [<c0135df6>] kthread+0xa6/0xe0 (48) Jul 6 01:09:08 astarte kernel: [<c01013e9>] kernel_thread_helper+0x5/0xc (537083932) Jul 6 01:09:08 astarte kernel: --------------------------- Jul 6 01:09:08 astarte kernel: | preempt count: 00000004 ] Jul 6 01:09:08 astarte kernel: | 4-level deep critical section nesting: Jul 6 01:09:08 astarte kernel: ---------------------------------------- Jul 6 01:09:08 astarte kernel: .. [<c014040c>] .... add_preempt_count+0x1c/0x20 Jul 6 01:09:08 astarte kernel: .....[<c012080f>] .. ( <= __WARN_ON+0x2f/0xc0) Jul 6 01:09:08 astarte kernel: .. [<c013db78>] .... rt_up+0x38/0x470 Jul 6 01:09:08 astarte kernel: .....[<c01201d2>] .. ( <= release_console_sem+0xc2/0x140) Jul 6 01:09:08 astarte kernel: .. [<c014040c>] .... add_preempt_count+0x1c/0x20 Jul 6 01:09:08 astarte kernel: .....[<c012080f>] .. ( <= __WARN_ON+0x2f/0xc0) Jul 6 01:09:08 astarte kernel: .. [<c014040c>] .... add_preempt_count+0x1c/0x20 Jul 6 01:09:08 astarte kernel: .....[<c014133b>] .. ( <= print_traces+0x1b/0x60) Jul 6 01:09:08 astarte kernel: Jul 6 01:09:09 astarte kernel: ------------------------------ Jul 6 01:09:09 astarte kernel: | showing all locks held by: | (softirq-timer/0/3 [dffc2c90, 98]): Jul 6 01:09:09 astarte kernel: ------------------------------ > do the 51-02 (and later) kernels work on the UP Athlon box? I'll try out -51-06 (or later) on the UP Athlon when I get home. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-06 10:04 ` Ingo Molnar 2005-07-06 21:12 ` William Weston @ 2005-07-07 14:56 ` Steven Rostedt 2005-07-07 15:31 ` Ingo Molnar 2005-07-07 19:05 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 William Weston 2 siblings, 1 reply; 76+ messages in thread From: Steven Rostedt @ 2005-07-07 14:56 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel Hi Ingo, I've just downloaded 51-09 and tried running it here on a normal intel pentium4 box here at my customers site. It included some hotplug PCI modules (I don't know why since it's doesn't have hotplug devices) and I got some init_MUTEX_LOCKED bugs on them. Below you will find the patch (I assume that compat_semaphore is still used). Anyway, I also want to let you know that the e100 does not work. It's detected, but it wont bring up DHCP, and when I manually configued it, it just froze (the process not the machine). But when I did a sysrq-t, the machine froze up after it completed with some RT yeilding bug. Here's what was last to spit out: NETDEV WATCHDOG: eth0: transmit timed out BUG: events/0:10 RT task yield()-ing! [<c01042ef>] dump_stack+0x1f/0x30 (20) [<c02e8b1a>] yield+0x5a/0x60 (20) [<c029bec5>] dev_deactivate+0x65/0x70 (20) [<c0296ae8>] linkwatch_run_queue+0xf8/0x110 (40) [<c0296b31>] linkwatch_event+0x31/0x70 (16) [<c013081b>] worker_thread+0x17b/0x230 (124) [<c01353de>] kthread+0xae/0xc0 (48) [<c0101445>] kernel_thread_helper+0x5/0x10 (814882844) --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c014007a>] .... add_preempt_count+0x1a/0x20 .....[<c0140fcd>] .. ( <= print_traces+0x1d/0x60) ------------------------------ | showing all locks held by: | (events/0/10 [cf6ca030, 98]): ------------------------------ #001: [c0390684] {rtnl_sem.lock} ... acquired at: linkwatch_event+0x2c/0x70 This was right after the trace and all else froze. Anyway, I'll debug it tomorrow since I'm about to go home for the day. And here's the patch for the hotplug stuff (sorry about the -p0). Index: drivers/pci/hotplug/ibmphp_hpc.c =================================================================== --- drivers/pci/hotplug/ibmphp_hpc.c (revision 237) +++ drivers/pci/hotplug/ibmphp_hpc.c (working copy) @@ -104,7 +104,7 @@ static struct semaphore sem_hpcaccess; // lock access to HPC static struct semaphore semOperations; // lock all operations and // access to data structures -static struct semaphore sem_exit; // make sure polling thread goes away +static struct compat_semaphore sem_exit; // make sure polling thread goes away //---------------------------------------------------------------------------- // local function prototypes //---------------------------------------------------------------------------- Index: drivers/pci/hotplug/cpqphp_ctrl.c =================================================================== --- drivers/pci/hotplug/cpqphp_ctrl.c (revision 237) +++ drivers/pci/hotplug/cpqphp_ctrl.c (working copy) @@ -45,8 +45,8 @@ u8 behind_bridge, struct resource_lists *resources); static void interrupt_event_handler(struct controller *ctrl); -static struct semaphore event_semaphore; /* mutex for process loop (up if something to process) */ -static struct semaphore event_exit; /* guard ensure thread has exited before calling it quits */ +static struct compat_semaphore event_semaphore; /* mutex for process loop (up if something to process) */ +static struct compat_semaphore event_exit; /* guard ensure thread has exited before calling it quits */ static int event_finished; static unsigned long pushbutton_pending; /* = 0 */ Index: drivers/pci/hotplug/shpchp_ctrl.c =================================================================== --- drivers/pci/hotplug/shpchp_ctrl.c (revision 237) +++ drivers/pci/hotplug/shpchp_ctrl.c (working copy) @@ -47,8 +47,8 @@ u8 behind_bridge, struct resource_lists *resources, u8 bridge_bus, u8 bridge_dev); static void interrupt_event_handler(struct controller *ctrl); -static struct semaphore event_semaphore; /* mutex for process loop (up if something to process) */ -static struct semaphore event_exit; /* guard ensure thread has exited before calling it quits */ +static struct compat_semaphore event_semaphore; /* mutex for process loop (up if something to process) */ +static struct compat_semaphore event_exit; /* guard ensure thread has exited before calling it quits */ static int event_finished; static unsigned long pushbutton_pending; /* = 0 */ Index: drivers/pci/hotplug/cpci_hotplug_core.c =================================================================== --- drivers/pci/hotplug/cpci_hotplug_core.c (revision 237) +++ drivers/pci/hotplug/cpci_hotplug_core.c (working copy) @@ -60,8 +60,8 @@ static atomic_t extracting; int cpci_debug; static struct cpci_hp_controller *controller; -static struct semaphore event_semaphore; /* mutex for process loop (up if something to process) */ -static struct semaphore thread_exit; /* guard ensure thread has exited before calling it quits */ +static struct compat_semaphore event_semaphore; /* mutex for process loop (up if something to process) */ +static struct compat_semaphore thread_exit; /* guard ensure thread has exited before calling it quits */ static int thread_finished = 1; static int enable_slot(struct hotplug_slot *slot); Index: drivers/pci/hotplug/pciehp_ctrl.c =================================================================== --- drivers/pci/hotplug/pciehp_ctrl.c (revision 237) +++ drivers/pci/hotplug/pciehp_ctrl.c (working copy) @@ -48,8 +48,8 @@ u8 behind_bridge, struct resource_lists *resources, u8 bridge_bus, u8 bridge_dev); static void interrupt_event_handler(struct controller *ctrl); -static struct semaphore event_semaphore; /* mutex for process loop (up if something to process) */ -static struct semaphore event_exit; /* guard ensure thread has exited before calling it quits */ +static struct compat_semaphore event_semaphore; /* mutex for process loop (up if something to process) */ +static struct compat_semaphore event_exit; /* guard ensure thread has exited before calling it quits */ static int event_finished; static unsigned long pushbutton_pending; /* = 0 */ static unsigned long surprise_rm_pending; /* = 0 */ ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 14:56 ` Steven Rostedt @ 2005-07-07 15:31 ` Ingo Molnar 2005-07-07 15:42 ` Steven Rostedt 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 15:31 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-kernel * Steven Rostedt <rostedt@goodmis.org> wrote: > Hi Ingo, > > I've just downloaded 51-09 and tried running it here on a normal intel > pentium4 box here at my customers site. It included some hotplug PCI > modules (I don't know why since it's doesn't have hotplug devices) and > I got some init_MUTEX_LOCKED bugs on them. Below you will find the > patch (I assume that compat_semaphore is still used). thanks, i've applied it, and it's now in 51-11. > Anyway, I also want to let you know that the e100 does not work. It's > detected, but it wont bring up DHCP, and when I manually configued it, > it just froze (the process not the machine). But when I did a sysrq-t, > the machine froze up after it completed with some RT yeilding bug. > Here's what was last to spit out: is PCI_MSI enabled by any chance? That is known to break level-triggered IOAPIC irqs and devices. > NETDEV WATCHDOG: eth0: transmit timed out > BUG: events/0:10 RT task yield()-ing! i have replaced this yield() with msleep(1) in -51-11, but that's just fixing the symptom. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 15:31 ` Ingo Molnar @ 2005-07-07 15:42 ` Steven Rostedt 2005-07-07 16:10 ` Steven Rostedt 0 siblings, 1 reply; 76+ messages in thread From: Steven Rostedt @ 2005-07-07 15:42 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel Darn subversion! I just started a massive commit, and I can't leave work till it's done. So you still got me here ;-) On Thu, 7 Jul 2005, Ingo Molnar wrote: > > > Anyway, I also want to let you know that the e100 does not work. It's > > detected, but it wont bring up DHCP, and when I manually configued it, > > it just froze (the process not the machine). But when I did a sysrq-t, > > the machine froze up after it completed with some RT yeilding bug. > > Here's what was last to spit out: > > is PCI_MSI enabled by any chance? That is known to break level-triggered > IOAPIC irqs and devices. > As a matter of fact it is... I'll turn it off now and try it out. If the commit is still going, I'll get you a response about the result. -- Steve ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 15:42 ` Steven Rostedt @ 2005-07-07 16:10 ` Steven Rostedt 2005-07-07 16:48 ` Ingo Molnar 2005-07-07 17:51 ` Ingo Molnar 0 siblings, 2 replies; 76+ messages in thread From: Steven Rostedt @ 2005-07-07 16:10 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Thu, 7 Jul 2005, Steven Rostedt wrote: > > Darn subversion! I just started a massive commit, and I can't leave work > till it's done. So you still got me here ;-) That commit is still going. I can see why subversion was not used for kernel development. > > On Thu, 7 Jul 2005, Ingo Molnar wrote: > > > > > Anyway, I also want to let you know that the e100 does not work. It's > > > detected, but it wont bring up DHCP, and when I manually configued it, > > > it just froze (the process not the machine). But when I did a sysrq-t, > > > the machine froze up after it completed with some RT yeilding bug. > > > Here's what was last to spit out: > > > > is PCI_MSI enabled by any chance? That is known to break level-triggered > > IOAPIC irqs and devices. > > > > As a matter of fact it is... I'll turn it off now and try it out. > If the commit is still going, I'll get you a response about the result. > It did the trick. I got a network. But I also got a hell of a lot of 'enqueued dead tasks'. But stupid me forgot to turn on capture in minicom, and haven't been able to reproduce the problem. I rebooted the machine which blew away all evidence of what occured, and it's now fine. I'll reboot a few more times to see if I can get it to break again. -- Steve ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 16:10 ` Steven Rostedt @ 2005-07-07 16:48 ` Ingo Molnar 2005-07-08 7:02 ` Steven Rostedt 2005-07-08 20:55 ` Bill Davidsen 2005-07-07 17:51 ` Ingo Molnar 1 sibling, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 16:48 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-kernel * Steven Rostedt <rostedt@goodmis.org> wrote: > It did the trick. I got a network. But I also got a hell of a lot of > 'enqueued dead tasks'. But stupid me forgot to turn on capture in > minicom, and haven't been able to reproduce the problem. I rebooted > the machine which blew away all evidence of what occured, and it's now > fine. I'll reboot a few more times to see if I can get it to break > again. minicom has a scrollback feature (Alt-B), does that have the oops in history perhaps? It can cache a couple of bootups typically. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 16:48 ` Ingo Molnar @ 2005-07-08 7:02 ` Steven Rostedt 2005-07-08 9:54 ` Steven Rostedt 2005-07-08 20:55 ` Bill Davidsen 1 sibling, 1 reply; 76+ messages in thread From: Steven Rostedt @ 2005-07-08 7:02 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Thu, 7 Jul 2005, Ingo Molnar wrote: > > * Steven Rostedt <rostedt@goodmis.org> wrote: > > > It did the trick. I got a network. But I also got a hell of a lot of > > 'enqueued dead tasks'. But stupid me forgot to turn on capture in > > minicom, and haven't been able to reproduce the problem. I rebooted > > the machine which blew away all evidence of what occured, and it's now > > fine. I'll reboot a few more times to see if I can get it to break > > again. > > minicom has a scrollback feature (Alt-B), does that have the oops in > history perhaps? It can cache a couple of bootups typically. > Is there also a scrollback for after you log out of minicom and turn off the machine ;-) Thanks, I didn't know about that feature of minicom. I should really start reading the fine manuals. Unfortunately, the minicom was on my laptop which I turned off yesterday and only today did I see your note. To try and reproduce it again, I've added in /etc/rc3.d an S98reboot that will switch the system to runlevel 6 again, and repeat the process over and over. All this while connect to minicom and capturing. Hopefully it will eventually show the same bug. If not, maybe it was just a fluke. -- Steve ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-08 7:02 ` Steven Rostedt @ 2005-07-08 9:54 ` Steven Rostedt 2005-07-08 10:00 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Steven Rostedt @ 2005-07-08 9:54 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Fri, 8 Jul 2005, Steven Rostedt wrote: > > To try and reproduce it again, I've added in /etc/rc3.d an S98reboot that > will switch the system to runlevel 6 again, and repeat the process over > and over. All this while connect to minicom and capturing. Hopefully it > will eventually show the same bug. If not, maybe it was just a fluke. > OK, I just rebooted the machine a 100 times, and not one error. Maybe it was just because the system failed on an earlier boot? I don't know, but I can't spend any more time on it. I'll keep my minicom's active at all times now. -- Steve ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-08 9:54 ` Steven Rostedt @ 2005-07-08 10:00 ` Ingo Molnar 0 siblings, 0 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 10:00 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-kernel * Steven Rostedt <rostedt@goodmis.org> wrote: > > To try and reproduce it again, I've added in /etc/rc3.d an S98reboot that > > will switch the system to runlevel 6 again, and repeat the process over > > and over. All this while connect to minicom and capturing. Hopefully it > > will eventually show the same bug. If not, maybe it was just a fluke. > > OK, I just rebooted the machine a 100 times, and not one error. Maybe > it was just because the system failed on an earlier boot? I don't > know, but I can't spend any more time on it. I'll keep my minicom's > active at all times now. hm, the "dead task enqueued!" is a real message that signals some real nastiness. It should not occur under any circumstance - nor have i seen it during the past couple of months. So lets keep an eye on it, but if 100 reboots didnt trigger it there's not much we can do but wait and see. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 16:48 ` Ingo Molnar 2005-07-08 7:02 ` Steven Rostedt @ 2005-07-08 20:55 ` Bill Davidsen 1 sibling, 0 replies; 76+ messages in thread From: Bill Davidsen @ 2005-07-08 20:55 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel Ingo Molnar wrote: > * Steven Rostedt <rostedt@goodmis.org> wrote: > > >>It did the trick. I got a network. But I also got a hell of a lot of >>'enqueued dead tasks'. But stupid me forgot to turn on capture in >>minicom, and haven't been able to reproduce the problem. I rebooted >>the machine which blew away all evidence of what occured, and it's now >>fine. I'll reboot a few more times to see if I can get it to break >>again. > > > minicom has a scrollback feature (Alt-B), does that have the oops in > history perhaps? It can cache a couple of bootups typically. Although the kermit protocol is not much used anymore, the kermit program is still a capable terminal interface, and includes logging. For those who don't like telnet or minicom, here's another program to dislike. -- bill davidsen <davidsen@tmr.com> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 2005-07-07 16:10 ` Steven Rostedt 2005-07-07 16:48 ` Ingo Molnar @ 2005-07-07 17:51 ` Ingo Molnar 2005-07-07 21:18 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 Chuck Harding 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 17:51 UTC (permalink / raw) To: Steven Rostedt; +Cc: linux-kernel * Steven Rostedt <rostedt@goodmis.org> wrote: > > > is PCI_MSI enabled by any chance? That is known to break level-triggered > > > IOAPIC irqs and devices. > > > > As a matter of fact it is... I'll turn it off now and try it out. > > If the commit is still going, I'll get you a response about the result. > > It did the trick. I got a network. [...] ok, i found the PCI_MSI irq handling bug - the fix is in -51-12. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-07 17:51 ` Ingo Molnar @ 2005-07-07 21:18 ` Chuck Harding 2005-07-08 5:41 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Chuck Harding @ 2005-07-07 21:18 UTC (permalink / raw) To: Ingo Molnar; +Cc: Linux Kernel Discussion List [-- Attachment #1: Type: TEXT/PLAIN, Size: 1378 bytes --] On Thu, 7 Jul 2005, Ingo Molnar wrote: > > * Steven Rostedt <rostedt@goodmis.org> wrote: > >>>> is PCI_MSI enabled by any chance? That is known to break level-triggered >>>> IOAPIC irqs and devices. >>> >>> As a matter of fact it is... I'll turn it off now and try it out. >>> If the commit is still going, I'll get you a response about the result. >> >> It did the trick. I got a network. [...] > > ok, i found the PCI_MSI irq handling bug - the fix is in -51-12. > > Ingo Which still exhibits the lockup of sox. I built a 2.6.12 vanilla kernel using the same .config as I used for 51-12 and the failure did not happen. just the process of booting up causes later invocations of sox to lockup in the D state. If I don't login to X and just run from a VT I can get it to lockup by running something like: $ for ((i=0;i<20;i++)) > do > echo -ne "\r$i " > (play /usr/share/sounds/KDE_Beep_ShortBeep.wav &) > done after about 14 or so iterations of the loop and thereafter no more sound can be played. (my .config is attached) -- Charles D. (Chuck) Harding <charding@llnl.gov> Voice: 925-423-8879 Senior Computer Associate ICCD Fax: 925-423-6961 Lawrence Livermore National Laboratory Computation Directorate Livermore, CA USA http://www.llnl.gov GPG Public Key ID: B9EB6601 ------------------ http://tinyurl.com/5w5ey ----------------------- [-- Attachment #2: config file for 51-12 --] [-- Type: TEXT/PLAIN, Size: 32813 bytes --] # # Automatically generated make config: don't edit # Linux kernel version: 2.6.12-RT-V0.7.51-12 # Thu Jul 7 12:25:34 2005 # CONFIG_X86=y CONFIG_MMU=y CONFIG_UID16=y CONFIG_GENERIC_ISA_DMA=y CONFIG_GENERIC_IOMAP=y # # Code maturity level options # CONFIG_EXPERIMENTAL=y CONFIG_CLEAN_COMPILE=y CONFIG_BROKEN_ON_SMP=y CONFIG_LOCK_KERNEL=y CONFIG_INIT_ENV_ARG_LIMIT=32 # # General setup # CONFIG_LOCALVERSION="" CONFIG_SWAP=y CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y CONFIG_BSD_PROCESS_ACCT=y # CONFIG_BSD_PROCESS_ACCT_V3 is not set CONFIG_SYSCTL=y CONFIG_AUDIT=y CONFIG_AUDITSYSCALL=y CONFIG_HOTPLUG=y CONFIG_KOBJECT_UEVENT=y CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y # CONFIG_EMBEDDED is not set CONFIG_KALLSYMS=y # CONFIG_KALLSYMS_ALL is not set CONFIG_KALLSYMS_EXTRA_PASS=y CONFIG_PRINTK=y CONFIG_BUG=y CONFIG_BASE_FULL=y CONFIG_FUTEX=y CONFIG_EPOLL=y CONFIG_SHMEM=y CONFIG_CC_ALIGN_FUNCTIONS=0 CONFIG_CC_ALIGN_LABELS=0 CONFIG_CC_ALIGN_LOOPS=0 CONFIG_CC_ALIGN_JUMPS=0 # CONFIG_TINY_SHMEM is not set CONFIG_BASE_SMALL=0 # # Loadable module support # CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y # CONFIG_MODULE_FORCE_UNLOAD is not set CONFIG_OBSOLETE_MODPARM=y CONFIG_MODVERSIONS=y # CONFIG_MODULE_SRCVERSION_ALL is not set CONFIG_KMOD=y # # Processor type and features # CONFIG_X86_PC=y # CONFIG_X86_ELAN is not set # CONFIG_X86_VOYAGER is not set # CONFIG_X86_NUMAQ is not set # CONFIG_X86_SUMMIT is not set # CONFIG_X86_BIGSMP is not set # CONFIG_X86_VISWS is not set # CONFIG_X86_GENERICARCH is not set # CONFIG_X86_ES7000 is not set # CONFIG_M386 is not set # CONFIG_M486 is not set # CONFIG_M586 is not set # CONFIG_M586TSC is not set # CONFIG_M586MMX is not set # CONFIG_M686 is not set # CONFIG_MPENTIUMII is not set # CONFIG_MPENTIUMIII is not set # CONFIG_MPENTIUMM is not set CONFIG_MPENTIUM4=y # CONFIG_MK6 is not set # CONFIG_MK7 is not set # CONFIG_MK8 is not set # CONFIG_MCRUSOE is not set # CONFIG_MEFFICEON is not set # CONFIG_MWINCHIPC6 is not set # CONFIG_MWINCHIP2 is not set # CONFIG_MWINCHIP3D is not set # CONFIG_MGEODEGX1 is not set # CONFIG_MCYRIXIII is not set # CONFIG_MVIAC3_2 is not set CONFIG_X86_GENERIC=y CONFIG_X86_CMPXCHG=y CONFIG_X86_XADD=y CONFIG_X86_L1_CACHE_SHIFT=7 CONFIG_GENERIC_CALIBRATE_DELAY=y CONFIG_X86_WP_WORKS_OK=y CONFIG_X86_INVLPG=y CONFIG_X86_BSWAP=y CONFIG_X86_POPAD_OK=y CONFIG_X86_GOOD_APIC=y CONFIG_X86_INTEL_USERCOPY=y CONFIG_X86_USE_PPRO_CHECKSUM=y CONFIG_HPET_TIMER=y CONFIG_HPET_EMULATE_RTC=y # CONFIG_SMP is not set # CONFIG_PREEMPT_NONE is not set # CONFIG_PREEMPT_VOLUNTARY is not set # CONFIG_PREEMPT_DESKTOP is not set CONFIG_PREEMPT_RT=y CONFIG_PREEMPT=y CONFIG_PREEMPT_SOFTIRQS=y CONFIG_PREEMPT_HARDIRQS=y CONFIG_PREEMPT_RCU=y CONFIG_PREEMPT_BKL=y CONFIG_RWSEM_GENERIC_SPINLOCK=y CONFIG_ASM_SEMAPHORES=y CONFIG_X86_UP_APIC=y CONFIG_X86_UP_IOAPIC=y CONFIG_X86_LOCAL_APIC=y CONFIG_X86_IO_APIC=y CONFIG_X86_IOAPIC_FAST=y CONFIG_X86_TSC=y CONFIG_X86_MCE=y # CONFIG_X86_MCE_NONFATAL is not set CONFIG_X86_MCE_P4THERMAL=y # CONFIG_TOSHIBA is not set # CONFIG_I8K is not set # CONFIG_X86_REBOOTFIXUPS is not set CONFIG_MICROCODE=m CONFIG_X86_MSR=m CONFIG_X86_CPUID=m # # Firmware Drivers # CONFIG_EDD=m # CONFIG_NOHIGHMEM is not set CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y CONFIG_HIGHPTE=y # CONFIG_MATH_EMULATION is not set CONFIG_MTRR=y # CONFIG_EFI is not set CONFIG_HAVE_DEC_LOCK=y CONFIG_REGPARM=y CONFIG_SECCOMP=y # # Power management options (ACPI, APM) # CONFIG_PM=y # CONFIG_PM_DEBUG is not set # CONFIG_SOFTWARE_SUSPEND is not set # # ACPI (Advanced Configuration and Power Interface) Support # CONFIG_ACPI=y CONFIG_ACPI_BOOT=y CONFIG_ACPI_INTERPRETER=y CONFIG_ACPI_SLEEP=y CONFIG_ACPI_SLEEP_PROC_FS=y # CONFIG_ACPI_AC is not set # CONFIG_ACPI_BATTERY is not set CONFIG_ACPI_BUTTON=y CONFIG_ACPI_VIDEO=y CONFIG_ACPI_FAN=y CONFIG_ACPI_PROCESSOR=y CONFIG_ACPI_THERMAL=y # CONFIG_ACPI_ASUS is not set # CONFIG_ACPI_IBM is not set # CONFIG_ACPI_TOSHIBA is not set CONFIG_ACPI_BLACKLIST_YEAR=2001 # CONFIG_ACPI_DEBUG is not set CONFIG_ACPI_BUS=y CONFIG_ACPI_EC=y CONFIG_ACPI_POWER=y CONFIG_ACPI_PCI=y CONFIG_ACPI_SYSTEM=y CONFIG_X86_PM_TIMER=y # CONFIG_ACPI_CONTAINER is not set # # APM (Advanced Power Management) BIOS Support # # CONFIG_APM is not set # # CPU Frequency scaling # CONFIG_CPU_FREQ=y CONFIG_CPU_FREQ_TABLE=y # CONFIG_CPU_FREQ_DEBUG is not set CONFIG_CPU_FREQ_STAT=y # CONFIG_CPU_FREQ_STAT_DETAILS is not set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y # CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set CONFIG_CPU_FREQ_GOV_PERFORMANCE=y CONFIG_CPU_FREQ_GOV_POWERSAVE=m CONFIG_CPU_FREQ_GOV_USERSPACE=m CONFIG_CPU_FREQ_GOV_ONDEMAND=m CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m # # CPUFreq processor drivers # CONFIG_X86_ACPI_CPUFREQ=y # CONFIG_X86_POWERNOW_K6 is not set # CONFIG_X86_POWERNOW_K7 is not set # CONFIG_X86_POWERNOW_K8 is not set # CONFIG_X86_GX_SUSPMOD is not set CONFIG_X86_SPEEDSTEP_CENTRINO=y CONFIG_X86_SPEEDSTEP_CENTRINO_ACPI=y CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y CONFIG_X86_SPEEDSTEP_ICH=y CONFIG_X86_SPEEDSTEP_SMI=y CONFIG_X86_P4_CLOCKMOD=y # CONFIG_X86_CPUFREQ_NFORCE2 is not set # CONFIG_X86_LONGRUN is not set # CONFIG_X86_LONGHAUL is not set # # shared options # # CONFIG_X86_ACPI_CPUFREQ_PROC_INTF is not set CONFIG_X86_SPEEDSTEP_LIB=y # CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK is not set # # Bus options (PCI, PCMCIA, EISA, MCA, ISA) # CONFIG_PCI=y # CONFIG_PCI_GOBIOS is not set # CONFIG_PCI_GOMMCONFIG is not set # CONFIG_PCI_GODIRECT is not set CONFIG_PCI_GOANY=y CONFIG_PCI_BIOS=y CONFIG_PCI_DIRECT=y CONFIG_PCI_MMCONFIG=y # CONFIG_PCIEPORTBUS is not set # CONFIG_PCI_MSI is not set CONFIG_PCI_LEGACY_PROC=y # CONFIG_PCI_NAMES is not set # CONFIG_PCI_DEBUG is not set CONFIG_ISA_DMA_API=y # CONFIG_ISA is not set # CONFIG_MCA is not set # CONFIG_SCx200 is not set # # PCCARD (PCMCIA/CardBus) support # # CONFIG_PCCARD is not set # # PCI Hotplug Support # # CONFIG_HOTPLUG_PCI is not set # # Executable file formats # CONFIG_BINFMT_ELF=y CONFIG_BINFMT_AOUT=m CONFIG_BINFMT_MISC=y # # Device Drivers # # # Generic Driver Options # CONFIG_STANDALONE=y CONFIG_PREVENT_FIRMWARE_BUILD=y CONFIG_FW_LOADER=m # CONFIG_DEBUG_DRIVER is not set # # Memory Technology Devices (MTD) # CONFIG_MTD=m # CONFIG_MTD_DEBUG is not set # CONFIG_MTD_CONCAT is not set # CONFIG_MTD_PARTITIONS is not set # # User Modules And Translation Layers # # CONFIG_MTD_CHAR is not set # CONFIG_MTD_BLOCK is not set # CONFIG_MTD_BLOCK_RO is not set # CONFIG_FTL is not set # CONFIG_NFTL is not set # CONFIG_INFTL is not set # # RAM/ROM/Flash chip drivers # # CONFIG_MTD_CFI is not set # CONFIG_MTD_JEDECPROBE is not set CONFIG_MTD_MAP_BANK_WIDTH_1=y CONFIG_MTD_MAP_BANK_WIDTH_2=y CONFIG_MTD_MAP_BANK_WIDTH_4=y # CONFIG_MTD_MAP_BANK_WIDTH_8 is not set # CONFIG_MTD_MAP_BANK_WIDTH_16 is not set # CONFIG_MTD_MAP_BANK_WIDTH_32 is not set CONFIG_MTD_CFI_I1=y CONFIG_MTD_CFI_I2=y # CONFIG_MTD_CFI_I4 is not set # CONFIG_MTD_CFI_I8 is not set # CONFIG_MTD_RAM is not set # CONFIG_MTD_ROM is not set # CONFIG_MTD_ABSENT is not set # # Mapping drivers for chip access # # CONFIG_MTD_COMPLEX_MAPPINGS is not set # # Self-contained MTD device drivers # # CONFIG_MTD_PMC551 is not set # CONFIG_MTD_SLRAM is not set # CONFIG_MTD_PHRAM is not set # CONFIG_MTD_MTDRAM is not set # CONFIG_MTD_BLKMTD is not set # CONFIG_MTD_BLOCK2MTD is not set # # Disk-On-Chip Device Drivers # # CONFIG_MTD_DOC2000 is not set # CONFIG_MTD_DOC2001 is not set # CONFIG_MTD_DOC2001PLUS is not set # # NAND Flash Device Drivers # # CONFIG_MTD_NAND is not set # # Parallel port support # CONFIG_PARPORT=m CONFIG_PARPORT_PC=m CONFIG_PARPORT_SERIAL=m # CONFIG_PARPORT_PC_FIFO is not set # CONFIG_PARPORT_PC_SUPERIO is not set # CONFIG_PARPORT_GSC is not set CONFIG_PARPORT_1284=y # # Plug and Play support # # CONFIG_PNP is not set # # Block devices # # CONFIG_BLK_DEV_FD is not set # CONFIG_PARIDE is not set # CONFIG_BLK_CPQ_DA is not set # CONFIG_BLK_CPQ_CISS_DA is not set # CONFIG_BLK_DEV_DAC960 is not set # CONFIG_BLK_DEV_UMEM is not set # CONFIG_BLK_DEV_COW_COMMON is not set CONFIG_BLK_DEV_LOOP=m CONFIG_BLK_DEV_CRYPTOLOOP=m CONFIG_BLK_DEV_NBD=m # CONFIG_BLK_DEV_SX8 is not set # CONFIG_BLK_DEV_UB is not set CONFIG_BLK_DEV_RAM=y CONFIG_BLK_DEV_RAM_COUNT=16 CONFIG_BLK_DEV_RAM_SIZE=16384 CONFIG_BLK_DEV_INITRD=y CONFIG_INITRAMFS_SOURCE="" CONFIG_LBD=y # CONFIG_CDROM_PKTCDVD is not set # # IO Schedulers # CONFIG_IOSCHED_NOOP=y CONFIG_IOSCHED_AS=y CONFIG_IOSCHED_DEADLINE=y CONFIG_IOSCHED_CFQ=y # CONFIG_ATA_OVER_ETH is not set # # ATA/ATAPI/MFM/RLL support # CONFIG_IDE=y CONFIG_BLK_DEV_IDE=y # # Please see Documentation/ide.txt for help/info on IDE drives # # CONFIG_BLK_DEV_IDE_SATA is not set # CONFIG_BLK_DEV_HD_IDE is not set CONFIG_BLK_DEV_IDEDISK=y CONFIG_IDEDISK_MULTI_MODE=y CONFIG_BLK_DEV_IDECD=y # CONFIG_BLK_DEV_IDETAPE is not set CONFIG_BLK_DEV_IDEFLOPPY=y # CONFIG_IDE_TASK_IOCTL is not set # # IDE chipset support/bugfixes # CONFIG_IDE_GENERIC=y # CONFIG_BLK_DEV_CMD640 is not set CONFIG_BLK_DEV_IDEPCI=y CONFIG_IDEPCI_SHARE_IRQ=y # CONFIG_BLK_DEV_OFFBOARD is not set CONFIG_BLK_DEV_GENERIC=y # CONFIG_BLK_DEV_OPTI621 is not set # CONFIG_BLK_DEV_RZ1000 is not set CONFIG_BLK_DEV_IDEDMA_PCI=y # CONFIG_BLK_DEV_IDEDMA_FORCED is not set CONFIG_IDEDMA_PCI_AUTO=y # CONFIG_IDEDMA_ONLYDISK is not set # CONFIG_BLK_DEV_AEC62XX is not set # CONFIG_BLK_DEV_ALI15X3 is not set # CONFIG_BLK_DEV_AMD74XX is not set # CONFIG_BLK_DEV_ATIIXP is not set # CONFIG_BLK_DEV_CMD64X is not set # CONFIG_BLK_DEV_TRIFLEX is not set # CONFIG_BLK_DEV_CY82C693 is not set # CONFIG_BLK_DEV_CS5520 is not set # CONFIG_BLK_DEV_CS5530 is not set # CONFIG_BLK_DEV_HPT34X is not set # CONFIG_BLK_DEV_HPT366 is not set # CONFIG_BLK_DEV_SC1200 is not set CONFIG_BLK_DEV_PIIX=y # CONFIG_BLK_DEV_NS87415 is not set # CONFIG_BLK_DEV_PDC202XX_OLD is not set # CONFIG_BLK_DEV_PDC202XX_NEW is not set # CONFIG_BLK_DEV_SVWKS is not set # CONFIG_BLK_DEV_SIIMAGE is not set # CONFIG_BLK_DEV_SIS5513 is not set # CONFIG_BLK_DEV_SLC90E66 is not set # CONFIG_BLK_DEV_TRM290 is not set # CONFIG_BLK_DEV_VIA82CXXX is not set # CONFIG_IDE_ARM is not set CONFIG_BLK_DEV_IDEDMA=y # CONFIG_IDEDMA_IVB is not set CONFIG_IDEDMA_AUTO=y # CONFIG_BLK_DEV_HD is not set # # SCSI device support # # CONFIG_SCSI is not set # # Multi-device support (RAID and LVM) # # CONFIG_MD is not set # # Fusion MPT device support # # # IEEE 1394 (FireWire) support # # CONFIG_IEEE1394 is not set # # I2O device support # # CONFIG_I2O is not set # # Networking support # CONFIG_NET=y # # Networking options # CONFIG_PACKET=y CONFIG_PACKET_MMAP=y CONFIG_UNIX=y CONFIG_NET_KEY=m CONFIG_INET=y CONFIG_IP_MULTICAST=y CONFIG_IP_ADVANCED_ROUTER=y CONFIG_IP_MULTIPLE_TABLES=y CONFIG_IP_ROUTE_FWMARK=y CONFIG_IP_ROUTE_MULTIPATH=y # CONFIG_IP_ROUTE_MULTIPATH_CACHED is not set CONFIG_IP_ROUTE_VERBOSE=y # CONFIG_IP_PNP is not set CONFIG_NET_IPIP=m CONFIG_NET_IPGRE=m CONFIG_NET_IPGRE_BROADCAST=y CONFIG_IP_MROUTE=y CONFIG_IP_PIMSM_V1=y CONFIG_IP_PIMSM_V2=y # CONFIG_ARPD is not set CONFIG_SYN_COOKIES=y CONFIG_INET_AH=m CONFIG_INET_ESP=m CONFIG_INET_IPCOMP=m CONFIG_INET_TUNNEL=m CONFIG_IP_TCPDIAG=y # CONFIG_IP_TCPDIAG_IPV6 is not set # # IP: Virtual Server Configuration # CONFIG_IP_VS=m # CONFIG_IP_VS_DEBUG is not set CONFIG_IP_VS_TAB_BITS=12 # # IPVS transport protocol load balancing support # CONFIG_IP_VS_PROTO_TCP=y CONFIG_IP_VS_PROTO_UDP=y CONFIG_IP_VS_PROTO_ESP=y CONFIG_IP_VS_PROTO_AH=y # # IPVS scheduler # CONFIG_IP_VS_RR=m CONFIG_IP_VS_WRR=m CONFIG_IP_VS_LC=m CONFIG_IP_VS_WLC=m CONFIG_IP_VS_LBLC=m CONFIG_IP_VS_LBLCR=m CONFIG_IP_VS_DH=m CONFIG_IP_VS_SH=m CONFIG_IP_VS_SED=m CONFIG_IP_VS_NQ=m # # IPVS application helper # CONFIG_IP_VS_FTP=m CONFIG_IPV6=m CONFIG_IPV6_PRIVACY=y CONFIG_INET6_AH=m CONFIG_INET6_ESP=m CONFIG_INET6_IPCOMP=m CONFIG_INET6_TUNNEL=m CONFIG_IPV6_TUNNEL=m CONFIG_NETFILTER=y # CONFIG_NETFILTER_DEBUG is not set CONFIG_BRIDGE_NETFILTER=y # # IP: Netfilter Configuration # CONFIG_IP_NF_CONNTRACK=m CONFIG_IP_NF_CT_ACCT=y # CONFIG_IP_NF_CONNTRACK_MARK is not set CONFIG_IP_NF_CT_PROTO_SCTP=m CONFIG_IP_NF_FTP=m CONFIG_IP_NF_IRC=m CONFIG_IP_NF_TFTP=m CONFIG_IP_NF_AMANDA=m CONFIG_IP_NF_QUEUE=m CONFIG_IP_NF_IPTABLES=m CONFIG_IP_NF_MATCH_LIMIT=m CONFIG_IP_NF_MATCH_IPRANGE=m CONFIG_IP_NF_MATCH_MAC=m CONFIG_IP_NF_MATCH_PKTTYPE=m CONFIG_IP_NF_MATCH_MARK=m CONFIG_IP_NF_MATCH_MULTIPORT=m CONFIG_IP_NF_MATCH_TOS=m CONFIG_IP_NF_MATCH_RECENT=m CONFIG_IP_NF_MATCH_ECN=m CONFIG_IP_NF_MATCH_DSCP=m CONFIG_IP_NF_MATCH_AH_ESP=m CONFIG_IP_NF_MATCH_LENGTH=m CONFIG_IP_NF_MATCH_TTL=m CONFIG_IP_NF_MATCH_TCPMSS=m CONFIG_IP_NF_MATCH_HELPER=m CONFIG_IP_NF_MATCH_STATE=m CONFIG_IP_NF_MATCH_CONNTRACK=m CONFIG_IP_NF_MATCH_OWNER=m CONFIG_IP_NF_MATCH_PHYSDEV=m CONFIG_IP_NF_MATCH_ADDRTYPE=m CONFIG_IP_NF_MATCH_REALM=m CONFIG_IP_NF_MATCH_SCTP=m CONFIG_IP_NF_MATCH_COMMENT=m # CONFIG_IP_NF_MATCH_HASHLIMIT is not set CONFIG_IP_NF_FILTER=m CONFIG_IP_NF_TARGET_REJECT=m CONFIG_IP_NF_TARGET_LOG=m CONFIG_IP_NF_TARGET_ULOG=m CONFIG_IP_NF_TARGET_TCPMSS=m CONFIG_IP_NF_NAT=m CONFIG_IP_NF_NAT_NEEDED=y CONFIG_IP_NF_TARGET_MASQUERADE=m CONFIG_IP_NF_TARGET_REDIRECT=m CONFIG_IP_NF_TARGET_NETMAP=m CONFIG_IP_NF_TARGET_SAME=m CONFIG_IP_NF_NAT_SNMP_BASIC=m CONFIG_IP_NF_NAT_IRC=m CONFIG_IP_NF_NAT_FTP=m CONFIG_IP_NF_NAT_TFTP=m CONFIG_IP_NF_NAT_AMANDA=m CONFIG_IP_NF_MANGLE=m CONFIG_IP_NF_TARGET_TOS=m CONFIG_IP_NF_TARGET_ECN=m CONFIG_IP_NF_TARGET_DSCP=m CONFIG_IP_NF_TARGET_MARK=m CONFIG_IP_NF_TARGET_CLASSIFY=m CONFIG_IP_NF_RAW=m CONFIG_IP_NF_TARGET_NOTRACK=m CONFIG_IP_NF_ARPTABLES=m CONFIG_IP_NF_ARPFILTER=m CONFIG_IP_NF_ARP_MANGLE=m # # IPv6: Netfilter Configuration (EXPERIMENTAL) # # CONFIG_IP6_NF_QUEUE is not set CONFIG_IP6_NF_IPTABLES=m CONFIG_IP6_NF_MATCH_LIMIT=m CONFIG_IP6_NF_MATCH_MAC=m CONFIG_IP6_NF_MATCH_RT=m CONFIG_IP6_NF_MATCH_OPTS=m CONFIG_IP6_NF_MATCH_FRAG=m CONFIG_IP6_NF_MATCH_HL=m CONFIG_IP6_NF_MATCH_MULTIPORT=m CONFIG_IP6_NF_MATCH_OWNER=m CONFIG_IP6_NF_MATCH_MARK=m CONFIG_IP6_NF_MATCH_IPV6HEADER=m CONFIG_IP6_NF_MATCH_AHESP=m CONFIG_IP6_NF_MATCH_LENGTH=m CONFIG_IP6_NF_MATCH_EUI64=m CONFIG_IP6_NF_MATCH_PHYSDEV=m CONFIG_IP6_NF_FILTER=m CONFIG_IP6_NF_TARGET_LOG=m CONFIG_IP6_NF_MANGLE=m CONFIG_IP6_NF_TARGET_MARK=m CONFIG_IP6_NF_RAW=m # # Bridge: Netfilter Configuration # CONFIG_BRIDGE_NF_EBTABLES=m CONFIG_BRIDGE_EBT_BROUTE=m CONFIG_BRIDGE_EBT_T_FILTER=m CONFIG_BRIDGE_EBT_T_NAT=m CONFIG_BRIDGE_EBT_802_3=m CONFIG_BRIDGE_EBT_AMONG=m CONFIG_BRIDGE_EBT_ARP=m CONFIG_BRIDGE_EBT_IP=m CONFIG_BRIDGE_EBT_LIMIT=m CONFIG_BRIDGE_EBT_MARK=m CONFIG_BRIDGE_EBT_PKTTYPE=m CONFIG_BRIDGE_EBT_STP=m CONFIG_BRIDGE_EBT_VLAN=m CONFIG_BRIDGE_EBT_ARPREPLY=m CONFIG_BRIDGE_EBT_DNAT=m CONFIG_BRIDGE_EBT_MARK_T=m CONFIG_BRIDGE_EBT_REDIRECT=m CONFIG_BRIDGE_EBT_SNAT=m CONFIG_BRIDGE_EBT_LOG=m # CONFIG_BRIDGE_EBT_ULOG is not set CONFIG_XFRM=y CONFIG_XFRM_USER=y # # SCTP Configuration (EXPERIMENTAL) # CONFIG_IP_SCTP=m # CONFIG_SCTP_DBG_MSG is not set # CONFIG_SCTP_DBG_OBJCNT is not set # CONFIG_SCTP_HMAC_NONE is not set # CONFIG_SCTP_HMAC_SHA1 is not set CONFIG_SCTP_HMAC_MD5=y CONFIG_ATM=m CONFIG_ATM_CLIP=m # CONFIG_ATM_CLIP_NO_ICMP is not set CONFIG_ATM_LANE=m # CONFIG_ATM_MPOA is not set CONFIG_ATM_BR2684=m # CONFIG_ATM_BR2684_IPFILTER is not set CONFIG_BRIDGE=m CONFIG_VLAN_8021Q=m # CONFIG_DECNET is not set # CONFIG_LLC2 is not set # CONFIG_IPX is not set # CONFIG_ATALK is not set # CONFIG_X25 is not set # CONFIG_LAPB is not set CONFIG_NET_DIVERT=y # CONFIG_ECONET is not set # CONFIG_WAN_ROUTER is not set # # QoS and/or fair queueing # CONFIG_NET_SCHED=y CONFIG_NET_SCH_CLK_JIFFIES=y # CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set # CONFIG_NET_SCH_CLK_CPU is not set CONFIG_NET_SCH_CBQ=m CONFIG_NET_SCH_HTB=m CONFIG_NET_SCH_HFSC=m CONFIG_NET_SCH_ATM=m CONFIG_NET_SCH_PRIO=m CONFIG_NET_SCH_RED=m CONFIG_NET_SCH_SFQ=m CONFIG_NET_SCH_TEQL=m CONFIG_NET_SCH_TBF=m CONFIG_NET_SCH_GRED=m CONFIG_NET_SCH_DSMARK=m CONFIG_NET_SCH_NETEM=m CONFIG_NET_SCH_INGRESS=m CONFIG_NET_QOS=y CONFIG_NET_ESTIMATOR=y CONFIG_NET_CLS=y # CONFIG_NET_CLS_BASIC is not set CONFIG_NET_CLS_TCINDEX=m CONFIG_NET_CLS_ROUTE4=m CONFIG_NET_CLS_ROUTE=y CONFIG_NET_CLS_FW=m CONFIG_NET_CLS_U32=m CONFIG_CLS_U32_PERF=y CONFIG_NET_CLS_IND=y # CONFIG_CLS_U32_MARK is not set CONFIG_NET_CLS_RSVP=m CONFIG_NET_CLS_RSVP6=m # CONFIG_NET_EMATCH is not set # CONFIG_NET_CLS_ACT is not set CONFIG_NET_CLS_POLICE=y # # Network testing # # CONFIG_NET_PKTGEN is not set # CONFIG_NETPOLL is not set # CONFIG_NET_POLL_CONTROLLER is not set # CONFIG_HAMRADIO is not set # CONFIG_IRDA is not set # CONFIG_BT is not set CONFIG_NETDEVICES=y CONFIG_DUMMY=m # CONFIG_BONDING is not set # CONFIG_EQUALIZER is not set # CONFIG_TUN is not set # # ARCnet devices # # CONFIG_ARCNET is not set # # Ethernet (10 or 100Mbit) # CONFIG_NET_ETHERNET=y CONFIG_MII=m # CONFIG_HAPPYMEAL is not set # CONFIG_SUNGEM is not set CONFIG_NET_VENDOR_3COM=y CONFIG_VORTEX=m CONFIG_TYPHOON=m # # Tulip family network device support # # CONFIG_NET_TULIP is not set # CONFIG_HP100 is not set # CONFIG_NET_PCI is not set # # Ethernet (1000 Mbit) # # CONFIG_ACENIC is not set # CONFIG_DL2K is not set # CONFIG_E1000 is not set # CONFIG_NS83820 is not set # CONFIG_HAMACHI is not set # CONFIG_YELLOWFIN is not set # CONFIG_R8169 is not set # CONFIG_SK98LIN is not set # CONFIG_TIGON3 is not set # CONFIG_BNX2 is not set # # Ethernet (10000 Mbit) # # CONFIG_IXGB is not set # CONFIG_S2IO is not set # # Token Ring devices # # CONFIG_TR is not set # # Wireless LAN (non-hamradio) # # CONFIG_NET_RADIO is not set # # Wan interfaces # # CONFIG_WAN is not set # # ATM drivers # # CONFIG_ATM_TCP is not set # CONFIG_ATM_LANAI is not set # CONFIG_ATM_ENI is not set # CONFIG_ATM_FIRESTREAM is not set # CONFIG_ATM_ZATM is not set # CONFIG_ATM_NICSTAR is not set # CONFIG_ATM_IDT77252 is not set # CONFIG_ATM_AMBASSADOR is not set # CONFIG_ATM_HORIZON is not set # CONFIG_ATM_IA is not set # CONFIG_ATM_FORE200E_MAYBE is not set # CONFIG_ATM_HE is not set # CONFIG_FDDI is not set # CONFIG_HIPPI is not set # CONFIG_PLIP is not set # CONFIG_PPP is not set # CONFIG_SLIP is not set # CONFIG_SHAPER is not set # CONFIG_NETCONSOLE is not set # # ISDN subsystem # # CONFIG_ISDN is not set # # Telephony Support # # CONFIG_PHONE is not set # # Input device support # CONFIG_INPUT=y # # Userland interfaces # CONFIG_INPUT_MOUSEDEV=y # CONFIG_INPUT_MOUSEDEV_PSAUX is not set CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024 CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768 # CONFIG_INPUT_JOYDEV is not set # CONFIG_INPUT_TSDEV is not set CONFIG_INPUT_EVDEV=y # CONFIG_INPUT_EVBUG is not set # # Input Device Drivers # CONFIG_INPUT_KEYBOARD=y CONFIG_KEYBOARD_ATKBD=y # CONFIG_KEYBOARD_SUNKBD is not set # CONFIG_KEYBOARD_LKKBD is not set # CONFIG_KEYBOARD_XTKBD is not set # CONFIG_KEYBOARD_NEWTON is not set CONFIG_INPUT_MOUSE=y # CONFIG_MOUSE_PS2 is not set # CONFIG_MOUSE_SERIAL is not set # CONFIG_MOUSE_VSXXXAA is not set CONFIG_INPUT_JOYSTICK=y # CONFIG_JOYSTICK_ANALOG is not set # CONFIG_JOYSTICK_A3D is not set # CONFIG_JOYSTICK_ADI is not set # CONFIG_JOYSTICK_COBRA is not set # CONFIG_JOYSTICK_GF2K is not set # CONFIG_JOYSTICK_GRIP is not set # CONFIG_JOYSTICK_GRIP_MP is not set # CONFIG_JOYSTICK_GUILLEMOT is not set # CONFIG_JOYSTICK_INTERACT is not set # CONFIG_JOYSTICK_SIDEWINDER is not set # CONFIG_JOYSTICK_TMDC is not set # CONFIG_JOYSTICK_IFORCE is not set # CONFIG_JOYSTICK_WARRIOR is not set # CONFIG_JOYSTICK_MAGELLAN is not set # CONFIG_JOYSTICK_SPACEORB is not set # CONFIG_JOYSTICK_SPACEBALL is not set # CONFIG_JOYSTICK_STINGER is not set # CONFIG_JOYSTICK_TWIDJOY is not set # CONFIG_JOYSTICK_DB9 is not set # CONFIG_JOYSTICK_GAMECON is not set # CONFIG_JOYSTICK_TURBOGRAFX is not set # CONFIG_JOYSTICK_JOYDUMP is not set # CONFIG_INPUT_TOUCHSCREEN is not set CONFIG_INPUT_MISC=y CONFIG_INPUT_PCSPKR=m CONFIG_INPUT_UINPUT=m # # Hardware I/O ports # CONFIG_SERIO=y CONFIG_SERIO_I8042=y CONFIG_SERIO_SERPORT=y # CONFIG_SERIO_CT82C710 is not set # CONFIG_SERIO_PARKBD is not set # CONFIG_SERIO_PCIPS2 is not set CONFIG_SERIO_LIBPS2=y # CONFIG_SERIO_RAW is not set # CONFIG_GAMEPORT is not set # # Character devices # CONFIG_VT=y CONFIG_VT_CONSOLE=y CONFIG_HW_CONSOLE=y # CONFIG_SERIAL_NONSTANDARD is not set # # Serial drivers # CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y # CONFIG_SERIAL_8250_ACPI is not set CONFIG_SERIAL_8250_NR_UARTS=4 CONFIG_SERIAL_8250_EXTENDED=y # CONFIG_SERIAL_8250_MANY_PORTS is not set CONFIG_SERIAL_8250_SHARE_IRQ=y CONFIG_SERIAL_8250_DETECT_IRQ=y CONFIG_SERIAL_8250_MULTIPORT=y CONFIG_SERIAL_8250_RSA=y # # Non-8250 serial port support # CONFIG_SERIAL_CORE=y CONFIG_SERIAL_CORE_CONSOLE=y # CONFIG_SERIAL_JSM is not set CONFIG_UNIX98_PTYS=y # CONFIG_LEGACY_PTYS is not set # CONFIG_PRINTER is not set # CONFIG_PPDEV is not set # CONFIG_TIPAR is not set # # IPMI # # CONFIG_IPMI_HANDLER is not set # # Watchdog Cards # # CONFIG_WATCHDOG is not set # CONFIG_HW_RANDOM is not set CONFIG_NVRAM=m CONFIG_RTC=y CONFIG_RTC_HISTOGRAM=y CONFIG_BLOCKER=y CONFIG_LPPTEST=m # CONFIG_DTLK is not set # CONFIG_R3964 is not set # CONFIG_APPLICOM is not set # CONFIG_SONYPI is not set # # Ftape, the floppy tape device driver # # CONFIG_FTAPE is not set CONFIG_AGP=y # CONFIG_AGP_ALI is not set CONFIG_AGP_ATI=y # CONFIG_AGP_AMD is not set # CONFIG_AGP_AMD64 is not set CONFIG_AGP_INTEL=m # CONFIG_AGP_NVIDIA is not set # CONFIG_AGP_SIS is not set # CONFIG_AGP_SWORKS is not set # CONFIG_AGP_VIA is not set # CONFIG_AGP_EFFICEON is not set CONFIG_DRM=y # CONFIG_DRM_TDFX is not set CONFIG_DRM_R128=m CONFIG_DRM_RADEON=m CONFIG_DRM_I810=m CONFIG_DRM_I830=m CONFIG_DRM_I915=m # CONFIG_DRM_MGA is not set # CONFIG_DRM_SIS is not set # CONFIG_MWAVE is not set CONFIG_RAW_DRIVER=y CONFIG_HPET=y # CONFIG_HPET_RTC_IRQ is not set CONFIG_HPET_MMAP=y CONFIG_MAX_RAW_DEVS=8192 CONFIG_HANGCHECK_TIMER=y # # TPM devices # # CONFIG_TCG_TPM is not set # # I2C support # # CONFIG_I2C is not set # # Dallas's 1-wire bus # # CONFIG_W1 is not set # # Misc devices # # CONFIG_IBM_ASM is not set # # Multimedia devices # CONFIG_VIDEO_DEV=m # # Video For Linux # # # Video Adapters # # CONFIG_VIDEO_BWQCAM is not set # CONFIG_VIDEO_CQCAM is not set # CONFIG_VIDEO_W9966 is not set # CONFIG_VIDEO_CPIA is not set # CONFIG_VIDEO_STRADIS is not set # CONFIG_VIDEO_MXB is not set # CONFIG_VIDEO_DPC is not set # CONFIG_VIDEO_HEXIUM_ORION is not set # CONFIG_VIDEO_HEXIUM_GEMINI is not set # # Radio Adapters # # CONFIG_RADIO_GEMTEK_PCI is not set # CONFIG_RADIO_MAXIRADIO is not set # CONFIG_RADIO_MAESTRO is not set # # Digital Video Broadcasting Devices # # CONFIG_DVB is not set # # Graphics support # # CONFIG_FB is not set CONFIG_VIDEO_SELECT=y # # Console display driver support # CONFIG_VGA_CONSOLE=y CONFIG_DUMMY_CONSOLE=y # # Sound # CONFIG_SOUND=m # # Advanced Linux Sound Architecture # CONFIG_SND=m CONFIG_SND_TIMER=m CONFIG_SND_PCM=m CONFIG_SND_HWDEP=m CONFIG_SND_RAWMIDI=m CONFIG_SND_SEQUENCER=m CONFIG_SND_SEQ_DUMMY=m CONFIG_SND_OSSEMUL=y CONFIG_SND_MIXER_OSS=m CONFIG_SND_PCM_OSS=m CONFIG_SND_SEQUENCER_OSS=y CONFIG_SND_RTCTIMER=m # CONFIG_SND_VERBOSE_PRINTK is not set # CONFIG_SND_DEBUG is not set # # Generic devices # CONFIG_SND_MPU401_UART=m CONFIG_SND_OPL3_LIB=m CONFIG_SND_VX_LIB=m CONFIG_SND_DUMMY=m CONFIG_SND_VIRMIDI=m CONFIG_SND_MTPAV=m # CONFIG_SND_SERIAL_U16550 is not set CONFIG_SND_MPU401=m # # PCI devices # CONFIG_SND_AC97_CODEC=m CONFIG_SND_ALI5451=m CONFIG_SND_ATIIXP=m CONFIG_SND_ATIIXP_MODEM=m CONFIG_SND_AU8810=m CONFIG_SND_AU8820=m CONFIG_SND_AU8830=m CONFIG_SND_AZT3328=m CONFIG_SND_BT87X=m # CONFIG_SND_BT87X_OVERCLOCK is not set CONFIG_SND_CS46XX=m CONFIG_SND_CS46XX_NEW_DSP=y CONFIG_SND_CS4281=m CONFIG_SND_EMU10K1=m CONFIG_SND_EMU10K1X=m CONFIG_SND_CA0106=m CONFIG_SND_KORG1212=m CONFIG_SND_MIXART=m CONFIG_SND_NM256=m CONFIG_SND_RME32=m CONFIG_SND_RME96=m CONFIG_SND_RME9652=m CONFIG_SND_HDSP=m CONFIG_SND_TRIDENT=m CONFIG_SND_YMFPCI=m CONFIG_SND_ALS4000=m CONFIG_SND_CMIPCI=m CONFIG_SND_ENS1370=m CONFIG_SND_ENS1371=m CONFIG_SND_ES1938=m CONFIG_SND_ES1968=m CONFIG_SND_MAESTRO3=m CONFIG_SND_FM801=m CONFIG_SND_FM801_TEA575X=m CONFIG_SND_ICE1712=m CONFIG_SND_ICE1724=m CONFIG_SND_INTEL8X0=m CONFIG_SND_INTEL8X0M=m CONFIG_SND_SONICVIBES=m CONFIG_SND_VIA82XX=m CONFIG_SND_VIA82XX_MODEM=m CONFIG_SND_VX222=m CONFIG_SND_HDA_INTEL=m # # USB devices # # CONFIG_SND_USB_AUDIO is not set # CONFIG_SND_USB_USX2Y is not set # # Open Sound System # # CONFIG_SOUND_PRIME is not set # # USB support # CONFIG_USB_ARCH_HAS_HCD=y CONFIG_USB_ARCH_HAS_OHCI=y CONFIG_USB=y # CONFIG_USB_DEBUG is not set # # Miscellaneous USB options # CONFIG_USB_DEVICEFS=y # CONFIG_USB_BANDWIDTH is not set # CONFIG_USB_DYNAMIC_MINORS is not set CONFIG_USB_SUSPEND=y # CONFIG_USB_OTG is not set # # USB Host Controller Drivers # CONFIG_USB_EHCI_HCD=m CONFIG_USB_EHCI_SPLIT_ISO=y CONFIG_USB_EHCI_ROOT_HUB_TT=y CONFIG_USB_OHCI_HCD=m # CONFIG_USB_OHCI_BIG_ENDIAN is not set CONFIG_USB_OHCI_LITTLE_ENDIAN=y CONFIG_USB_UHCI_HCD=m # CONFIG_USB_SL811_HCD is not set # # USB Device Class drivers # # CONFIG_USB_AUDIO is not set # CONFIG_USB_BLUETOOTH_TTY is not set CONFIG_USB_MIDI=m CONFIG_USB_ACM=m CONFIG_USB_PRINTER=m # # NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support' may also be needed; see USB_STORAGE Help for more information # # CONFIG_USB_STORAGE is not set # # USB Input Devices # CONFIG_USB_HID=y CONFIG_USB_HIDINPUT=y CONFIG_HID_FF=y CONFIG_HID_PID=y CONFIG_LOGITECH_FF=y CONFIG_THRUSTMASTER_FF=y CONFIG_USB_HIDDEV=y # CONFIG_USB_AIPTEK is not set # CONFIG_USB_WACOM is not set # CONFIG_USB_KBTAB is not set # CONFIG_USB_POWERMATE is not set # CONFIG_USB_MTOUCH is not set # CONFIG_USB_EGALAX is not set # CONFIG_USB_XPAD is not set # CONFIG_USB_ATI_REMOTE is not set # # USB Imaging devices # # CONFIG_USB_MDC800 is not set # # USB Multimedia devices # # CONFIG_USB_DABUSB is not set # CONFIG_USB_VICAM is not set # CONFIG_USB_DSBR is not set # CONFIG_USB_IBMCAM is not set # CONFIG_USB_KONICAWC is not set # CONFIG_USB_OV511 is not set # CONFIG_USB_SE401 is not set # CONFIG_USB_SN9C102 is not set # CONFIG_USB_STV680 is not set # CONFIG_USB_PWC is not set # # USB Network Adapters # # CONFIG_USB_CATC is not set # CONFIG_USB_KAWETH is not set # CONFIG_USB_PEGASUS is not set # CONFIG_USB_RTL8150 is not set # CONFIG_USB_USBNET is not set # CONFIG_USB_MON is not set # # USB port drivers # # CONFIG_USB_USS720 is not set # # USB Serial Converter support # # CONFIG_USB_SERIAL is not set # # USB Miscellaneous drivers # # CONFIG_USB_EMI62 is not set # CONFIG_USB_EMI26 is not set # CONFIG_USB_AUERSWALD is not set # CONFIG_USB_RIO500 is not set # CONFIG_USB_LEGOTOWER is not set # CONFIG_USB_LCD is not set # CONFIG_USB_LED is not set # CONFIG_USB_CYTHERM is not set # CONFIG_USB_PHIDGETKIT is not set # CONFIG_USB_PHIDGETSERVO is not set # CONFIG_USB_IDMOUSE is not set # CONFIG_USB_SISUSBVGA is not set # CONFIG_USB_TEST is not set # # USB ATM/DSL drivers # # CONFIG_USB_ATM is not set # CONFIG_USB_SPEEDTOUCH is not set # # USB Gadget Support # # CONFIG_USB_GADGET is not set # # MMC/SD Card support # # CONFIG_MMC is not set # # InfiniBand support # # CONFIG_INFINIBAND is not set # # File systems # CONFIG_EXT2_FS=y CONFIG_EXT2_FS_XATTR=y CONFIG_EXT2_FS_POSIX_ACL=y CONFIG_EXT2_FS_SECURITY=y CONFIG_EXT3_FS=y CONFIG_EXT3_FS_XATTR=y CONFIG_EXT3_FS_POSIX_ACL=y CONFIG_EXT3_FS_SECURITY=y CONFIG_JBD=y # CONFIG_JBD_DEBUG is not set CONFIG_FS_MBCACHE=y CONFIG_REISERFS_FS=m # CONFIG_REISERFS_CHECK is not set # CONFIG_REISERFS_PROC_INFO is not set CONFIG_REISERFS_FS_XATTR=y CONFIG_REISERFS_FS_POSIX_ACL=y CONFIG_REISERFS_FS_SECURITY=y # CONFIG_JFS_FS is not set CONFIG_FS_POSIX_ACL=y # # XFS support # # CONFIG_XFS_FS is not set # CONFIG_MINIX_FS is not set # CONFIG_ROMFS_FS is not set CONFIG_QUOTA=y # CONFIG_QFMT_V1 is not set CONFIG_QFMT_V2=y CONFIG_QUOTACTL=y CONFIG_DNOTIFY=y CONFIG_AUTOFS_FS=m CONFIG_AUTOFS4_FS=m # # CD-ROM/DVD Filesystems # CONFIG_ISO9660_FS=y CONFIG_JOLIET=y CONFIG_ZISOFS=y CONFIG_ZISOFS_FS=y CONFIG_UDF_FS=m CONFIG_UDF_NLS=y # # DOS/FAT/NT Filesystems # # CONFIG_MSDOS_FS is not set # CONFIG_VFAT_FS is not set # CONFIG_NTFS_FS is not set # # Pseudo filesystems # CONFIG_PROC_FS=y CONFIG_PROC_KCORE=y CONFIG_SYSFS=y # CONFIG_DEVFS_FS is not set CONFIG_DEVPTS_FS_XATTR=y CONFIG_DEVPTS_FS_SECURITY=y CONFIG_TMPFS=y CONFIG_TMPFS_XATTR=y CONFIG_TMPFS_SECURITY=y CONFIG_HUGETLBFS=y CONFIG_HUGETLB_PAGE=y CONFIG_RAMFS=y # # Miscellaneous filesystems # # CONFIG_ADFS_FS is not set # CONFIG_AFFS_FS is not set # CONFIG_HFS_FS is not set # CONFIG_HFSPLUS_FS is not set # CONFIG_BEFS_FS is not set # CONFIG_BFS_FS is not set # CONFIG_EFS_FS is not set # CONFIG_JFFS_FS is not set # CONFIG_JFFS2_FS is not set CONFIG_CRAMFS=m # CONFIG_VXFS_FS is not set # CONFIG_HPFS_FS is not set # CONFIG_QNX4FS_FS is not set # CONFIG_SYSV_FS is not set # CONFIG_UFS_FS is not set # # Network File Systems # CONFIG_NFS_FS=m CONFIG_NFS_V3=y CONFIG_NFS_V4=y CONFIG_NFS_DIRECTIO=y CONFIG_NFSD=m CONFIG_NFSD_V3=y CONFIG_NFSD_V4=y CONFIG_NFSD_TCP=y CONFIG_LOCKD=m CONFIG_LOCKD_V4=y CONFIG_EXPORTFS=m CONFIG_SUNRPC=m CONFIG_SUNRPC_GSS=m CONFIG_RPCSEC_GSS_KRB5=m CONFIG_RPCSEC_GSS_SPKM3=m CONFIG_SMB_FS=m # CONFIG_SMB_NLS_DEFAULT is not set CONFIG_CIFS=m # CONFIG_CIFS_STATS is not set CONFIG_CIFS_XATTR=y CONFIG_CIFS_POSIX=y # CONFIG_CIFS_EXPERIMENTAL is not set # CONFIG_NCP_FS is not set # CONFIG_CODA_FS is not set # CONFIG_AFS_FS is not set # # Partition Types # # CONFIG_PARTITION_ADVANCED is not set CONFIG_MSDOS_PARTITION=y # # Native Language Support # CONFIG_NLS=y CONFIG_NLS_DEFAULT="utf8" CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_CODEPAGE_737=m CONFIG_NLS_CODEPAGE_775=m CONFIG_NLS_CODEPAGE_850=m CONFIG_NLS_CODEPAGE_852=m CONFIG_NLS_CODEPAGE_855=m CONFIG_NLS_CODEPAGE_857=m CONFIG_NLS_CODEPAGE_860=m CONFIG_NLS_CODEPAGE_861=m CONFIG_NLS_CODEPAGE_862=m CONFIG_NLS_CODEPAGE_863=m CONFIG_NLS_CODEPAGE_864=m CONFIG_NLS_CODEPAGE_865=m CONFIG_NLS_CODEPAGE_866=m CONFIG_NLS_CODEPAGE_869=m CONFIG_NLS_CODEPAGE_936=m CONFIG_NLS_CODEPAGE_950=m CONFIG_NLS_CODEPAGE_932=m CONFIG_NLS_CODEPAGE_949=m CONFIG_NLS_CODEPAGE_874=m CONFIG_NLS_ISO8859_8=m CONFIG_NLS_CODEPAGE_1250=m CONFIG_NLS_CODEPAGE_1251=m CONFIG_NLS_ASCII=y CONFIG_NLS_ISO8859_1=m CONFIG_NLS_ISO8859_2=m CONFIG_NLS_ISO8859_3=m CONFIG_NLS_ISO8859_4=m CONFIG_NLS_ISO8859_5=m CONFIG_NLS_ISO8859_6=m CONFIG_NLS_ISO8859_7=m CONFIG_NLS_ISO8859_9=m CONFIG_NLS_ISO8859_13=m CONFIG_NLS_ISO8859_14=m CONFIG_NLS_ISO8859_15=m CONFIG_NLS_KOI8_R=m CONFIG_NLS_KOI8_U=m CONFIG_NLS_UTF8=m # # Profiling support # CONFIG_PROFILING=y CONFIG_OPROFILE=m # # Kernel hacking # # CONFIG_PRINTK_TIME is not set # CONFIG_PRINTK_IGNORE_LOGLEVEL is not set CONFIG_DEBUG_KERNEL=y CONFIG_MAGIC_SYSRQ=y CONFIG_LOG_BUF_SHIFT=17 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_DEBUG_SLAB is not set CONFIG_DEBUG_PREEMPT=y CONFIG_DEBUG_IRQ_FLAGS=y # CONFIG_WAKEUP_TIMING is not set CONFIG_PREEMPT_TRACE=y # CONFIG_CRITICAL_PREEMPT_TIMING is not set # CONFIG_CRITICAL_IRQSOFF_TIMING is not set # CONFIG_RT_DEADLOCK_DETECT is not set # CONFIG_DEBUG_RT_LOCKING_MODE is not set # CONFIG_DEBUG_KOBJECT is not set # CONFIG_DEBUG_HIGHMEM is not set CONFIG_DEBUG_BUGVERBOSE=y CONFIG_DEBUG_INFO=y # CONFIG_DEBUG_FS is not set # CONFIG_USE_FRAME_POINTER is not set CONFIG_EARLY_PRINTK=y # CONFIG_DEBUG_STACKOVERFLOW is not set # CONFIG_KPROBES is not set # CONFIG_DEBUG_STACK_USAGE is not set # CONFIG_DEBUG_PAGEALLOC is not set # CONFIG_4KSTACKS is not set CONFIG_X86_FIND_SMP_CONFIG=y CONFIG_X86_MPPARSE=y # # Security options # # CONFIG_KEYS is not set CONFIG_SECURITY=y CONFIG_SECURITY_NETWORK=y CONFIG_SECURITY_CAPABILITIES=y # CONFIG_SECURITY_ROOTPLUG is not set # CONFIG_SECURITY_SECLVL is not set CONFIG_SECURITY_SELINUX=y CONFIG_SECURITY_SELINUX_BOOTPARAM=y CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1 CONFIG_SECURITY_SELINUX_DISABLE=y CONFIG_SECURITY_SELINUX_DEVELOP=y CONFIG_SECURITY_SELINUX_AVC_STATS=y CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1 # # Cryptographic options # CONFIG_CRYPTO=y CONFIG_CRYPTO_HMAC=y CONFIG_CRYPTO_NULL=m CONFIG_CRYPTO_MD4=m CONFIG_CRYPTO_MD5=m CONFIG_CRYPTO_SHA1=y CONFIG_CRYPTO_SHA256=m CONFIG_CRYPTO_SHA512=m CONFIG_CRYPTO_WP512=m CONFIG_CRYPTO_TGR192=m CONFIG_CRYPTO_DES=m CONFIG_CRYPTO_BLOWFISH=m CONFIG_CRYPTO_TWOFISH=m CONFIG_CRYPTO_SERPENT=m CONFIG_CRYPTO_AES_586=m CONFIG_CRYPTO_CAST5=m CONFIG_CRYPTO_CAST6=m CONFIG_CRYPTO_TEA=m CONFIG_CRYPTO_ARC4=m CONFIG_CRYPTO_KHAZAD=m # CONFIG_CRYPTO_ANUBIS is not set CONFIG_CRYPTO_DEFLATE=m CONFIG_CRYPTO_MICHAEL_MIC=m CONFIG_CRYPTO_CRC32C=m # CONFIG_CRYPTO_TEST is not set # # Hardware crypto devices # # CONFIG_CRYPTO_DEV_PADLOCK is not set # # Library routines # CONFIG_CRC_CCITT=m CONFIG_CRC32=y CONFIG_LIBCRC32C=m CONFIG_ZLIB_INFLATE=y CONFIG_ZLIB_DEFLATE=m CONFIG_GENERIC_HARDIRQS=y CONFIG_GENERIC_IRQ_PROBE=y CONFIG_X86_BIOS_REBOOT=y CONFIG_PC=y ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-07 21:18 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 Chuck Harding @ 2005-07-08 5:41 ` Ingo Molnar 2005-07-08 8:00 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 5:41 UTC (permalink / raw) To: Chuck Harding; +Cc: Linux Kernel Discussion List * Chuck Harding <charding@llnl.gov> wrote: > Which still exhibits the lockup of sox. I built a 2.6.12 vanilla kernel > using the same .config as I used for 51-12 and the failure did not happen. > just the process of booting up causes later invocations of sox to lockup > in the D state. If I don't login to X and just run from a VT I can get > it to lockup by running something like: > $ for ((i=0;i<20;i++)) > >do > >echo -ne "\r$i " > >(play /usr/share/sounds/KDE_Beep_ShortBeep.wav &) > >done > > after about 14 or so iterations of the loop and thereafter no more > sound can be played. tried it and cannot reproduce it, so i'll need the full backtrace of all tasks in your system, whenever sox gets stuck, via: echo t > /proc/sysrq-trigger dmesg -s 10000000 > toingo.txt Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-08 5:41 ` Ingo Molnar @ 2005-07-08 8:00 ` Ingo Molnar 2005-07-08 18:06 ` Chuck Harding 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 8:00 UTC (permalink / raw) To: Chuck Harding; +Cc: Linux Kernel Discussion List, Jaroslav Kysela * Ingo Molnar <mingo@elte.hu> wrote: > tried it and cannot reproduce it, so i'll need the full backtrace of > all tasks in your system, whenever sox gets stuck, via: managed to reproduce it via your script - and with RT_DEADLOCK_DETECT turned on the circular deadlock was immediately detected (see the trace below). It turns out this is an upstream locking bug in the sound subsystem, which has been fixed already - i've merged the upstream fix to -51-14. Could you check whether sox works for you now? Ingo ============================================ [ BUG: circular locking deadlock detected! ] -------------------------------------------- sox/1828 is deadlocking current task sox/1824 1) sox/1824 is trying to acquire this lock: [cf073eac] {(struct semaphore *)(&card->power_lock)} .. held by: sox: 1828 [c901f9d0, 119] ... acquired at: snd_pcm_prepare+0x22/0x6d ... trying at: snd_pcm_drain+0x187/0x3a2 2) sox/1828 is blocked on this lock: [c04e4240] {snd_pcm_link_rwsem.lock} .. held by: sox: 1824 [ca3fc9d0, 115] ... acquired at: snd_pcm_drain+0x37/0x3a2 ------------------------------ | showing all locks held by: | (sox/1824 [ca3fc9d0, 115]): ------------------------------ #001: [c04e4240] {snd_pcm_link_rwsem.lock} ... acquired at: snd_pcm_drain+0x37/0x3a2 ------------------------------ | showing all locks held by: | (sox/1828 [c901f9d0, 119]): ------------------------------ #001: [cf073eac] {(struct semaphore *)(&card->power_lock)} ... acquired at: snd_pcm_prepare+0x22/0x6d sox/1828's [blocked] stackdump: c9038d98 00000082 c05ae8cc c05ae420 0000000a 00000000 c9038dec 00000000 c9ac89d0 00011080 926e71c9 00000029 c9ac89d0 c901f9d0 c901faf8 c9038000 00000000 c9eef000 c9038db4 c0441321 c013c236 c9038db4 c013e9ab c9038000 Call Trace: [<c0441321>] schedule+0x47/0x124 (28) [<c044289c>] __down+0x2f5/0x653 (108) [<c04433fc>] rt_down_read+0x26/0x2e (16) [<c030273f>] snd_pcm_action_nonatomic+0x21/0x83 (36) [<c0303263>] snd_pcm_prepare+0x5a/0x6d (32) [<c0305869>] snd_pcm_playback_ioctl1+0x4c/0x2a1 (68) [<c0305db9>] snd_pcm_kernel_playback_ioctl+0x36/0x40 (28) [<c0312953>] snd_pcm_oss_prepare+0x2a/0x5b (24) [<c03129bd>] snd_pcm_oss_make_ready+0x39/0x58 (20) [<c0313572>] snd_pcm_oss_sync+0x37/0x2b0 (64) [<c017dc2d>] do_ioctl+0x7d/0x8c (36) [<c017ddc4>] vfs_ioctl+0x61/0x1c2 (40) [<c017df96>] sys_ioctl+0x71/0x7f (40) [<c01030e8>] sysenter_past_esp+0x61/0x89 (-4020) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c013e852>] .... add_preempt_count+0x1a/0x1c .....[<c0440c1f>] .. ( <= __schedule+0x4f/0x70a) .. [<c013e852>] .... add_preempt_count+0x1a/0x1c .....[<c0440cb1>] .. ( <= __schedule+0xe1/0x70a) ------------------------------ | showing all locks held by: | (sox/1828 [c901f9d0, 119]): ------------------------------ #001: [cf073eac] {(struct semaphore *)(&card->power_lock)} ... acquired at: snd_pcm_prepare+0x22/0x6d sox/1824's [current] stackdump: [<c0104082>] dump_stack+0x1f/0x21 (20) [<c0138206>] check_deadlock+0x20f/0x32e (48) [<c0138f7c>] task_blocks_on_lock+0x3a/0xe1 (44) [<c04427ee>] __down+0x247/0x653 (108) [<c013af13>] rt_down+0x29/0x45d (44) [<c030352d>] snd_pcm_drain+0x187/0x3a2 (96) [<c0305869>] snd_pcm_playback_ioctl1+0x4c/0x2a1 (68) [<c0305db9>] snd_pcm_kernel_playback_ioctl+0x36/0x40 (28) [<c031361a>] snd_pcm_oss_sync+0xdf/0x2b0 (64) [<c0314b25>] snd_pcm_oss_release+0x2b/0xd8 (28) [<c016b511>] __fput+0x13d/0x181 (36) [<c0169d43>] filp_close+0x4f/0x83 (28) [<c0169ddd>] sys_close+0x66/0x81 (32) [<c01030e8>] sysenter_past_esp+0x61/0x89 (-4020) --------------------------- | preempt count: 00000001 ] | 1-level deep critical section nesting: ---------------------------------------- .. [<c013e852>] .... add_preempt_count+0x1a/0x1c .....[<c013f969>] .. ( <= print_traces+0x1b/0x50) ------------------------------ | showing all locks held by: | (sox/1824 [ca3fc9d0, 116]): ------------------------------ #001: [c04e4240] {snd_pcm_link_rwsem.lock} ... acquired at: snd_pcm_drain+0x37/0x3a2 showing all tasks: S init: 1 [cff1c9d0, 116] (not blocked) S softirq-high/0: 2 [cff209d0, 110] (not blocked) S softirq-timer/0: 3 [cff249d0, 105] (not blocked) S softirq-net-tx/: 4 [cff289d0, 110] (not blocked) S softirq-net-rx/: 5 [cff2c9d0, 105] (not blocked) S softirq-scsi/0: 6 [cff309d0, 110] (not blocked) S softirq-tasklet: 7 [cff349d0, 105] (not blocked) S desched/0: 8 [cff389d0, 105] (not blocked) S watchdog/0: 9 [cff3c9d0, 0] (not blocked) S events/0: 10 [cff5d9d0, 98] (not blocked) S khelper: 11 [cff619d0, 111] (not blocked) S kthread: 12 [cfe789d0, 110] (not blocked) S kblockd/0: 14 [cfea39d0, 110] (not blocked) S khubd: 17 [cfa039d0, 125] (not blocked) S pdflush: 133 [cfa0e9d0, 120] (not blocked) S pdflush: 134 [cf4929d0, 115] (not blocked) S aio/0: 136 [cf4a29d0, 112] (not blocked) S kswapd0: 135 [cf4969d0, 125] (not blocked) S IRQ 8: 207 [cf54e9d0, 50] (not blocked) S IRQ 12: 217 [cf6c49d0, 51] (not blocked) S kseriod: 214 [cf69f9d0, 115] (not blocked) S IRQ 14: 255 [cf6d89d0, 52] (not blocked) S IRQ 15: 257 [cf7149d0, 53] (not blocked) S IRQ 1: 279 [cf0849d0, 54] (not blocked) S IRQ 17: 297 [cf0b29d0, 55] (not blocked) S kjournald: 336 [cf16e9d0, 115] (not blocked) S udevd: 1055 [ceb7c9d0, 110] (not blocked) S IRQ 16: 1230 [ce1199d0, 56] (not blocked) S syslogd: 1284 [ceba39d0, 117] (not blocked) S klogd: 1286 [cf0d39d0, 116] (not blocked) S sshd: 1306 [ce7759d0, 116] (not blocked) S gpm: 1316 [ce7fc9d0, 115] (not blocked) S crond: 1404 [ce4b89d0, 118] (not blocked) S xfs: 1428 [ce75e9d0, 118] (not blocked) S atd: 1443 [ce4949d0, 119] (not blocked) S dbus-daemon: 1451 [ce9dc9d0, 116] (not blocked) S dbus-daemon: 1452 [ce1a59d0, 121] (not blocked) S mingetty: 1532 [cda6f9d0, 118] (not blocked) S mingetty: 1533 [caaf29d0, 116] (not blocked) S mingetty: 1534 [ce9af9d0, 119] (not blocked) S mingetty: 1548 [caa979d0, 120] (not blocked) S mingetty: 1549 [ca2d89d0, 121] (not blocked) S mingetty: 1582 [c9e449d0, 120] (not blocked) S sshd: 1605 [ca3149d0, 116] (not blocked) S sshd: 1701 [c9e2b9d0, 116] (not blocked) S bash: 1722 [ca2729d0, 116] (not blocked) S su: 1792 [ceb1c9d0, 117] (not blocked) S bash: 1793 [c9e019d0, 116] (not blocked) R bash: 1809 [ca3d89d0, 119] (not blocked) S play: 1811 [ca8269d0, 117] (not blocked) S play: 1813 [c9e159d0, 118] (not blocked) R play: 1815 [ca2ea9d0, 118] (not blocked) R play: 1817 [c9ac89d0, 118] (not blocked) S play: 1819 [caa319d0, 122] (not blocked) S play: 1821 [ca08a9d0, 123] (not blocked) D sox: 1824 [ca3fc9d0, 116] (not blocked) D sox: 1828 [c901f9d0, 119] blocked on: [c04e4240] {snd_pcm_link_rwsem.lock} .. held by: sox: 1824 [ca3fc9d0, 116] ... acquired at: snd_pcm_drain+0x37/0x3a2 R sox: 1832 [ca0d79d0, 125] (not blocked) D bash: 1833 [c9c389d0, 120] blocked on: [c04cb004] {kernel_sem.lock} .. held by: play: 1817 [c9ac89d0, 118] ... acquired at: lock_kernel+0x25/0x38 S sox: 1835 [c96669d0, 118] blocked on: [c9efbcd0] {(struct semaphore *)(&tty->atomic_write)} .. held by: bash: 1809 [ca3d89d0, 119] ... acquired at: tty_write+0xb6/0x20d D play: 1837 [c9b749d0, 120] blocked on: [c04cb004] {kernel_sem.lock} .. held by: play: 1817 [c9ac89d0, 118] ... acquired at: lock_kernel+0x25/0x38 R bash: 1839 [c9b819d0, 121] (not blocked) --------------------------- | showing all locks held: | --------------------------- #001: [c07aaa80] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #002: [c07aa6ac] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #003: [c07aa87c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #004: [c07ab0c0] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #005: [c07aacec] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #006: [c07aaebc] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #007: [c07ab700] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #008: [c07ab32c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #009: [c07ab4fc] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #010: [c07abd40] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #011: [c07ab96c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #012: [c07abb3c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #013: [c07ac380] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #014: [c07abfac] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #015: [c07ac17c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #016: [c07ac9c0] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #017: [c07ac5ec] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #018: [c07ac7bc] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #019: [c07ad000] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #020: [c07acc2c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #021: [c07acdfc] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #022: [c07ad640] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #023: [c07ad26c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #024: [c07ad43c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #025: [c07adc80] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #026: [c07ad8ac] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #027: [c07ada7c] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #028: [c07ae2c0] {(struct semaphore *)(&hwif->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x93/0x162 #029: [c07adeec] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #030: [c07ae0bc] {(struct semaphore *)(&drive->gendev_rel_sem)} .. held by: init: 1 [cff1c9d0, 116] ... acquired at: init_hwif_data+0x151/0x162 #031: [ceb0bc98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1532 [cda6f9d0, 118] ... acquired at: read_chan+0x3d2/0x632 #032: [c9ac9c98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1533 [caaf29d0, 116] ... acquired at: read_chan+0x3d2/0x632 #033: [cba8cc98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1534 [ce9af9d0, 119] ... acquired at: read_chan+0x3d2/0x632 #034: [c9ed3c98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1548 [caa979d0, 120] ... acquired at: read_chan+0x3d2/0x632 #035: [c9f02c98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1582 [c9e449d0, 120] ... acquired at: read_chan+0x3d2/0x632 #036: [cba50c98] {(struct semaphore *)(&tty->atomic_read)} .. held by: mingetty: 1549 [ca2d89d0, 121] ... acquired at: read_chan+0x3d2/0x632 #037: [c04e4240] {snd_pcm_link_rwsem.lock} .. held by: sox: 1824 [ca3fc9d0, 117] ... acquired at: snd_pcm_drain+0x37/0x3a2 #038: [cf073eac] {(struct semaphore *)(&card->power_lock)} .. held by: sox: 1828 [c901f9d0, 119] ... acquired at: snd_pcm_prepare+0x22/0x6d #039: [c9efbcd0] {(struct semaphore *)(&tty->atomic_write)} .. held by: bash: 1809 [ca3d89d0, 119] ... acquired at: tty_write+0xb6/0x20d #040: [c04cb004] {kernel_sem.lock} .. held by: play: 1817 [c9ac89d0, 118] ... acquired at: lock_kernel+0x25/0x38 #041: [c0565a00] {tasklist_lock} .. held by: sox: 1824 [ca3fc9d0, 117] ... acquired at: _read_trylock+0x13/0x15 ============================================= [ turning off deadlock detection. Please report this trace. ] ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-08 8:00 ` Ingo Molnar @ 2005-07-08 18:06 ` Chuck Harding 0 siblings, 0 replies; 76+ messages in thread From: Chuck Harding @ 2005-07-08 18:06 UTC (permalink / raw) To: Ingo Molnar; +Cc: Linux Kernel Discussion List, Jaroslav Kysela On Fri, 8 Jul 2005, Ingo Molnar wrote: > > * Ingo Molnar <mingo@elte.hu> wrote: > >> tried it and cannot reproduce it, so i'll need the full backtrace of >> all tasks in your system, whenever sox gets stuck, via: > > managed to reproduce it via your script - and with RT_DEADLOCK_DETECT > turned on the circular deadlock was immediately detected (see the trace > below). It turns out this is an upstream locking bug in the sound > subsystem, which has been fixed already - i've merged the upstream fix > to -51-14. Could you check whether sox works for you now? > > Ingo > That did it. I can now hammer the sound system and sox won't hang anymore. Thanks!! -- Charles D. (Chuck) Harding <charding@llnl.gov> Voice: 925-423-8879 Senior Computer Associate ICCD Fax: 925-423-6961 Lawrence Livermore National Laboratory Computation Directorate Livermore, CA USA http://www.llnl.gov GPG Public Key ID: B9EB6601 ------------------ http://tinyurl.com/5w5ey ----------------------- -- You've certainly got smooth skin - between the wrinkles, that is. -- ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-06 10:04 ` Ingo Molnar 2005-07-06 21:12 ` William Weston 2005-07-07 14:56 ` Steven Rostedt @ 2005-07-07 19:05 ` William Weston 2005-07-07 19:18 ` Ingo Molnar 2005-07-08 9:23 ` Ingo Molnar 2 siblings, 2 replies; 76+ messages in thread From: William Weston @ 2005-07-07 19:05 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Wed, 6 Jul 2005, Ingo Molnar wrote: > do the 51-02 (and later) kernels work on the UP Athlon box? Hi Ingo, -51-06 and -51-08 are looking stable on the UP Athlon box with the following patch which causes edge triggered hardirqs to be masked when pending _and/or_ disabled (instead of both pending _and_ disabled) and backs out a change from -50-44 that prevents pending edge triggered irqs from being unmasked: --- linux.orig/arch/i386/kernel/io_apic.c 2005-07-06 11:08:39.000000000 -0700 +++ linux/arch/i386/kernel/io_apic.c 2005-07-07 03:40:37.000000000 -0700 @@ -1949,8 +1949,7 @@ static void ack_edge_ioapic_irq(unsigned int irq) { move_irq(irq); - if ((irq_desc[irq].status & (IRQ_PENDING | IRQ_DISABLED)) - == (IRQ_PENDING | IRQ_DISABLED)) + if (irq_desc[irq].status & (IRQ_PENDING | IRQ_DISABLED)) mask_IO_APIC_irq(irq); ack_APIC_irq(); } @@ -1997,8 +1996,6 @@ unsigned long v; int i; - if (!(irq_desc[irq].status & IRQ_DISABLED)) - unmask_IO_APIC_irq(irq); /* * It appears there is an erratum which affects at least version 0x11 * of I/O APIC (that's the 82093AA and cores integrated into various --- linux.orig/kernel/irq/manage.c 2005-07-06 11:08:40.000000000 -0700 +++ linux/kernel/irq/manage.c 2005-07-07 01:04:38.000000000 -0700 @@ -434,6 +434,8 @@ * The ->end() handler has to deal with interrupts which got * disabled while the handler was running. */ + if (!(desc->status & IRQ_DISABLED)) + desc->handler->enable(irq); desc->handler->end(irq); } spin_unlock_irq(&desc->lock); The first chunk is based on my understanding of the comments in io_apic.c, and is not necessary to keep the UP Athlon box stable. Is my logic OK here, or am I missing something really important? Without the last two chunks of this patch, the UP Athlon box locks up hard as soon as jackd is started up. Best Regards, --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-07 19:05 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 William Weston @ 2005-07-07 19:18 ` Ingo Molnar 2005-07-07 19:46 ` William Weston 2005-07-08 9:23 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 19:18 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > Without the last two chunks of this patch, the UP Athlon box locks up > hard as soon as jackd is started up. hm, do you have CONFIG_PCI_MSI enabled by any chance? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-07 19:18 ` Ingo Molnar @ 2005-07-07 19:46 ` William Weston 2005-07-07 19:52 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-07 19:46 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Thu, 7 Jul 2005, Ingo Molnar wrote: > > Without the last two chunks of this patch, the UP Athlon box locks up > > hard as soon as jackd is started up. > > hm, do you have CONFIG_PCI_MSI enabled by any chance? I've never enabled CONFIG_PCI_MSI. What's your experience when it comes to stability and performance? --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-07 19:46 ` William Weston @ 2005-07-07 19:52 ` Ingo Molnar 0 siblings, 0 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 19:52 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > On Thu, 7 Jul 2005, Ingo Molnar wrote: > > > > Without the last two chunks of this patch, the UP Athlon box locks up > > > hard as soon as jackd is started up. > > > > hm, do you have CONFIG_PCI_MSI enabled by any chance? > > I've never enabled CONFIG_PCI_MSI. What's your experience when it > comes to stability and performance? it's really not needed unless you have PCI-X hardware, but it sometimes gets enabled and it had IRQ handling bugs under PREEMPT_HARDIRQS (and PREEMPT_RT), which i fixed in the -51-12 patch. But if you didnt have it enabled then there's no IO-APIC change affecting your config. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-07 19:05 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 William Weston 2005-07-07 19:18 ` Ingo Molnar @ 2005-07-08 9:23 ` Ingo Molnar 2005-07-08 9:29 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 9:23 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > -51-06 and -51-08 are looking stable on the UP Athlon box with the > following patch which causes edge triggered hardirqs to be masked when > pending _and/or_ disabled (instead of both pending _and_ disabled) and > backs out a change from -50-44 that prevents pending edge triggered > irqs from being unmasked: ok, you are right, the edge case was mishandled - but i think it was already mishandled upstream, we just never (or rarely) triggered it. I've reworked this area based on your patch, could you check -51-15, does it work for you? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-08 9:23 ` Ingo Molnar @ 2005-07-08 9:29 ` Ingo Molnar 2005-07-08 9:45 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 9:29 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > > -51-06 and -51-08 are looking stable on the UP Athlon box with the > > following patch which causes edge triggered hardirqs to be masked when > > pending _and/or_ disabled (instead of both pending _and_ disabled) and > > backs out a change from -50-44 that prevents pending edge triggered > > irqs from being unmasked: > > ok, you are right, the edge case was mishandled - but i think it was > already mishandled upstream, we just never (or rarely) triggered it. > I've reworked this area based on your patch, could you check -51-15, > does it work for you? if it doesnt work, could you disable CONFIG_X86_IOAPIC_FAST? Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 2005-07-08 9:29 ` Ingo Molnar @ 2005-07-08 9:45 ` Ingo Molnar 2005-07-08 20:30 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 William Weston 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 9:45 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * Ingo Molnar <mingo@elte.hu> wrote: > > > -51-06 and -51-08 are looking stable on the UP Athlon box with the > > > following patch which causes edge triggered hardirqs to be masked when > > > pending _and/or_ disabled (instead of both pending _and_ disabled) and > > > backs out a change from -50-44 that prevents pending edge triggered > > > irqs from being unmasked: > > > > ok, you are right, the edge case was mishandled - but i think it was > > already mishandled upstream, we just never (or rarely) triggered it. > > I've reworked this area based on your patch, could you check -51-15, > > does it work for you? > > if it doesnt work, could you disable CONFIG_X86_IOAPIC_FAST? or rather, please try -51-16. One of my testsystems produced weird interrupt storms, which were caused by the IOAPIC_POSTFLUSH optimization. In -51-16 i've turned that optimization off, globally. Maybe this explains some of the other lockups reported. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 2005-07-08 9:45 ` Ingo Molnar @ 2005-07-08 20:30 ` William Weston 0 siblings, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-08 20:30 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Fri, 8 Jul 2005, Ingo Molnar wrote: > > > ok, you are right, the edge case was mishandled - but i think it was > > > already mishandled upstream, we just never (or rarely) triggered it. > > > I've reworked this area based on your patch, could you check -51-15, > > > does it work for you? I figured it would take a little reworking, especially with your recent MSI changes. This was just a 'works for me' patch. Your end_irq() is cleaner, IMHO. > > if it doesnt work, could you disable CONFIG_X86_IOAPIC_FAST? > > or rather, please try -51-16. One of my testsystems produced weird > interrupt storms, which were caused by the IOAPIC_POSTFLUSH > optimization. In -51-16 i've turned that optimization off, globally. > Maybe this explains some of the other lockups reported. OK. Running -51-17 on the Athlon box. CONFIG_X86_IOAPIC_FAST is still enabled. No lockups. No jack xruns. Max wakeup-latency is 27us (so far) on the debug config. Nothing out of the ordinary logged except for the new stack-footprint maximums during boot. Any need to go back and check -51-16 as well? Or -51-17 on a non-debug config? --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-03 18:12 ` Ingo Molnar 2005-07-05 20:43 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 William Weston @ 2005-07-07 1:26 ` William Weston 2005-07-07 10:48 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-07 1:26 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Sun, 3 Jul 2005, Ingo Molnar wrote: > ok, found a bug that could explain the situation: mutex sleeps+wakeups > were incorrectly credited as 'interactive sleep' periods, causing the dd > processes to be boosted incorrectly. The dd processes created a workload > in which they blocked each other in such a pattern that they got boosted > periodically, starving pretty much every other task. > > the fix is significant and affects alot of workloads, and should further > improve interactivity in noticeable ways. I'm not 100% sure it solves > all the starvation problems (e.g. how could normal-prio dd tasks starve > the SCHED_FIFO irq threads that drove SysRq?), but the results so far > look promising. > > i've uploaded the -50-45 patch, can you under this kernel trigger a > 'meltdown' on your SMT box? Still looking into this issue on -51-06. Found something really odd: SCHED_NORMAL tasks will start to inherit the priority value of some other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO priority, X and all of its children will inherit the same priority value after login. Other random processes will inherit this, too -- sometimes init... SCHED_NORMAL tasks suddenly inheriting priority values in the range normally reserved for SCHED_FIFO could explain at least part of the meltdown I've been seeing. Any thoughts? Cheers, --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 2005-07-07 1:26 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 William Weston @ 2005-07-07 10:48 ` Ingo Molnar 2005-07-07 18:49 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-06 William Weston 2005-07-07 20:02 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 William Weston 0 siblings, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-07 10:48 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > Still looking into this issue on -51-06. Found something really odd: > SCHED_NORMAL tasks will start to inherit the priority value of some > other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO > priority, X and all of its children will inherit the same priority > value after login. Other random processes will inherit this, too -- > sometimes init... > > SCHED_NORMAL tasks suddenly inheriting priority values in the range > normally reserved for SCHED_FIFO could explain at least part of the > meltdown I've been seeing. Any thoughts? is this inheritance perpetual? It is normal for some tasks to be boosted momentarily, but if the condition remains even after jackd has exited, it's clearly an anomaly. (lets call it "RT priority leakage".) Priority leakage on SMP was fixed recently, but there could be other bugs remaining. There's priority-leakage debugging code in the -RT kernel, which is activated if you enable CONFIG_DEBUG_PREEMPT. This debugging code, if triggered, should produce 'BUG: comm/123 leaked RT prio 123 (123)?" type of messages. But ... due to a bug it would not print out anything but would lock up hard if it detects the condition (and if RT_DEADLOCK_DETECT is enabled). I have fixed this reporting bug in the -51-08 kernel. Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-06 2005-07-07 10:48 ` Ingo Molnar @ 2005-07-07 18:49 ` William Weston 2005-07-07 20:02 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 William Weston 1 sibling, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-07 18:49 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Thu, 7 Jul 2005, Ingo Molnar wrote: > > Still looking into this issue on -51-06. Found something really odd: > > SCHED_NORMAL tasks will start to inherit the priority value of some > > other SCHED_FIFO task. If JACK is started at a given SCHED_FIFO > > priority, X and all of its children will inherit the same priority > > value after login. Other random processes will inherit this, too -- > > sometimes init... > > > > SCHED_NORMAL tasks suddenly inheriting priority values in the range > > normally reserved for SCHED_FIFO could explain at least part of the > > meltdown I've been seeing. Any thoughts? > > is this inheritance perpetual? It is normal for some tasks to be boosted > momentarily, but if the condition remains even after jackd has exited, > it's clearly an anomaly. (lets call it "RT priority leakage".) Priority > leakage on SMP was fixed recently, but there could be other bugs > remaining. Yes, this leakage perpetual. That's what struck me as odd. > There's priority-leakage debugging code in the -RT kernel, which is > activated if you enable CONFIG_DEBUG_PREEMPT. This debugging code, if > triggered, should produce 'BUG: comm/123 leaked RT prio 123 (123)?" type > of messages. But ... due to a bug it would not print out anything but > would lock up hard if it detects the condition (and if > RT_DEADLOCK_DETECT is enabled). I have fixed this reporting bug in the > -51-08 kernel. -51-08 is building now. I'll check out the latest, too. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-07 10:48 ` Ingo Molnar 2005-07-07 18:49 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-06 William Weston @ 2005-07-07 20:02 ` William Weston 2005-07-08 8:03 ` Ingo Molnar 1 sibling, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-07 20:02 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Thu, 7 Jul 2005, Ingo Molnar wrote: > is this inheritance perpetual? It is normal for some tasks to be boosted > momentarily, but if the condition remains even after jackd has exited, > it's clearly an anomaly. (lets call it "RT priority leakage".) Priority > leakage on SMP was fixed recently, but there could be other bugs > remaining. > > There's priority-leakage debugging code in the -RT kernel, which is > activated if you enable CONFIG_DEBUG_PREEMPT. This debugging code, if > triggered, should produce 'BUG: comm/123 leaked RT prio 123 (123)?" type > of messages. But ... due to a bug it would not print out anything but > would lock up hard if it detects the condition (and if > RT_DEADLOCK_DETECT is enabled). I have fixed this reporting bug in the > -51-08 kernel. -51-12 is still exhibhiting the RT priority leakage, but isn't producing any BUG messages. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-07 20:02 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 William Weston @ 2005-07-08 8:03 ` Ingo Molnar 2005-07-08 20:12 ` William Weston 2005-07-08 22:33 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 William Weston 0 siblings, 2 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-08 8:03 UTC (permalink / raw) To: William Weston; +Cc: linux-kernel * William Weston <weston@sysex.net> wrote: > -51-12 is still exhibhiting the RT priority leakage, but isn't > producing any BUG messages. could you check whether the priority leakage happens if you disable SMP? (if you can reproduce it easily) Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-08 8:03 ` Ingo Molnar @ 2005-07-08 20:12 ` William Weston 2005-07-09 21:24 ` Peter Zijlstra 2005-07-08 22:33 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 William Weston 1 sibling, 1 reply; 76+ messages in thread From: William Weston @ 2005-07-08 20:12 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel On Fri, 8 Jul 2005, Ingo Molnar wrote: > could you check whether the priority leakage happens if you disable SMP? > (if you can reproduce it easily) No priority leakages have been seen with UP configs on any of the machines I've been testing. The leakage is not hard to reproduce under SMT: start up jackd from a text vc with an rt prio of 60 (or some unique prio above the IRQ threads), then restart X and login. After several minutes, X and all of its children will be running at whatever prio jackd was started at (but still SCHED_NORMAL). Eventually, init, a handful of SCHED_NORMAL kernel threads, and other random processes are all running at the same priority. When reset to default priority with chrt or schedtool, these processes eventually revert back to the leaked priority level. When jackd is stopped, the priorities stay in their elevated state. If jackd is not started before logging in to X, then the priority of one of the SCHED_FF kernel threads is leaked to other processes in the same manner. --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-08 20:12 ` William Weston @ 2005-07-09 21:24 ` Peter Zijlstra 2005-07-10 11:18 ` Peter Zijlstra 0 siblings, 1 reply; 76+ messages in thread From: Peter Zijlstra @ 2005-07-09 21:24 UTC (permalink / raw) To: William Weston; +Cc: Ingo Molnar, linux-kernel On Fri, 2005-07-08 at 13:12 -0700, William Weston wrote: > On Fri, 8 Jul 2005, Ingo Molnar wrote: > > > could you check whether the priority leakage happens if you disable SMP? > > (if you can reproduce it easily) > > No priority leakages have been seen with UP configs on any of the > machines I've been testing. > > The leakage is not hard to reproduce under SMT: start up jackd from a > text vc with an rt prio of 60 (or some unique prio above the IRQ threads), > then restart X and login. After several minutes, X and all of its > children will be running at whatever prio jackd was started at (but still > SCHED_NORMAL). Eventually, init, a handful of SCHED_NORMAL kernel > threads, and other random processes are all running at the same priority. > When reset to default priority with chrt or schedtool, these processes > eventually revert back to the leaked priority level. When jackd is > stopped, the priorities stay in their elevated state. If jackd is not > started before logging in to X, then the priority of one of the SCHED_FF > kernel threads is leaked to other processes in the same manner. > I can reproduce priority leakage on my SMP system; any userspace process chrt'ed up and a lot will follow. This makes the system very unresponsive when doing a make -j5. Verified on 51-{6,18,23}. -- Peter Zijlstra <a.p.zijlstra@chello.nl> ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-09 21:24 ` Peter Zijlstra @ 2005-07-10 11:18 ` Peter Zijlstra 2005-07-10 15:10 ` Ingo Molnar 0 siblings, 1 reply; 76+ messages in thread From: Peter Zijlstra @ 2005-07-10 11:18 UTC (permalink / raw) To: William Weston; +Cc: Ingo Molnar, linux-kernel On Sat, 2005-07-09 at 23:24 +0200, Peter Zijlstra wrote: > On Fri, 2005-07-08 at 13:12 -0700, William Weston wrote: > > On Fri, 8 Jul 2005, Ingo Molnar wrote: > > > > > could you check whether the priority leakage happens if you disable SMP? > > > (if you can reproduce it easily) > > > > No priority leakages have been seen with UP configs on any of the > > machines I've been testing. > > > > The leakage is not hard to reproduce under SMT: start up jackd from a > > text vc with an rt prio of 60 (or some unique prio above the IRQ threads), > > then restart X and login. After several minutes, X and all of its > > children will be running at whatever prio jackd was started at (but still > > SCHED_NORMAL). Eventually, init, a handful of SCHED_NORMAL kernel > > threads, and other random processes are all running at the same priority. > > When reset to default priority with chrt or schedtool, these processes > > eventually revert back to the leaked priority level. When jackd is > > stopped, the priorities stay in their elevated state. If jackd is not > > started before logging in to X, then the priority of one of the SCHED_FF > > kernel threads is leaked to other processes in the same manner. > > > > I can reproduce priority leakage on my SMP system; any userspace process > chrt'ed up and a lot will follow. This makes the system very > unresponsive when doing a make -j5. Verified on 51-{6,18,23}. > The following patch seems to fix it for me, YMMV. --- kernel/sched.c~ 2005-07-08 10:27:59.000000000 +0200 +++ kernel/sched.c 2005-07-10 13:00:42.000000000 +0200 @@ -780,7 +780,8 @@ static void recalc_task_prio(task_t *p, } } - p->prio = p->normal_prio = effective_prio(p); + p->prio = effective_prio(p); + p->normal_prio = unlikely(rt_prio(p->normal_prio)) ? p->prio : __effective_prio(p); } /* @@ -1415,7 +1416,8 @@ void fastcall wake_up_new_task(task_t * p->sleep_avg = JIFFIES_TO_NS(CURRENT_BONUS(p) * CHILD_PENALTY / 100 * MAX_SLEEP_AVG / MAX_BONUS); - p->prio = p->normal_prio = effective_prio(p); + p->prio = effective_prio(p); + p->normal_prio = unlikely(rt_prio(p->normal_prio)) ? p->prio : __effective_prio(p); if (likely(cpu == this_cpu)) { if (!(clone_flags & CLONE_VM)) { @@ -1427,7 +1429,8 @@ void fastcall wake_up_new_task(task_t * if (unlikely(!current->array)) __activate_task(p, rq); else { - p->prio = p->normal_prio = current->prio; + p->prio = current->prio; + p->normal_prio = current->normal_prio; __activate_task_after(p, current, rq); } set_need_resched(); @@ -2744,7 +2747,8 @@ void scheduler_tick(void) if (!--p->time_slice) { dequeue_task(p, rq->active); set_tsk_need_resched(p); - p->prio = p->normal_prio = effective_prio(p); + p->prio = effective_prio(p); + p->normal_prio = unlikely(rt_prio(p->normal_prio)) ? p->prio : __effective_prio(p); p->time_slice = task_timeslice(p); p->first_time_slice = 0; -- Peter Zijlstra <a.p.zijlstra@chello.nl> ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-10 11:18 ` Peter Zijlstra @ 2005-07-10 15:10 ` Ingo Molnar 2005-07-10 15:43 ` Peter Zijlstra 0 siblings, 1 reply; 76+ messages in thread From: Ingo Molnar @ 2005-07-10 15:10 UTC (permalink / raw) To: Peter Zijlstra; +Cc: William Weston, linux-kernel * Peter Zijlstra <a.p.zijlstra@chello.nl> wrote: > > I can reproduce priority leakage on my SMP system; any userspace process > > chrt'ed up and a lot will follow. This makes the system very > > unresponsive when doing a make -j5. Verified on 51-{6,18,23}. > > > > The following patch seems to fix it for me, YMMV. > > --- kernel/sched.c~ 2005-07-08 10:27:59.000000000 +0200 > +++ kernel/sched.c 2005-07-10 13:00:42.000000000 +0200 > @@ -780,7 +780,8 @@ static void recalc_task_prio(task_t *p, > } > } > > - p->prio = p->normal_prio = effective_prio(p); > + p->prio = effective_prio(p); > + p->normal_prio = unlikely(rt_prio(p->normal_prio)) ? p->prio : __effective_prio(p); > } ahh, indeed, this code did not take boosting into account. Good catch! I'm wondering why this only showed up on SMP. I've fixed it a bit differently in my tree, by making the roles of the various priority fields and functions more obvious, see the delta patch below. I've also released the -51-23 patch with these changes included. Does this fix priority leakage on your SMP system? Ingo Index: linux/include/linux/sched.h =================================================================== --- linux.orig/include/linux/sched.h +++ linux/include/linux/sched.h @@ -1035,7 +1035,7 @@ extern int idle_cpu(int cpu); extern int sched_setscheduler(struct task_struct *, int, struct sched_param *); extern task_t *idle_task(int cpu); extern void mutex_setprio(task_t *p, int prio); -extern int mutex_getprio(task_t *p); +extern int normal_prio(task_t *p); void yield(void); void __yield(void); Index: linux/kernel/rt.c =================================================================== --- linux.orig/kernel/rt.c +++ linux/kernel/rt.c @@ -720,7 +720,7 @@ static void pi_setprio(struct rt_mutex * #ifdef CONFIG_RT_DEADLOCK_DETECT pi_prio++; - if (p->policy != SCHED_NORMAL && prio > mutex_getprio(p)) { + if (p->policy != SCHED_NORMAL && prio > normal_prio(p)) { TRACE_OFF(); printk("huh? (%d->%d??)\n", p->prio, prio); Index: linux/kernel/sched.c =================================================================== --- linux.orig/kernel/sched.c +++ linux/kernel/sched.c @@ -645,7 +645,7 @@ static inline void enqueue_task_head(str } /* - * effective_prio - return the priority that is based on the static + * __normal_prio - return the priority that is based on the static * priority but is modified by bonuses/penalties. * * We scale the actual sleep average [0 .... MAX_SLEEP_AVG] @@ -659,7 +659,7 @@ static inline void enqueue_task_head(str * Both properties are important to certain workloads. */ -static inline int __effective_prio(task_t *p) +static inline int __normal_prio(task_t *p) { int bonus, prio; @@ -670,57 +670,53 @@ static inline int __effective_prio(task_ prio = MAX_RT_PRIO; if (prio > MAX_PRIO-1) prio = MAX_PRIO-1; - return prio; -} -static int effective_prio(task_t *p) -{ - if (rt_task(p)) - return p->prio; - return __effective_prio(p); -} - -static inline void trace_start_sched_wakeup(task_t *p, runqueue_t *rq) -{ - if (TASK_PREEMPTS_CURR(p, rq) && (p != rq->curr)) - __trace_start_sched_wakeup(p); + return prio; } /* - * __activate_task - move a task to the runqueue. + * Calculate the expected normal priority: i.e. priority + * without taking RT-inheritance into account. Might be + * boosted by interactivity modifiers. Changes upon fork, + * setprio syscalls, and whenever the interactivity + * estimator recalculates. */ -static inline void __activate_task(task_t *p, runqueue_t *rq) +inline int normal_prio(task_t *p) { - trace_special_pid(p->pid, p->prio, rq->nr_running); - enqueue_task(p, rq->active); - rq->nr_running++; + int prio; + + if (p->policy != SCHED_NORMAL) + prio = MAX_RT_PRIO-1 - p->rt_priority; + else + prio = __normal_prio(p); + + trace_special_pid(p->pid, p->prio, prio); + return prio; } /* - * __activate_task_after - move a task to the runqueue, - * to execute after a specific task. + * Calculate the current priority, i.e. the priority + * taken into account by the scheduler. This value might + * be boosted by RT tasks, or might be boosted by + * interactivity modifiers. Will be RT if the task got + * RT-boosted. If not then it returns p->normal_prio. */ -static inline -void __activate_task_after(task_t *p, task_t *parent, runqueue_t *rq) +static void __recalc_task_prio(task_t *p) { - // FIXME: to head rather? - list_add_tail(&p->run_list, &parent->run_list); - p->array = parent->array; - p->array->nr_active++; - rq->nr_running++; - inc_rt_tasks(p, rq); + p->normal_prio = normal_prio(p); + /* + * If we are RT tasks or we were boosted to RT priority, + * keep the priority unchanged. Otherwise, update priority + * to the normal priority: + */ + if (!rt_prio(p->prio)) + p->prio = p->normal_prio; } /* - * __activate_idle_task - move idle task to the _front_ of runqueue. + * Recalculate p->normal_prio and p->prio after having slept, + * updating the sleep-average too: */ -static inline void __activate_idle_task(task_t *p, runqueue_t *rq) -{ - enqueue_task_head(p, rq->active); - rq->nr_running++; - WARN_ON(rt_task(p)); -} - static void recalc_task_prio(task_t *p, unsigned long long now) { /* Caller must always ensure 'now >= p->timestamp' */ @@ -780,7 +776,48 @@ static void recalc_task_prio(task_t *p, } } - p->prio = p->normal_prio = effective_prio(p); + __recalc_task_prio(p); +} + +static inline void trace_start_sched_wakeup(task_t *p, runqueue_t *rq) +{ + if (TASK_PREEMPTS_CURR(p, rq) && (p != rq->curr)) + __trace_start_sched_wakeup(p); +} + +/* + * __activate_task - move a task to the runqueue. + */ +static inline void __activate_task(task_t *p, runqueue_t *rq) +{ + trace_special_pid(p->pid, p->prio, rq->nr_running); + enqueue_task(p, rq->active); + rq->nr_running++; +} + +/* + * __activate_task_after - move a task to the runqueue, + * to execute after a specific task. + */ +static inline +void __activate_task_after(task_t *p, task_t *parent, runqueue_t *rq) +{ + // FIXME: to head rather? + list_add_tail(&p->run_list, &parent->run_list); + p->array = parent->array; + p->array->nr_active++; + rq->nr_running++; + inc_rt_tasks(p, rq); +} + +/* + * __activate_idle_task - move idle task to the _front_ of runqueue. + */ +static inline void __activate_idle_task(task_t *p, runqueue_t *rq) +{ + enqueue_task_head(p, rq->active); + rq->nr_running++; + WARN_ON(rt_task(p)); } /* @@ -1415,7 +1452,7 @@ void fastcall wake_up_new_task(task_t * p->sleep_avg = JIFFIES_TO_NS(CURRENT_BONUS(p) * CHILD_PENALTY / 100 * MAX_SLEEP_AVG / MAX_BONUS); - p->prio = p->normal_prio = effective_prio(p); + __recalc_task_prio(p); if (likely(cpu == this_cpu)) { if (!(clone_flags & CLONE_VM)) { @@ -1427,7 +1464,8 @@ void fastcall wake_up_new_task(task_t * if (unlikely(!current->array)) __activate_task(p, rq); else { - p->prio = p->normal_prio = current->prio; + p->prio = current->prio; + p->normal_prio = current->normal_prio; __activate_task_after(p, current, rq); } set_need_resched(); @@ -2730,6 +2768,10 @@ void scheduler_tick(void) /* * RR tasks need a special form of timeslice management. * FIFO tasks have no timeslices. + * + * On PREEMPT_RT, boosted tasks will also get into this + * branch and wont get their timeslice decreased until + * they have done their work. */ if ((p->policy == SCHED_RR) && !--p->time_slice) { p->time_slice = task_timeslice(p); @@ -2744,7 +2786,7 @@ void scheduler_tick(void) if (!--p->time_slice) { dequeue_task(p, rq->active); set_tsk_need_resched(p); - p->prio = p->normal_prio = effective_prio(p); + __recalc_task_prio(p); p->time_slice = task_timeslice(p); p->first_time_slice = 0; @@ -3682,7 +3724,7 @@ void set_user_nice(task_t *p, long nice) new_prio = NICE_TO_PRIO(nice); delta = new_prio - old_prio; p->static_prio = NICE_TO_PRIO(nice); - p->prio += delta; + __recalc_task_prio(p); if (array) { enqueue_task(p, array); @@ -3712,18 +3754,6 @@ int can_nice(const task_t *p, const int capable(CAP_SYS_NICE)); } -int mutex_getprio(task_t *p) -{ - int prio; - - if (p->policy != SCHED_NORMAL) - prio = MAX_RT_PRIO-1 - p->rt_priority; - else - prio = __effective_prio(p); - trace_special_pid(p->pid, p->prio, prio); - return prio; -} - /* * Used by the PREEMPT_RT code to implement * priority inheritance logic: @@ -3880,10 +3910,7 @@ static void __setscheduler(struct task_s BUG_ON(p->array); p->policy = policy; p->rt_priority = prio; - if (policy != SCHED_NORMAL) - p->prio = p->normal_prio = MAX_RT_PRIO-1 - p->rt_priority; - else - p->prio = p->normal_prio = p->static_prio; + __recalc_task_prio(p); } /** ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-10 15:10 ` Ingo Molnar @ 2005-07-10 15:43 ` Peter Zijlstra 2005-07-10 16:12 ` Peter Zijlstra 0 siblings, 1 reply; 76+ messages in thread From: Peter Zijlstra @ 2005-07-10 15:43 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel On Sun, 2005-07-10 at 17:10 +0200, Ingo Molnar wrote: > * Peter Zijlstra <a.p.zijlstra@chello.nl> wrote: > > > > I can reproduce priority leakage on my SMP system; any userspace process > > > chrt'ed up and a lot will follow. This makes the system very > > > unresponsive when doing a make -j5. Verified on 51-{6,18,23}. > > > > > > > The following patch seems to fix it for me, YMMV. > > > > --- kernel/sched.c~ 2005-07-08 10:27:59.000000000 +0200 > > +++ kernel/sched.c 2005-07-10 13:00:42.000000000 +0200 > > @@ -780,7 +780,8 @@ static void recalc_task_prio(task_t *p, > > } > > } > > > > - p->prio = p->normal_prio = effective_prio(p); > > + p->prio = effective_prio(p); > > + p->normal_prio = unlikely(rt_prio(p->normal_prio)) ? p->prio : __effective_prio(p); > > } > > ahh, indeed, this code did not take boosting into account. Good catch! > I'm wondering why this only showed up on SMP. I was thinking because of the agressive RT rebalance code this codepath is more exercised on SMP systems. > I've fixed it a bit > differently in my tree, by making the roles of the various priority > fields and functions more obvious, see the delta patch below. Yes, much nicer :-) > I've also > released the -51-23 patch with these changes included. Does this fix > priority leakage on your SMP system? > -51-24 right? I'll give it a spin. -- Peter Zijlstra <a.p.zijlstra@chello.nl> ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-10 15:43 ` Peter Zijlstra @ 2005-07-10 16:12 ` Peter Zijlstra 2005-07-10 18:26 ` Ingo Molnar 2005-07-11 18:49 ` William Weston 0 siblings, 2 replies; 76+ messages in thread From: Peter Zijlstra @ 2005-07-10 16:12 UTC (permalink / raw) To: Ingo Molnar; +Cc: William Weston, linux-kernel On Sun, 2005-07-10 at 17:43 +0200, Peter Zijlstra wrote: > > I've also > > released the -51-23 patch with these changes included. Does this fix > > priority leakage on your SMP system? > > > > -51-24 right? I'll give it a spin. > a quick test seems to indicate it is indeed solved. -- Peter Zijlstra <a.p.zijlstra@chello.nl> ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-10 16:12 ` Peter Zijlstra @ 2005-07-10 18:26 ` Ingo Molnar 2005-07-11 18:49 ` William Weston 1 sibling, 0 replies; 76+ messages in thread From: Ingo Molnar @ 2005-07-10 18:26 UTC (permalink / raw) To: Peter Zijlstra; +Cc: William Weston, linux-kernel * Peter Zijlstra <a.p.zijlstra@chello.nl> wrote: > On Sun, 2005-07-10 at 17:43 +0200, Peter Zijlstra wrote: > > > > I've also > > > released the -51-23 patch with these changes included. Does this fix > > > priority leakage on your SMP system? > > > > > > > -51-24 right? I'll give it a spin. > > > > a quick test seems to indicate it is indeed solved. great! Ingo ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 2005-07-10 16:12 ` Peter Zijlstra 2005-07-10 18:26 ` Ingo Molnar @ 2005-07-11 18:49 ` William Weston 1 sibling, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-11 18:49 UTC (permalink / raw) To: Peter Zijlstra; +Cc: Ingo Molnar, linux-kernel On Sun, 10 Jul 2005, Peter Zijlstra wrote: > On Sun, 2005-07-10 at 17:43 +0200, Peter Zijlstra wrote: > > > > I've also > > > released the -51-23 patch with these changes included. Does this fix > > > priority leakage on your SMP system? > > > > > > > -51-24 right? I'll give it a spin. > > > > a quick test seems to indicate it is indeed solved. Running -51-26. Priority leakage on the SMT box is gone. This indeed fixes the scheduler meltdown issue I was dealing with. Great work, guys! --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
* Re: Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 2005-07-08 8:03 ` Ingo Molnar 2005-07-08 20:12 ` William Weston @ 2005-07-08 22:33 ` William Weston 1 sibling, 0 replies; 76+ messages in thread From: William Weston @ 2005-07-08 22:33 UTC (permalink / raw) To: Ingo Molnar; +Cc: linux-kernel Hi Ingo, More SMT weirdness. Latency traces aren't looking quite right on -51-17 with my SMT debug config. Started jackd, immediately after boot (before logging in to X), then: [root@manzanita ~]# cat /proc/latency_trace cat/2149[CPU#1]: BUG in update_out_trace at kernel/latency.c:698 [<c01041a3>] dump_stack+0x23/0x30 (20) [<c012317b>] __WARN_ON+0x6b/0x90 (52) [<c0142237>] update_out_trace+0x467/0x4e0 (100) [<c0142590>] l_start+0x2e0/0x310 (72) [<c019415b>] seq_read+0x7b/0x2d0 (60) [<c0170f74>] vfs_read+0xd4/0x140 (36) [<c0171250>] sys_read+0x50/0x80 (44) [<c01031b0>] sysenter_past_esp+0x61/0x89 (-8116) --------------------------- | preempt count: 00000002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [<c01434dc>] .... add_preempt_count+0x1c/0x20 .....[<c0123137>] .. ( <= __WARN_ON+0x27/0x90) .. [<c01434dc>] .... add_preempt_count+0x1c/0x20 .....[<c014454d>] .. ( <= print_traces+0x1d/0x60) ------------------------------ | showing all locks held by: | (cat/2149 [dda0e030, 39]): ------------------------------ #001: [db76afa4] {(struct semaphore *)(&p->sem)} ... acquired at: seq_read+0x2b/0x2d0 #002: [c05fe424] {out_mutex.lock} ... acquired at: l_start+0x1b/0x310 #003: [c06d4404] {max_mutex.lock} ... acquired at: l_start+0x2db/0x310 CPU0: 0000003e0d590d0c (0000003e0d593ba8) ... #50 (0000003e0d610158) 0000003e0d611a78 CPU1: 0000003e0d6122c8 (0000003e0d6125b0) ... #4 (0000003e0d612dac) 0000003e0d6136bc CPU0 entries: 50 first stamp: 0000003e0d6122c8 last stamp: 0000003e0d6122c8 preemption latency trace v1.1.4 on 2.6.12-RT-V0.7.51-17-debug -------------------------------------------------------------------- latency: 159 us, #54/54, CPU#0 | (M:rt VP:0, KP:1, SP:1 HP:1 #P:2) ----------------- | task: softirq-timer/0-4 (uid:0 nice:-10 policy:0 rt_prio:0) ----------------- _------=> CPU# / _-----=> irqs-off | / _----=> need-resched || / _---=> hardirq/softirq ||| / _--=> preempt-depth |||| / ||||| delay cmd pid ||||| time | caller \ / ||||| \ | / vim:ft=help After logging into X: <15:26:38 root@manzanita ~># cat /proc/latency_trace preemption latency trace v1.1.4 on 2.6.12-RT-V0.7.51-17-debug -------------------------------------------------------------------- latency: 202 us, #102/102, CPU#0 | (M:rt VP:0, KP:1, SP:1 HP:1 #P:2) ----------------- | task: jackd-1929 (uid:500 nice:0 policy:1 rt_prio:60) ----------------- _------=> CPU# / _-----=> irqs-off | / _----=> need-resched || / _---=> hardirq/softirq ||| / _--=> preempt-depth |||| / ||||| delay cmd pid ||||| time | caller \ / ||||| \ | / <...>-1817 1.... 0us : set_palette (redraw_screen) IRQ 5-724 0Dnh2 0us : find_next_bit (pull_rt_tasks) <...>-1817 1.... 0us : is_console_locked (set_palette) IRQ 5-724 0Dnh2 0us : dependent_sleeper (__schedule) <...>-1817 1.... 0us : vgacon_set_palette (set_palette) IRQ 5-724 0Dnh2 1us : _raw_spin_unlock (dependent_sleeper) <...>-1817 1.... 1us : vga_set_palette (vgacon_set_palette) IRQ 5-724 0Dnh1 1us : preempt_schedule (dependent_sleeper) IRQ 5-724 0Dnh1 1us : _raw_spin_lock (dependent_sleeper) IRQ 5-724 0Dnh2 2us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh2 2us : _raw_spin_lock (dependent_sleeper) IRQ 5-724 0Dnh3 2us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh3 3us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh3 3us : _raw_spin_unlock (dependent_sleeper) IRQ 5-724 0Dnh2 3us : preempt_schedule (dependent_sleeper) IRQ 5-724 0Dnh2 3us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh2 4us : trace_array (__schedule) IRQ 5-724 0Dnh2 4us : trace_array <<...>-1929> (27 27) IRQ 5-724 0Dnh2 5us : trace_array <IRQ 5-724> (31 31) IRQ 5-724 0Dnh2 6us : trace_array (__schedule) <...>-1929 0Dnh2 8us+: __switch_to (__schedule) <...>-1929 0Dnh2 11us : __schedule <IRQ 5-724> (31 27) <...>-1929 0Dnh2 11us : finish_task_switch (__schedule) <...>-1929 0Dnh2 13us : smp_send_reschedule_allbutself (finish_task_switch) <...>-1929 0Dnh2 13us : __bitmap_weight (smp_send_reschedule_allbutself) <...>-1929 0Dnh2 13us : __send_IPI_shortcut (smp_send_reschedule_allbutself) <...>-1929 0Dnh2 13us : _raw_spin_unlock (finish_task_switch) <...>-1929 0Dnh1 15us : trace_stop_sched_switched (finish_task_switch) <...>-1929 0Dnh1 15us : _raw_spin_lock (trace_stop_sched_switched) <...>-1929 0Dnh2 15us : trace_stop_sched_switched <<...>-1929> (27 0) <...>-1817 1Dnh. 15us : smp_reschedule_interrupt (c026c17a 0 0) <...>-1929 0Dnh3 16us : _raw_spin_unlock (trace_stop_sched_switched) <...>-1817 1Dnh. 16us : preempt_schedule_irq (need_resched) <...>-1929 0Dnh2 16us!: trace_stop_sched_switched (finish_task_switch) IRQ 5-724 0Dnh2 0us : find_next_bit (pull_rt_tasks) IRQ 5-724 0Dnh2 0us : dependent_sleeper (__schedule) IRQ 5-724 0Dnh2 1us : _raw_spin_unlock (dependent_sleeper) IRQ 5-724 0Dnh1 1us : preempt_schedule (dependent_sleeper) IRQ 5-724 0Dnh1 1us : _raw_spin_lock (dependent_sleeper) IRQ 5-724 0Dnh2 2us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh2 2us : _raw_spin_lock (dependent_sleeper) IRQ 5-724 0Dnh3 2us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh3 3us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh3 3us : _raw_spin_unlock (dependent_sleeper) IRQ 5-724 0Dnh2 3us : preempt_schedule (dependent_sleeper) IRQ 5-724 0Dnh2 3us : find_next_bit (dependent_sleeper) IRQ 5-724 0Dnh2 4us : trace_array (__schedule) IRQ 5-724 0Dnh2 4us : trace_array <<...>-1929> (27 27) IRQ 5-724 0Dnh2 5us : trace_array <IRQ 5-724> (31 31) IRQ 5-724 0Dnh2 6us : trace_array (__schedule) <...>-1929 0Dnh2 8us+: __switch_to (__schedule) <...>-1929 0Dnh2 11us : __schedule <IRQ 5-724> (31 27) <...>-1929 0Dnh2 11us : finish_task_switch (__schedule) <...>-1929 0Dnh2 13us : smp_send_reschedule_allbutself (finish_task_switch) <...>-1929 0Dnh2 13us : __bitmap_weight (smp_send_reschedule_allbutself) <...>-1929 0Dnh2 13us : __send_IPI_shortcut (smp_send_reschedule_allbutself) <...>-1929 0Dnh2 13us : _raw_spin_unlock (finish_task_switch) <...>-1929 0Dnh1 15us : trace_stop_sched_switched (finish_task_switch) <...>-1929 0Dnh1 15us : _raw_spin_lock (trace_stop_sched_switched) <...>-1929 0Dnh2 15us : trace_stop_sched_switched <<...>-1929> (27 0) <...>-1929 0Dnh3 16us : _raw_spin_unlock (trace_stop_sched_switched) <...>-1929 0Dnh2 16us!: trace_stop_sched_switched (finish_task_switch) vim:ft=help --ww ^ permalink raw reply [flat|nested] 76+ messages in thread
end of thread, other threads:[~2005-07-13 17:23 UTC | newest] Thread overview: 76+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2005-06-28 17:27 Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Karsten Wiese 2005-06-28 20:21 ` Ingo Molnar 2005-06-28 20:30 ` Ingo Molnar 2005-06-28 23:51 ` Karsten Wiese 2005-06-29 6:34 ` Ingo Molnar 2005-06-29 7:00 ` Ingo Molnar 2005-06-29 9:15 ` William Weston 2005-06-29 12:56 ` Ingo Molnar 2005-06-30 1:50 ` William Weston 2005-06-29 14:48 ` Karsten Wiese 2005-06-29 19:38 ` Ingo Molnar 2005-06-29 23:36 ` Karsten Wiese 2005-06-30 17:52 ` Karsten Wiese 2005-06-30 19:46 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-37 William Weston 2005-06-30 19:52 ` Ingo Molnar 2005-06-30 20:48 ` Ingo Molnar 2005-06-30 22:17 ` Gene Heskett 2005-06-30 22:15 ` Gene Heskett 2005-06-30 20:38 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-38 Michal Schmidt 2005-06-30 20:47 ` Ingo Molnar 2005-06-30 20:50 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 Ingo Molnar 2005-06-30 22:27 ` Karsten Wiese 2005-06-30 22:59 ` William Weston 2005-07-01 1:00 ` William Weston 2005-07-01 0:15 ` Karsten Wiese 2005-07-01 7:18 ` Ingo Molnar 2005-07-01 19:34 ` Chuck Harding 2005-07-02 1:46 ` William Weston 2005-07-02 2:02 ` Lee Revell 2005-07-04 8:53 ` William Weston 2005-07-04 11:16 ` Ingo Molnar 2005-07-13 1:18 ` Lee Revell 2005-07-13 1:28 ` Lee Revell 2005-07-13 6:35 ` Ingo Molnar 2005-07-13 17:20 ` Lee Revell 2005-07-03 14:04 ` Ingo Molnar 2005-07-03 18:12 ` Ingo Molnar 2005-07-05 20:43 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-45 William Weston 2005-07-06 10:04 ` Ingo Molnar 2005-07-06 21:12 ` William Weston 2005-07-07 14:56 ` Steven Rostedt 2005-07-07 15:31 ` Ingo Molnar 2005-07-07 15:42 ` Steven Rostedt 2005-07-07 16:10 ` Steven Rostedt 2005-07-07 16:48 ` Ingo Molnar 2005-07-08 7:02 ` Steven Rostedt 2005-07-08 9:54 ` Steven Rostedt 2005-07-08 10:00 ` Ingo Molnar 2005-07-08 20:55 ` Bill Davidsen 2005-07-07 17:51 ` Ingo Molnar 2005-07-07 21:18 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 Chuck Harding 2005-07-08 5:41 ` Ingo Molnar 2005-07-08 8:00 ` Ingo Molnar 2005-07-08 18:06 ` Chuck Harding 2005-07-07 19:05 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-08 William Weston 2005-07-07 19:18 ` Ingo Molnar 2005-07-07 19:46 ` William Weston 2005-07-07 19:52 ` Ingo Molnar 2005-07-08 9:23 ` Ingo Molnar 2005-07-08 9:29 ` Ingo Molnar 2005-07-08 9:45 ` Ingo Molnar 2005-07-08 20:30 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 William Weston 2005-07-07 1:26 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.50-24 William Weston 2005-07-07 10:48 ` Ingo Molnar 2005-07-07 18:49 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-06 William Weston 2005-07-07 20:02 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-12 William Weston 2005-07-08 8:03 ` Ingo Molnar 2005-07-08 20:12 ` William Weston 2005-07-09 21:24 ` Peter Zijlstra 2005-07-10 11:18 ` Peter Zijlstra 2005-07-10 15:10 ` Ingo Molnar 2005-07-10 15:43 ` Peter Zijlstra 2005-07-10 16:12 ` Peter Zijlstra 2005-07-10 18:26 ` Ingo Molnar 2005-07-11 18:49 ` William Weston 2005-07-08 22:33 ` Real-Time Preemption, -RT-2.6.12-final-V0.7.51-17 William Weston
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).